Pages: 1 2 [3] 4   Go Down

Author Topic: DxO Mark  (Read 33348 times)

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
DxO Mark
« Reply #40 on: November 22, 2008, 10:02:45 pm »

Hi,

There are amplifiers in the signal path. The Sony chip has on chip ADCs but Nikon normally bypasses these and uses a 14 bit external ADC instead. Even the chip itself could be better as the D90 is a later model than.

I don't think that DXO is a bogus site just because you or I don't understand their methodology or don't agree with their results. On the contrary, I really appreciate that their share their results.

Best regards
Erik

Quote from: Tony Beach
There is something seriously wrong with the conclusions this website offers.  I was reading a post at another photography forum that referred to the D300 ISO performance on this page:  http://www.dxomark.com/index.php/eng/DxOMark-Sensor and when I went there I noticed that the D90 ISO performance (which uses the same sensor) was rated about half a stop better.  I believe people relying on this sort of information in making a DSLR purchase are going to be seriously mislead as there is simply no way the D90 is a half stop better than the D300, and this makes me highly suspicious of all of the charts offered by this website.  To put it bluntly, this site as it currently stands is simply bogus.
Logged
Erik Kaffehr
 

Tony Beach

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 452
    • http://imageevent.com/tonybeach/twelveimages
DxO Mark
« Reply #41 on: November 22, 2008, 10:37:43 pm »

Quote from: ErikKaffehr
Hi,

There are amplifiers in the signal path. The Sony chip has on chip ADCs but Nikon normally bypasses these and uses a 14 bit external ADC instead. Even the chip itself could be better as the D90 is a later model than.

I don't think that DXO is a bogus site just because you or I don't understand their methodology or don't agree with their results. On the contrary, I really appreciate that their share their results.

Best regards
Erik

The comparison is to the D90, another Nikon camera using the same sensor and it only records 12 bits whereas the D300 can record 14 bits; I believe both use a 16 bit pipeline, I know the D300 does.  To quote the relevant portion (for the purposes of this discussion) of Thom Hogan's review of the D90:

"...for the main types of noise and high ISO values, the D90 is essentially a mimic of the D300....Go read what I wrote in my D300 review about noise handling; the D90 is nearly identical."

That sure doesn't look like there's a half stop difference to me -- perhaps Thom Hogan just doesn't see the differences that DXOMark is asserting are there.

These geeky, technical analysis' of cameras are already flawed even if they were accurate:  http://theonlinephotographer.typepad.com/t...l-vindicat.html.  However, the data presented by DXOMark is flawed and that makes it particularly egregious to offer it to for the purpose of "informing" some unwitting person who wants to buy the "better" DSLR.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DxO Mark
« Reply #42 on: November 23, 2008, 12:28:54 am »

Quote from: Tony Beach
I noticed that the D90 ISO performance (which uses the same sensor) was rated about half a stop better

A note without any further qualification/comparison: the sensors of the D90 and D300 are not identical. Although the overall dimension in pixels is the same, the masked areas are different as are the spectral responses.

This does not preclude, that the the sensors' electronics are the same. On the other hand, the different noise characteristics speak against the "same sensor" even regarding the electronics.
Logged
Gabor

Tony Beach

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 452
    • http://imageevent.com/tonybeach/twelveimages
DxO Mark
« Reply #43 on: November 23, 2008, 10:12:45 am »

Quote from: Panopeeper
This does not preclude, that the the sensors' electronics are the same. On the other hand, the different noise characteristics speak against the "same sensor" even regarding the electronics.

Okay, you prefer to believe some numbers presented by DXOMark over what Thom Hogan observes.  It appears to me that in the real world (the world based on knowledgeable photographer's experiences with the cameras) the D90 and D300 have very nearly identical image quality; only in fantasy land does the D90 have noticeably superior image quality.
Logged

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
DxO Mark
« Reply #44 on: November 23, 2008, 02:29:43 pm »

Quote from: Tony Beach
Okay, you prefer to believe some numbers presented by DXOMark over what Thom Hogan observes.  It appears to me that in the real world (the world based on knowledgeable photographer's experiences with the cameras) the D90 and D300 have very nearly identical image quality; only in fantasy land does the D90 have noticeably superior image quality.

For instance, in the fantasy land that is the DPR Forums  
http://forums.dpreview.com/forums/read.asp...essage=30005750
Logged
emil

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DxO Mark
« Reply #45 on: November 23, 2008, 02:39:29 pm »

Quote from: Tony Beach
Okay, you prefer to believe some numbers presented by DXOMark over what Thom Hogan observes

I prefer to believe my own measurements over what Thom Hogan observes. I came to said result and posted it much before this DXOMark comparison appeared.

It has to be noted, that my measurements relate to the noise at a given ISO, but that is only half of the subject; the other half is, how correct the nominal ISO settings are. When shooting in low-light situation, it does not solve the problem that one camera has much less noise @ ISO 1600 than the other, if the first camera requires one stop higher ISO in the same setting. (Of course, one full stop difference would be quite extreme). I can not measure this; I am using raw files made by others, mostly by Imaging Resources. I guess DXOMark publishes such a comparison as well.

On the other hand, from the point of maximum dynamic range, only the noise counts.

Anyway, the possible incorrectness of the nominal ISO values does not mitigate the fact, that the sensors of D90 and D300 are not the same; in fact they are very different.

Samples of my measurements are for example in D90 vs D300, a layered TIFF.
Logged
Gabor

Tony Beach

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 452
    • http://imageevent.com/tonybeach/twelveimages
DxO Mark
« Reply #46 on: November 23, 2008, 04:58:50 pm »

I see that you both are referring to comparison files from Imaging Resources.  First thing I notice about these files is that they are exposed differently; for instance, at ISO 800 the D300 is exposed at 1/125 while the D90 is exposed at 1/100, that's a 1/3 of stop difference.  I suspect that I do not shoot the same way Imaging Resources shoots, I utilize uni-WB to try to achieve a maximum EV and then adjust EV with the RAW converter.  Nonetheless, despite the 1/3 EV difference, looking at the ISO 800 images I see practically nothing to distinguish them from each other -- so digging around in shadows to isolate differences is an exercise in pixel peeping that does nothing to elucidate which camera would be a better choice for an individual photographer -- as I stated earlier in this thread, all of these cameras will deliver more than adequate files for the vast majority of photographers under the vast majority of applications.

I wish all of you enjoyment in your pursuit of whatever it is you are searching for (my suspicion is that real photography is not the goal of these pointless exercises).  I just hope that people looking for a "better" camera choice are not swayed by these flawed analysis'.
Logged

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
DxO Mark
« Reply #47 on: November 23, 2008, 06:01:41 pm »

Quote from: Tony Beach
I see that you both are referring to comparison files from Imaging Resources.  First thing I notice about these files is that they are exposed differently; for instance, at ISO 800 the D300 is exposed at 1/125 while the D90 is exposed at 1/100, that's a 1/3 of stop difference.  I suspect that I do not shoot the same way Imaging Resources shoots, I utilize uni-WB to try to achieve a maximum EV and then adjust EV with the RAW converter.  Nonetheless, despite the 1/3 EV difference, looking at the ISO 800 images I see practically nothing to distinguish them from each other -- so digging around in shadows to isolate differences is an exercise in pixel peeping that does nothing to elucidate which camera would be a better choice for an individual photographer -- as I stated earlier in this thread, all of these cameras will deliver more than adequate files for the vast majority of photographers under the vast majority of applications.

I wish all of you enjoyment in your pursuit of whatever it is you are searching for (my suspicion is that real photography is not the goal of these pointless exercises).  I just hope that people looking for a "better" camera choice are not swayed by these flawed analysis'.

I'm happy for you that you don't need to push the performance of your cameras to the limit.  BTW, uni-WB does nothing to the RAW file.  It may help you to set the exposure if you're not in the habit of metering the scene before you shoot, by checking the histogram after you shoot.  

There is a difference, it's quite obvious in the IR images, and much more than the 1/3 stop difference in exposure.  Agreed though that not all people need the difference, any of the cameras in the thread I linked is more than capable of making great images at low to moderate ISO.
« Last Edit: November 23, 2008, 06:07:12 pm by ejmartin »
Logged
emil

Tony Beach

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 452
    • http://imageevent.com/tonybeach/twelveimages
DxO Mark
« Reply #48 on: November 23, 2008, 06:25:07 pm »

Quote from: ejmartin
I'm happy for you that you don't need to push the performance of your cameras to the limit.

Nonsense.  I do push my cameras to their performance limits, and that is more than what I typically need.  How do you find the current cameras lacking?

Quote
BTW, uni-WB does nothing to the RAW file.  It may help you to set the exposure if you're not in the habit of metering the scene before you shoot, by checking the histogram after you shoot.

It definitely helps, it is essential.  What's more, other settings are involved too, including most importantly brightness.   I find that the true native ISO of the D300 is closer to ISO 125 compared to what the D200 does.

Quote
There is a difference, it's quite obvious in the IR images, and much more than the 1/3 stop difference in exposure.

Laughable.  I have an IR converted D200 and wouldn't waste my time doing IR shots with a CMOS sensor camera.

Perhaps you meant Imaging Resource, well I find their shooting conditions to be uncontrolled as demonstrated by using different EVs, different focal lengths, and who knows what else is different.  With so many differences, it's difficult to determine what the sensor's actual differences are.

Quote
Agreed though that not all people need the difference, any of the cameras in the thread I linked is more than capable of making great images at low to moderate ISO.

That's my point, for the vast majority of users all of these cameras are more then up to the job.
« Last Edit: November 23, 2008, 06:26:05 pm by Tony Beach »
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DxO Mark
« Reply #49 on: November 23, 2008, 09:50:18 pm »

Quote from: Tony Beach
I see that you both are referring to comparison files from Imaging Resources.  First thing I notice about these files is that they are exposed differently; for instance, at ISO 800 the D300 is exposed at 1/125 while the D90 is exposed at 1/100, that's a 1/3 of stop difference

This is totally irrelevant regarding the noise, as is the lens, the distance, the subject, the illumination. ("Calibrating" the ISO is a different issue.)

Quote
despite the 1/3 EV difference, looking at the ISO 800 images I see practically nothing to distinguish them from each other -- so digging around in shadows to isolate differences is an exercise in pixel peeping that does nothing to elucidate which camera would be a better choice for an individual photographer -- as I stated earlier in this thread, all of these cameras will deliver more than adequate files for the vast majority of photographers under the vast majority of applications

I'm afraid you are not in the position to judge this; you did not ask the "vast majority" of photographers, nor did I. You are suggesting, that noise and dynamic range are non-issues with these cameras; I don't doubt that this is true regarding your photography, but looking around on several forums I am convinced, that this can not be said generally.

People are shooting in low light like in concerts or with very short shutter time without professional lighting like indoor sports in a high school gym, thus they are forced to go with high ISO. Some of the shots are still strongly underexposed; some areas can be in the seventh, eighths or ninths EV (from saturation), and in that region the noise is unavoidable with today's DSLRs (at ISO 1600 and higher), the question is only how bad. These nonsensical looking tests are analysing just that: how is the noise in the very deep shadows.

The other side of the same coin is the dynamic range. I am very often in situations, that highlights are already clipping but other areas of the image are far underexposed @ ISO 100 or 200. How much I can reclaim from the shadows while showing the details is limited by the noise. I can determine with my measurements the difference between two sensors in stops at a given noise level.
Logged
Gabor

Tony Beach

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 452
    • http://imageevent.com/tonybeach/twelveimages
DxO Mark
« Reply #50 on: November 24, 2008, 12:28:42 am »

Quote from: Panopeeper
You are suggesting, that noise and dynamic range are non-issues with these cameras; I don't doubt that this is true regarding your photography, but looking around on several forums I am convinced, that this can not be said generally.

There is not enough difference between a D90 and a D300 to get excited about.  That's the word I got back from Thom Hogan tonight when I asked him directly about this.  If you want to shoot in low light, then a D700 or D3 is your tool of choice, and anyone who thinks they will get noticeably better performance from a D90 than a D300 is misinformed.  For some, that misinformation will come from DXO Mark, Imaging Resource, and even this forum (at least this thread).  Arguing that someone who is not getting results they are satisfied with at ISO 1600 with a D300 would be satisfied with a D90 instead is setting them up for disappointment; the same is true at base ISO as it pertains to DR.

Regarding the issues of not using the same lens, the same focal length, or the same EV; that highlights the lack of rigorous control at Imaging Resource and which is manifested by the 1/3 of a stop discrepancy in their comparative files.  If you do not see that as a flaw than you are simply not paying attention.  If I increase the illumination of the scene and then set the D300 to expose at 1/100 instead of 1/125 (just as the D90 was,), then the already negligible differences are practically erased.



Can you see a difference in the lighting between these two?  This probably explains the 1/3 of a stop difference in EVs between the D90 and D300 files.  Also, when measuring noise, focus is an important consideration -- actually, you need to defocus the scene to avoid measuring actual details and confusing them with sensor noise, which is why using different lenses and having different focus points are important to getting accurate measurements.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DxO Mark
« Reply #51 on: November 24, 2008, 02:29:21 pm »

Quote from: Tony Beach
There is not enough difference between a D90 and a D300 to get excited about.  That's the word I got back from Thom Hogan tonight when I asked him directly about this
I would be very excited about half stop difference.

Quote
anyone who thinks they will get noticeably better performance from a D90 than a D300 is misinformed
Based on the raw image I have it is you, who is misinformed or uninformed.

Quote
Arguing that someone who is not getting results they are satisfied with at ISO 1600 with a D300 would be satisfied with a D90 instead is setting them up for disappointment; the same is true at base ISO as it pertains to DR
On what do you base your assessment? On how two anyway not noisy images compare as "whole"?

Quote
Regarding the issues of not using the same lens, the same focal length, or the same EV; that highlights the lack of rigorous control at Imaging Resource and which is manifested by the 1/3 of a stop discrepancy in their comparative files
You don't understand the very basics of measuring the noise. If you give me two shots with the same ISO, containing smooth, unicolored, evenly lit areas in the very shadows, the best if several such spots in different darkness, I can measure and compare the noise, no matter which camera, which lens, which scenery, which illumination.

The layers of the TIFF I linked above show the principle of measurement: the noise is measured relatively to the actual pixel level in an area, on the raw channel. It does not matter which color, which lens, whatever. What matters is how high the pixel level and how high the noise is. If I find such areas with closely matching pixel levels, then the noise level shows which sensor is better. If I find areas with the same noise level, then the difference between the relative pixel levels shows how "much" one sensor is better than the other. (One needs to take several samples for a reliable result.)

I uploaded a compilation of such measurements as an Excel chart (the 5D2 data is not reliable); this is only as demo, it does not include the D300 nor the D90.

Quote
Can you see a difference in the lighting between these two?
This is irrelevant. For example at ISO 200 neither the D300 nor the D90 images from IR can be characterized as noisy at all. However, those images are suitable to measure the noise level and predict, how the sensors compare at ISO 200 if the shot contained severally underexposed areas. That is, what dynamic range is about.

Pick very dark areas (smooth, etc. as described above) from any ISO 1600 image and measure the noise; the consolidated result shows, what you can expect in a really low light situation. If the noise in one sensor is the same as in the other but on a half stop darker area, then the first one can create an equal image in lower light, or a less noisy image in the same light (assumed, that the ISO calibrations are equal).
Logged
Gabor

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
DxO Mark
« Reply #52 on: November 24, 2008, 04:46:28 pm »

Tony, just for having the D90 a 12 bit sensor while being the D300 14 bit, one could think of differences in the ADC which could make better any of the two models, specially regarding noise. Different bitdepth should make us think of different camera hardware and thus characteristics.

I particular regarding DR, I wouldn't just look at the final DR figure which is obtained just using a specific SNR threshold criteria. DxO Mark provides full SNR plots for each camera and there it's easy to see the behaviour regarding noise at any ISO and at any EV (unfortunately they chose an uncomfortable log10 scale for the X-axis rather than log2=EV, but converting it is not difficult):




Just by looking at that graph, and assuming measurements were properly done, it's easy to see that the D90 beats the D300 at any ISO and at any RAW exposure. And the difference increases the higher the ISO value, which means the D90 performs specially better in very low light conditions where a high ISO value is needed than the D300, and could thus deliver a noticeably higher DR for certain applications like indoor shooting.

A point that should be studied however is if RAW noise reduction is being applied. I think DxO Mark's RAW measurements, specially regarding noise, should be attached to some noise reduction and/or texture quality (contrast, detail) analysis. Camera vendors seem to be interested in making the RAW data less RAW everytime, and this can affect all these measurements.

BR
« Last Edit: November 24, 2008, 04:59:33 pm by GLuijk »
Logged

jani

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1624
    • Øyet
DxO Mark
« Reply #53 on: November 24, 2008, 05:52:37 pm »

Quote from: GLuijk
Just by looking at that graph, and assuming measurements were properly done, it's easy to see that the D90 beats the D300 at any ISO and at any RAW exposure. And the difference increases the higher the ISO value, which means the D90 performs specially better in very low light conditions where a high ISO value is needed than the D300, and could thus deliver a noticeably higher DR for certain applications like indoor shooting.
Considering that there's a year between the two cameras, this shouldn't be surprising.

Camera manufacturers have been known to make progress from one year to the next, and Nikon has been less shy than e.g. Canon in allowing the cheaper, more recent camera to catch up technologically.

Quote
A point that should be studied however is if RAW noise reduction is being applied. I think DxO Mark's RAW measurements, specially regarding noise, should be attached to some noise reduction and/or texture quality (contrast, detail) analysis. Camera vendors seem to be interested in making the RAW data less RAW everytime, and this can affect all these measurements.
That's food for thought, but I wonder what useful information we would gain from that approach.
Logged
Jan

Tony Beach

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 452
    • http://imageevent.com/tonybeach/twelveimages
DxO Mark
« Reply #54 on: November 25, 2008, 01:01:53 am »

Quote from: Panopeeper
Based on the raw image I have it is you, who is misinformed or uninformed.

From Imaging Resource?



These are screenshots from PhotoBola Raw Image Analysis, and they show where (on the bottom) the D300 file is clipped as a result of its being 1/3 stop underexposed compared to the D90 image; 1/250 compared to 1/200.

I actually wonder if you are a photographer; I have never seen a photograph of yours, just endless pixel peeping nonsense.  I know you are not a scientist, as no competent scientist would draw conclusions from such flawed comparisons.
Logged

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
DxO Mark
« Reply #55 on: November 25, 2008, 03:50:00 am »

Quote from: Tony Beach
I actually wonder if you are a photographer; I have never seen a photograph of yours, just endless pixel peeping nonsense.
I never saw a photograph of yours and that doesn't mean you are not a photographer.
Did you try http://www.panopeeper.com/?
« Last Edit: November 25, 2008, 03:51:20 am by GLuijk »
Logged

Dan Wells

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1044
DxO Mark
« Reply #56 on: November 25, 2008, 10:11:41 am »

I would like to see DxO labs attach some sort of noise thresholds to these DR tests... Imaging Resource's DR numbers, which I agree are less scientific, are shown at four different noise thresholds, and by far the closest match to the DxO numbers is at the most noisy of the four (really reaching into the shadows for detail most people won't want to print), which is generally within 0.2 stop, and often 0.1 stop, of he DxO number. IR's "high quality" DR numbers range from 1 to 2 stops less than this, and that varies substantially by camera - since DxO is already doing noise profiling, wouldn't it be fairly easy for them to report low-noise DR as well? They don't even need all four of IR's numbers, just the extreme (which they have) plus the low noise number on the other end...

                                              -Dan
Logged

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
DxO Mark
« Reply #57 on: November 25, 2008, 10:38:43 am »

Quote from: Dan Wells
I would like to see DxO labs attach some sort of noise thresholds to these DR tests...
They use the SNR=0 dB criteria. Anyway they provide full SNR plots, so you can calculate the DR according to any criteria you like best (e.g. the more useful in photographic terms SNR=12 dB=2EV).
« Last Edit: November 25, 2008, 10:41:53 am by GLuijk »
Logged

jani

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1624
    • Øyet
DxO Mark
« Reply #58 on: November 25, 2008, 02:11:17 pm »

Quote from: GLuijk
I never saw a photograph of yours and that doesn't mean you are not a photographer.
He has posted at least five to [a href=\'index.php?showtopic=29488\']ObPhoto thread #1[/a].
Logged
Jan

jani

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1624
    • Øyet
DxO Mark
« Reply #59 on: November 25, 2008, 02:11:52 pm »

Quote from: GLuijk
They use the SNR=0 dB criteria. Anyway they provide full SNR plots, so you can calculate the DR according to any criteria you like best (e.g. the more useful in photographic terms SNR=12 dB=2EV).
What's the rationale behind that?
Logged
Jan
Pages: 1 2 [3] 4   Go Up