Pages: 1 ... 5 6 [7] 8 9 ... 11   Go Down

Author Topic: DXOmark ranks DB image quality well below DSLR!  (Read 58090 times)

Carsten W

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 627
DXOmark ranks DB image quality well below DSLR!
« Reply #120 on: May 18, 2009, 04:12:42 am »

Quote from: Ray
Interesting! What I see from the graphs is that the D3X still trounces the P65+ with regard to dynamic range. In fact, the only advantages of the P65+ that are apparent in the DXOmark results are better noise, better tonal range and better color sensitivity, at base ISO only and on a small print (8"x12").

Minor addition: what we see is that DxO's measurements of DR still have the D3x come out on top. I have not yet heard of someone who has seen results from both who would agree with that, so all this points out is that how they measure it is not how we see it. In other words, the results are useless.
Logged
Carsten W - [url=http://500px.com/Carste

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
DXOmark ranks DB image quality well below DSLR!
« Reply #121 on: May 18, 2009, 05:37:55 am »

Quote from: carstenw
Minor addition: what we see is that DxO's measurements of DR still have the D3x come out on top. I have not yet heard of someone who has seen results from both who would agree with that, so all this points out is that how they measure it is not how we see it. In other words, the results are useless.

I have not yet seen any comparisons of real-world images, period. If someone is convinced their P45+ or P65+ delivers better DR than the D3X, then please show the comparisons. Hearing from such people is not sufficient. This is not audio. Seeing is believing.
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
DXOmark ranks DB image quality well below DSLR!
« Reply #122 on: May 18, 2009, 11:26:42 pm »

I would like to add to this issue of the reliability of the DXO tests, that I personally have never found them to be at significant variance with my own tests.

There are two instances where I compared the performance qualities of different cameras for my own benefit, and then later compared my impressions with the results at DXOmark. I found a surprisingly accurate correlation between my results and DXOmark.

All reasonable people would therefore understand my skepticism at claims that the DXOmark results in respect of MFDBs are a nonsense.

For the benefit of those who have an open mind, the two comparisons I made were:

(1) The Canon 5D versus the Nikon D3 at high ISO, in respect of noise and DR.

(2) The Canon 50D versus the Canon 5D in respect of noise and DR after ISO adjustment had been made to equalise DoF and shutter speed. (Ie. 50D at F4 and ISO 100, compared with the 5D at F6.3 and ISO 320).

I therefore take with a pinch of salt any unsubstantiated claims from MFDB owners that the DXOmark results are nonsense. I think such claims are merely bluster. However, I have an open mind. Show me some comparisons, and demonstrate that such comparisons are genuine and have been taken with sound technique and flawless methodology, and I'll reconsider my position.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DXOmark ranks DB image quality well below DSLR!
« Reply #123 on: May 19, 2009, 12:46:45 am »

Quote from: Ray
(2) The Canon 50D versus the Canon 5D in respect of noise and DR after ISO adjustment had been made to equalise DoF and shutter speed. (Ie. 50D at F4 and ISO 100, compared with the 5D at F6.3 and ISO 320)
Ray, do you mind uploading one of the ISO 320 raw files? If you have one heavily overexposed (raw clipping), that would be great, but if not, then any one is ok.
« Last Edit: May 19, 2009, 12:48:29 am by Panopeeper »
Logged
Gabor

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
DXOmark ranks DB image quality well below DSLR!
« Reply #124 on: May 19, 2009, 01:53:46 am »

Quote from: carstenw
Minor addition: what we see is that DxO's measurements of DR still have the D3x come out on top. I have not yet heard of someone who has seen results from both who would agree with that, so all this points out is that how they measure it is not how we see it. In other words, the results are useless.

Hum... care to share any factual comparisons? I don't really see how shadows and color transitons could get significantly cleaner than those of the D3x and still retain a photographic nature. This is now just about the tone mapping application used.



Cheers,
Bernard

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
DXOmark ranks DB image quality well below DSLR!
« Reply #125 on: May 19, 2009, 10:04:27 am »

Quote from: Panopeeper
Ray, do you mind uploading one of the ISO 320 raw files? If you have one heavily overexposed (raw clipping), that would be great, but if not, then any one is ok.

No worries! But give me a couple of days. Those test images were transferred to DVD discs some time ago and are stored at a different location, and I don't have my 5D with me at present.

Cheers!
Logged

Amery

  • Newbie
  • *
  • Offline Offline
  • Posts: 1
DXOmark ranks DB image quality well below DSLR!
« Reply #126 on: May 19, 2009, 04:02:11 pm »

This guy did a brief comparison of the P65+, H3DII-50, H3DII-39, CF22, 5DII, D3X, D700, D90, and Fuji S5:

http://www.flickr.com/photos/dos-chin/sets...57614936120567/

The subject matter isn't a good indicator of DR performance and there aren't any high ISO shots. However the full size files are available for download and comparison(some jpeg compression to fit flickr file size limitations). Hope this helps.
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
DXOmark ranks DB image quality well below DSLR!
« Reply #127 on: May 19, 2009, 10:26:08 pm »

Quote from: Amery
The subject matter isn't a good indicator of DR performance and there aren't any high ISO shots. However the full size files are available for download and comparison(some jpeg compression to fit flickr file size limitations). Hope this helps.

Thanks for the link, but it doesn't really help. The DXOmark results indicate that at base ISO (which appears to be actually ISO 44) the P65+ does display better tonality, greater color sensitivity and lower noise than the D3X, as one might expect, but interestingly only at base ISO. One imagines that such differences would be subtle and probably lost with the fairly severe amount of jpeg compression that's been applied to these Flickr images. (The 170mb of the P65+ has been compressed to just 6.65MB, approx a 25:1 compression.) The resolution advantage of the P65+ is not under dispute.

The really surprising aspect of the DXO results is that the DR of the D3X appears to be so much better than that of the P65+. Even when both images are downsampled to an 8"x12" print size, which of course improves both the DR and noise of the P65+ to a greater extent than it does for the the D3X, the D3X still retains a whopping 2 stops DR advantage at the 'real' ISO of 170, compared with the real ISO of 178 for the P65+.

The best-case scenario for the P45+, regarding dynamic range, is found at ISO 44 where DR is only a 2/3rds' stop worse than that of the D3X at ISO 78, at 8"x12" size.

It's these DR results that seem really surprising, considering that the P65+ is a larger sensor.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DXOmark ranks DB image quality well below DSLR!
« Reply #128 on: May 19, 2009, 10:47:17 pm »

Quote from: Ray
Even when both images are downsampled to an 8"x12" print size, which of course improves both the DR and noise of the P65+ to a greater extent than it does for the the D3X
This is plain rubbish. Downsampling does not increase the dynamic range the very least.

Quote
the D3X still retains a whopping 2 stops DR advantage at the 'real' ISO of 170, compared with the real ISO of 178 for the P65+
I belive only that comparison, which I cooked myself (paraphrasing WSC).

Without having seen any image created by P65 Plus, I risk saying, that this "2 stops DR advantage" is the the product of a comparison performed in drunken stupor.
Logged
Gabor

paratom

  • Full Member
  • ***
  • Offline Offline
  • Posts: 205
DXOmark ranks DB image quality well below DSLR!
« Reply #129 on: May 20, 2009, 03:37:38 am »

Quote from: Panopeeper
This is plain rubbish. Downsampling does not increase the dynamic range the very least.


I belive only that comparison, which I cooked myself (paraphrasing WSC).

Without having seen any image created by P65 Plus, I risk saying, that this "2 stops DR advantage" is the the product of a comparison performed in drunken stupor.

I use a D3x and a Sinar54LV back on a Hy6/Rollei system.
I have tried to run comparison several times but in the end it is hard to judge. First-how to convert them and with which paramameters. Second the slightest difference in focus will allready make a difference.
I came to the conclusion that or really judging a system one has to use it for some time to get an overall impression.

My overall impression regarding IQ (at low ISO) of D3x and Sinar54LV is the following:
-Dynamic Range: is very good for me with both cameras - if I would have to bet which one is a little better here I would say the D3x
-now where I feel the Sinar back is better than the D3x is the following:
   - (micro) detail of Skin and comparable textures- here the images from the SinarBack just look smoother and more natural with more micro detail to me; Skin from the D3x looks good but still sometimes a little bit plastic look; Also Color in skin tones looks more natural for my taste from the MF back. Also I find the transition between sharp and unsharp overall better / smoother from the larger sensor (dont know why this is the case).

So for me and my taste overall IQ I get from the Sinar back is better. If I dont need to be fast and if I dont need higher ISO and if I dont mind the weight I prefer to use MF camera.
I also like the Hy6 user interface - very eays acceess to mirror up, exp bracketing and the mirror stays up and the posibility to use self timer AND mirror up at the same time (I think the D3x can not do it and I find this to be a problem).
Both cameras works good- the D3x the universal and flexible speed machine, the Hy6 with the last additional 2% of IQ.




Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
DXOmark ranks DB image quality well below DSLR!
« Reply #130 on: May 20, 2009, 10:14:06 am »

Quote from: Panopeeper
This is plain rubbish. Downsampling does not increase the dynamic range the very least.

Without having seen any image created by P65 Plus, I risk saying, that this "2 stops DR advantage" is the the product of a comparison performed in drunken stupor.

Gabor,
Downsampling the images to an 8"x12" size at 300ppi, which represents approximately an 8mp image, would appear to create a perception of increased DR, according to DXO.

At full pixel size, the DR of the P65+ sensor at base ISO of 44 is reported as 11.51EV. Downsampled to an 8x12 print size it jumps up to 12.97EV, almost 1.5 stops greater.

The D3X on the other hand, jumps from a DR of 12.84 to 13.65EV, an increase of less than 1 stop.

I would deduce that what's happening here is that the signal-to-noise is increased as a result of the downsampling. The reduction in apparent noise has the effect of increasing apparent DR. This is not difficult to understand, is it? Hasn't it already been established that downsampling an image reduces noise? Downsampling from 60mp to 8mp has to have a noise reduction effect. In fact, according to DXO, it reduces noise by a very significant 8.8dB.



Logged

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
DXOmark ranks DB image quality well below DSLR!
« Reply #131 on: May 20, 2009, 10:42:10 am »

The increased DR due to downsampling is only valid if you are willing to accept the simultaneous loss of resolution. If I'm going to buy a 60MP MFDB, I really don't care what its DR is at 8MP, I care about the DR at 60MP.
Logged

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
DXOmark ranks DB image quality well below DSLR!
« Reply #132 on: May 20, 2009, 11:57:38 am »

Quote from: Jonathan Wienke
The increased DR due to downsampling is only valid if you are willing to accept the simultaneous loss of resolution. If I'm going to buy a 60MP MFDB, I really don't care what its DR is at 8MP, I care about the DR at 60MP.
I see two cases: those when the extra resolution is an IQ advantage and those where it is not. (Never mind that all my photography is in the second case when it come to 20MP plus sensors!)

In situations where the extra resolution is an advantage, the higher resolution option is superior at least in that respect. Equal print size comparisons are always fairest, with visible differences in resolution as well as in noise being part of the overall IQ comparison.

In situations where the extra resolution is not an advantage, it is perfectly reasonable for the higher pixel count image with "surplus resolution" to be processed down to the lower but completely acceptable resolution of the lower pixel count image, with NR processing, or downsampling, ... or maybe just printing/displaying on screen at small enough size.

What never makes sense is comparing sensor IQ on the basis of images displayed at different sizes. Doing that can prove that an image is worse than itself, because displaying it at sufficiently large size can make noise and DR limits more visible than in a smaller print. Hence, per pixel measures from images of different pixel counts are in themselves misleading for IQ comparisons. Measuring "noise per pixel", "DR per pixel" and "lines of resolution per linear pixel count" are a bit like measuring the "horsepower per cylinder" when comparing 4-cylinder and 8-cylinder engines.
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
DXOmark ranks DB image quality well below DSLR!
« Reply #133 on: May 20, 2009, 12:50:01 pm »

Quote from: Jonathan Wienke
The increased DR due to downsampling is only valid if you are willing to accept the simultaneous loss of resolution. If I'm going to buy a 60MP MFDB, I really don't care what its DR is at 8MP, I care about the DR at 60MP.

Nothing to do with accepting the simultaneous loss of resolution. It's not an option that affects DR. 8mp delivers 8mp of resolution whether you accept it or not. At such resolution, the D3X delivers greater DR.

I care about the DR of all my prints whetever their size. I have quite a few prints that are very close to 8x12" in A4 size albums.

If DR is increased by close to a stop when the D3X image is downsampled by a factor of 3, then it might be reasonable to surmise that the D3X's DR would be reduced by a slightly lesser amount when the image is upsampled by 2.5x to the size of the P65+ file. Instead of 12.84EV at native resolution, it might be just 12EV at 60mp, which makes it still 1/2 a stop better than the P65+.
« Last Edit: May 20, 2009, 12:51:41 pm by Ray »
Logged

Dustbak

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2442
    • Pepperanddust
DXOmark ranks DB image quality well below DSLR!
« Reply #134 on: May 20, 2009, 02:55:31 pm »

I have severe difficulties grasping the concept of increased DR with downsampling. So, suddenly detail appears in whites or blacks where there wasn't to begin with at a much larger resolution? Is there an explanation for this?

Also the idea of less noise with downsampling though I am willing to accept that sooner. It might be less visible but I am pretty sure the percentage of noise will be the same unless the downsampling is accompanied with noise reduction that tosses away bad pixels/noise first.
« Last Edit: May 20, 2009, 02:56:06 pm by Dustbak »
Logged

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
DXOmark ranks DB image quality well below DSLR!
« Reply #135 on: May 20, 2009, 04:05:49 pm »

Quote from: Dustbak
I have severe difficulties grasping the concept of increased DR with downsampling. So, suddenly detail appears in whites or blacks where there wasn't to begin with at a much larger resolution? Is there an explanation for this?
The explanation is that the noise floor level that sets the bottom of the dynamic range is based on the level of random fluctuations of noise, it is not the minimum output level above zero. Merging multiple pixels into one larger one increases the signal in proportion to the number of pixel merged, but the total noise increases by less, as it is a mixture of positive and negative deviations from the "true" signal, so there is typically some cancellation of the noise from the different pixels. So the signal to noise ratio improves.

Standard signal theory says that if the noise at each of the merged pixels is independent, the noise in a super-pixel produced from N pixels is about sqrt(N) times the typical single pixel noise level, while the signal is N times greater, and so the signal to noise ratio and dynamic range increases in proportion to sqrt(N).


But enough theory: there is a very familiar and obvious example in the form of black and white negatives. Examined closely enough under a microscope, the raw data is exposed and unexposed silver halide crystals, each either pure black or pure white. This means extremely very low DR and very strong visible noise at the silver grain "pixel" level. But prints effectively blur many of these "silver grain pixels" into a fine gray scale with a far greater dynamic range.  And famously, prints using the same emulsion in a larger format and so enlarged less have finer tonal gradations, a sign of higher DR and higher S/N ratios in the signal reaching the eye from the print.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DXOmark ranks DB image quality well below DSLR!
« Reply #136 on: May 20, 2009, 11:51:59 pm »

Quote from: BJL
Standard signal theory says that if the noise at each of the merged pixels is independent, the noise in a super-pixel produced from N pixels is about sqrt(N) times the typical single pixel noise level, while the signal is N times greater, and so the signal to noise ratio and dynamic range increases in proportion to sqrt(N)
Noise is not the only governing factor; in fact, this is the lesser issue. Our subject is photography; the dynamic range is not limited alone by noise but by detail reproduction as well. Or do you believe, that the dynamnic range of a camera can be increased by aggressive noise reduction?

However, image reduction destroys details. Therefor the decreased noise level is useless, except on clean, textureless surfaces - but that's, what noise reduction software can solve.

Logged
Gabor

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
DXOmark ranks DB image quality well below DSLR!
« Reply #137 on: May 21, 2009, 01:22:26 am »

Quote from: Panopeeper
Noise is not the only governing factor; in fact, this is the lesser issue. Our subject is photography; the dynamic range is not limited alone by noise but by detail reproduction as well. Or do you believe, that the dynamnic range of a camera can be increased by aggressive noise reduction?

However, image reduction destroys details. Therefor the decreased noise level is useless, except on clean, textureless surfaces - but that's, what noise reduction software can solve.

Gabor,
Whilst I agree with BJL's excellent explanation on this issue, I think there's another point that needs to be addressed with regard to resolution, which is also related to your points above. The P65+ is undoubtedly capable of greater resolution than the D3X, so one might tend to think that such greater resolution would apply across the whole 'subject-brightness-range' of the scene.

This is not necessarily true. If the scene being photographed has a high SBR (say 15EV), the P65+ will not deliver better detail than the D3X in the deepest shadows, whatever the print size (assuming equal print size for the comparison, of course). This is what the statement, 'the D3X has better DR than the P65+', actually means in practice.

We must all have observed that image detail is always significantly degraded in the deepest shadows of any scene that actually has deep shadows. It doesn't matter what camera you use, if the scene being photographed has a high SBR, then detail in the deepest shadows will likely be crap, even at base ISO. This is precisely why it's often necessary to bracket exposure and merge to HDR, or, when possible, use fill flash to illuminate the shadows, when the shadows are not too far away.

Another interesting issue that emerges from these DXOmark results is the apparent huge discrepancy between the nominal ISO sensitivities of the P65+ and the real and actual ISO sensitivity, as measured by DXO.

For example, ISO 400 on the P65+ is actually ISO 178. That's a huge discrepancy. Whereas ISO 200 on the D3X is actually ISO 170, a minor discrepancy. Such variance of course complicates the procedures for comparing the DR of these two cameras. Anyone who wishes to do serious comparisons of these two cameras needs to take this issue into consideration. If DXO is right, a big if perhaps, and what could be the subject of another thread. But bear in mind that DXO probably employs highly qualified scientists who know what they are doing. At the same time, they are in the business of marketing their own RAW converter, so their claims should be questioned.

If we assume that DXO is correct with regard to its ISO testing, then anyone comparing the P65+ at ISO 400 with the D3X, should set the D3X at ISO 200. This is not giving an advantage to the D3X. It's simply getting things right. Phase appears to have lied about their ISO settings. Many other manufacturers also appear to lie. We expect some variance and discrepancies. But more than a whole stop of variance??? Perhaps lying is not the right word. I don't want to start a legal issue. Perhaps there's some other technical issue which DXO is not addressing.

One way or another, this issue should be sorted. ISO is an international standard. It's supposed to mean something specific. We're very short on international standards. They should be adhered to as much as possible.

If we compare the DR of the D3X at ISO 170 with the DR of the P65+ at ISO 178 at the pixel level (in accordance with DXO results), we find that the D3X has a full 3 stops advantage. That's huge and very surprising. I really think that some of you busy professionals who might have access to both a P65+ and a D3X should take the time to check this out.
Logged

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
DXOmark ranks DB image quality well below DSLR!
« Reply #138 on: May 21, 2009, 11:09:25 am »

Quote from: Panopeeper
Noise is not the only governing factor ... the dynamic range is not limited alone by noise but by detail reproduction as well.
You seem to be redefining "dynamic range" to suit your purposes. I was explicitly talking only about dynamic range, not some more general concept of image quality including "detail reproduction". Detail is another dimension of image quality, not at all a part of the meaning of dynamic range.

And if you do wish to include detail in an IQ comparison between two cameras, then it even more makes sense to consider DR comparisons with equalized image detail, which means with downsampling, NR or such applied to the higher pixel count image within the limits of still retaining as much image detail as the lower pixel count camera offers.

By the way, downsampling from 60MP Bayer CFA raw output to say a 24.5MP RGB format will have more image detail than 24.5MP Bayer CFA raw, so it is not so clear how to equalize image detail. Maybe downsampling to half or less the lowest sensor pixel count equalizes detail.
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
DXOmark ranks DB image quality well below DSLR!
« Reply #139 on: May 21, 2009, 12:28:15 pm »

Quote from: BJL
You seem to be redefining "dynamic range" to suit your purposes. I was explicitly talking only about dynamic range, not some more general concept of image quality including "detail reproduction". Detail is another dimension of image quality, not at all a part of the meaning of dynamic range.

Well, you've lost me here, BJL   . Surely reproduction of detail is the crucial defining element of DR for the practicising photographer. If someone claims that the D3X has a higher DR than the P65+, then surely that means that the D3X, at any print size, will produce more detail in the deepest shadows than the P65+, when the scene photographed has a higher SBR than the DR capabilities of both cameras, eg. 15EV.

If one wishes to make the point that the obviously higher resolution of the P65+, across most of the tonal range of most scenes captured, is preferred to a perhaps slight resolution advantage of the D3X in the deepest shadows, then that is quite understandable. However, the resolution advantage that any high pixel-count camera has across the mid to high tones does not contribute to its DR assessment, does it?
Logged
Pages: 1 ... 5 6 [7] 8 9 ... 11   Go Up