Luminous Landscape Forum

Site & Board Matters => About This Site => Topic started by: BernardLanguillier on February 03, 2009, 06:37:28 am

Title: Eyes vs Numbers
Post by: BernardLanguillier on February 03, 2009, 06:37:28 am
Hello Michael,

Interesting essay, thanks for posting it.

Resolution should indeed clearly be taken into account when rating cameras/backs. But even so, a number will indeed clearly never have the same meaning for different people with different needs and ends up being pretty useless (ideally the rating should be taking into account the print size).

The DxO mark is basically a measurement of sensor quality. Sensor quality doesn't tell the full story, but it is nonetheless relevant. I understand your point about the comparions being perhaps not 100% fair, but the superiority of the A900 along the metrics selected by DxO seems pretty logical when you consider that:

- The photosite size is about the same between 20+ MP DSLRs and 39MP backs (the D3/D700 photosites being significantly larger),
- The DSLRs use micro-lenses which contributes to better light efficiency,
- The DSLR technology is 3 years younger (nearly as long as a whole Bush mandate),
- Sony/Canon are much more talented semi-conductor companies than Kodak with a much longer history, a much more diverse range of application and a much higher number of knowledgeable engineers (and they work more...),
- They probably invest at least an order of magnitude more in fundamental semi-conductor research and tend to pool their resources between companies for activities of common interest.

So there are pretty good reasons to think that the DxO results provide a realistic view of what we have at hand.

As far as trusting one's eyes instead of measurement results, when a G10 is hard to disthinguish from a P45+ in a small print, wouldn't you say on the contrary that the DxO results are a pretty good match to what one actually sees?

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: michael on February 03, 2009, 08:56:56 am
You've hit the nail on the head Bernard, but not likely in the way that you think.

My little experiment with G10 and P45+ files last year showed the core of the issue. On smallish prints the difference between good and very good becomes essentially invisible. That's one of DxO's flaws. They "normalize" to an 8X10" print. What's that about?

If all I did were 8X10" prints shot at low ISO I'd likely sell all of my gear for something like a G10. But I don't. For me a small print is 13X19", and my normal prints for exhibition and sale are about 24 X 32". Many are larger. That's why I'm buying a P65+. That's why I was ultimately unhappy shooting with the 12MP Nikon D3 in Botswana last year. The high ISO shots were wonderful, but essentially too small for exhibition prints. I'd have likely been happier with a somewhat noisier but higher res cameras (the D3x, for example, if it had been available). I'm limited in the size of print that I can make with cameras much under 20MP.

And before someone jumps in to say how they once made a wonderfull billboard from a Minox negative, let's just stipulate that acceptability of image quality is a personal and subjective matter. My standards may simply be different (higher?) that someone else's.

Michael

Title: Eyes vs Numbers
Post by: NikosR on February 03, 2009, 09:08:29 am
Michael,

When DxO 'normalize' they do it to take into consideration the benefits stemming from increased resolution (they explain this quite succintly in their site) It is a mathematical calculation not an actual production of a print nor a 'downsizing' of the higher resolution files.

They reason they use the 8x10 size (at 300ppi I believe) is because they need to use the least common denominator WITHOUT requiring to interpolate (upsample) the data. The interpolation process itself (I'm not sure if this is even possible at the raw data level) would introduce another variable in the comparison.

They do 'fail' to take the very real benefits of increased useability of the larger resolution files into account, but this is not a directly measurable quantity.

What this really means in effect is that their results are supposed to be directly applicable for comparing cameras operating WITHIN THE LIMITS of their resolution. It is for the photographer to interprete this data and use it into his decision making taking ALSO into consideration what the increased resolution means to HIM in terms of intended use.
Title: Eyes vs Numbers
Post by: Ray on February 03, 2009, 09:16:15 am
Quote from: BernardLanguillier
Hello Michael,

Interesting essay, thanks for posting it.

Resolution should indeed clearly be taken into account when rating cameras/backs. But even so, a number will indeed clearly never have the same meaning for different people with different needs and ends up being pretty useless (ideally the rating should be taking into account the print size).

The DxO mark is basically a measurement of sensor quality. Sensor quality doesn't tell the full story, but it is nonetheless relevant. I understand your point about the comparions being perhaps not 100% fair, but the superiority of the A900 along the metrics selected by DxO seems pretty logical when you consider that:

- The photosite size is about the same between 20+ MP DSLRs and 39MP backs (the D3/D700 photosites being significantly larger),
- The DSLRs use micro-lenses which contributes to better light efficiency,
- The DSLR technology is 3 years younger (nearly as long as a whole Bush mandate),
- Sony/Canon are much more talented semi-conductor companies than Kodak with a much longer history, a much more diverse range of application and a much higher number of knowledgeable engineers (and they work more...),
- They probably invest at least an order of magnitude more in fundamental semi-conductor research and tend to pool their resources between companies for activities of common interest.

So there are pretty good reasons to think that the DxO results provide a realistic view of what we have at hand.

As far as trusting one's eyes instead of measurement results, when a G10 is hard to disthinguish from a P45+ in a small print, wouldn't you say on the contrary that the DxO results are a pretty good match to what one actually sees?

Cheers,
Bernard

This is another stimulating article from Michael and once again we have two threads going. But what a surprise that the DB results, that have just appeared on the DXOMark website, look worse than those for the D3X. But they don't look worse than those of the A900 at base ISO for all cameras.

It seems that once again the D3X is the stellar performer. I'm considering here the individual charts for SNR, DR and Tonal Range, rather than the combined, weighted overall figures, which I agree can be misleading.

What I would like to see, from people who are skeptical about the validity of the DXO results, are test images processed with the best converter for each camera, demonstrating how and to what degree the DXO figures are wrong or misleading.
Title: Eyes vs Numbers
Post by: rternbach on February 03, 2009, 09:33:28 am
Hi,
I would like to see added to the numerical and engineering measures the human perception element in the equation. That is, present to a panel of judges the captured images(don't reveal how they were captured and, no, I don't have an easy way to calibrate the judges for accuracy).  Have the panel members independently rate the images on a variety of dimensions. Throw that info up on the screen along with the more "objective" measures. Correlations may show up, or not. Methods of presentation and judging may need to be arbitrarily adjusted as a result of preliminary attempts. Keep all the data obtained and give it to someone who likes to do that sort of analysis. Personally speaking, I'd rather be out on the open seas with a few good cameras and a month of days to shoot and capture images.

Best regards to all,

Rudy Ternbach
Western Masstts., USA
--------------------------------------------------------------------------

Quote from: NikosR
Michael,

When DxO 'normalize' they do it to take into consideration the benefits stemming from increased resolution (they explain this quite succintly in their site) It is a mathematical calculation not an actual production of a print nor a 'downsizing' of the higher resolution files.

They reason they use the 8x10 size (at 300ppi I believe) is because they need to use the least common denominator WITHOUT requiring to interpolate (upsample) the data. The interpolation process itself (I'm not sure if this is even possible at the raw data level) would introduce another variable in the comparison.

They do 'fail' to take the very real benefits of increased useability of the larger resolution files into account, but this is not a directly measurable quantity.

What this really means in effect is that their results are supposed to be directly applicable for comparing cameras operating WITHIN THE LIMITS of their resolution. It is for the photographer to interprete this data and use it into his decision making taking ALSO into consideration what the increased resolution means to HIM in terms of intended use.
Title: Eyes vs Numbers
Post by: BernardLanguillier on February 03, 2009, 09:39:59 am
Quote from: John Schweikert
As a quickie, I compared the Leaf Aptus 75s, Nikon D3x and Canon 5DII, and DXO numbers make the Leaf inferior at low ISOs to the both DSLRs. No offense but that is just hogwash. I have a Leaf Aptus 22 and its dynamic range is far superior to the Canon 5DII which I have as well. An A75 is even better than an A22.

What is your assessement of Leaf vs D3x then?

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: BernardLanguillier on February 03, 2009, 09:54:31 am
Quote from: michael
You've hit the nail on the head Bernard, but not likely in the way that you think.

No, I think we agree perfectly Michael.

Most of my images have been in the 100+MP range this past year through stitching, and I perfectly understand the need for more resolution. I am not surprised the least bit by your interest for the P65+.

A recent 180 megapixel pano to prove the point.

(http://farm4.static.flickr.com/3365/3250811512_2b79e370af_o.jpg)

Cheers,
Bernard

Title: Eyes vs Numbers
Post by: michael on February 03, 2009, 10:20:05 am
A lovely image Bernard!

I'd live to see a big print. The web hardly does it justice.

Michael

Title: Eyes vs Numbers
Post by: rdonson on February 03, 2009, 10:45:59 am
Michael,

I think your essay was timely and greatly needed.  We've witnessed the megapixel wars and are probably starting into the lens resolving wars as we seek the ultimate in image quality.  As you point out it comes down to when do the measurements mean something in the real world.

It's probably time to extend your essay to include inkjet printers.  As you've said previously, the high end large format Epson, Canon and HP printers can all deliver stunning prints.  Yet we argue endlessly over minute details looking for which one is best.

Wisdom may be when we discover when good enough is good enough.
Title: Eyes vs Numbers
Post by: Ray on February 03, 2009, 11:02:58 am
Quote from: John Schweikert
No offense but that is just hogwash. I have a Leaf Aptus 22 and its dynamic range is far superior to the Canon 5DII which I have as well. An A75 is even better than an A22.


Can you show us your results, or must we take your word for it?

Anyone can get almost any result they want (within reason) if they choose their comparison subject well, to reduce the differences. For example, Michael has demonstrated with a particular subject at an A3+ size, that there's no discernible difference in image quality between the Canon G10 and the P45+, yet DXO test results indicates there's a wide gap between these two cameras in respect of SNR, DR and Tonal Range at a normalised 8x12" size.

Do we therfore claim that DXO results are wrong or misleading, or do we examine why the subject photographed might not be ideal to reveal such differences?

The limited size of the image or print clearly knocks out the resolution advantage of the P45+, which DXO doesn't address in any case, except in as far as it impacts upon SNR, DR etc when the image is downsampled.

The lack of extremes of subject brightness range, in Michael's comparison, makes it difficult to appreciate the additional DR of the P45+, which it surely has, and the heavily textured nature of the subject (twigs, leaves and bark) make it difficult to appreciate the smoother tonality and lower noise of the P45+ shot, which it surely has.

Such factors should be taken into consideration.
Title: Eyes vs Numbers
Post by: ErikKaffehr on February 03, 2009, 11:07:28 am
Hi,

One of the interesting aspects is that we now accept the need for megapixels. A while ago everyone was stressing large pixels and low noise, now we take a more balanced view where both noise and resolution are important.

My guess is from a very limited experience is that 12 MPixels are good enough for A2 prints, so 24 Mpixels should be good enough for A1. With good technique and taking some liberties with image quality a size larger would be possible.

In my view the DxO data are still worthwhile but combining them into a single figure of merit is to oversimplify. I would also say that there is little utility to quality you don't need. If your subject doesn't have a high DR you don't need high DR in your sensor. Prints have much less DR than screen or sensor. Having excess DR, resolution or whatever is of little utility, but can be nice the day you photograph black cats in a coal cellar or need to print really big.

Best regards
Erik

Quote from: rdonson
Michael,

I think your essay was timely and greatly needed.  We've witnessed the megapixel wars and are probably starting into the lens resolving wars as we seek the ultimate in image quality.  As you point out it comes down to when do the measurements mean something in the real world.

It's probably time to extend your essay to include inkjet printers.  As you've said previously, the high end large format Epson, Canon and HP printers can all deliver stunning prints.  Yet we argue endlessly over minute details looking for which one is best.

Wisdom may be when we discover when good enough is good enough.
Title: Eyes vs Numbers
Post by: hubell on February 03, 2009, 11:19:09 am
Quote from: michael
If all I did were 8X10" prints shot at low ISO I'd likely sell all of my gear for something like a G10. But I don't. For me a small print is 13X19", and my normal prints for exhibition and sale are about 24 X 32". Many are larger. That's why I'm buying a P65+. That's why I was ultimately unhappy shooting with the 12MP Nikon D3 in Botswana last year. The high ISO shots were wonderful, but essentially too small for exhibition prints. I'd have likely been happier with a somewhat noisier but higher res cameras (the D3x, for example, if it had been available). I'm limited in the size of print that I can make with cameras much under 20MP.
Michael

I do find it interesting that Phase One itself, when asked why someone should consider upgrading from a P45 to a P65, states  that the resolution advantage is relatively minor and the "real" advantages with the P65 are cleaner higher ISO files of 15mp and the POSSIBILITY of future upgrades(no indication of what upgrades or when). Well, for about 1/3 the price of the upgrade you could buy a D3X with 24mp files at 1600 ISO that will probably blow away those 15mp 1600 ISO P65 files. Plus, you get world class AF and a gorgeous LCD with the D3X.
I could afford to upgrade from a 39mp back to a 60mp back, but there is no way I am going for it unless I was convinced that, aside from the relatively minor resolution advantage and vague promises about good  things in the future, the new back offered significant improvements in dynamic range, a world class LCD and the ability to quickly and easily pull 20 different "looks" from a file like I could get by loading 20 different films into my old Pentax 67.
Title: Eyes vs Numbers
Post by: ndevlin on February 03, 2009, 11:25:29 am
Michael is dead-on with his assessment in this essay, but is perhaps a bit too kind to DxO.

Let me "put the cat amongst the pigeons" as much as DxO did with their MF chart, by asserting fairly baldly that their camera-measuring site and numbers are as close to useless nonsense as we need bother going. I'm not saying their data is wrong or that their intentions are anything but good, far from it.  

While I understand, and even struggle with myself, the measurbatory instinct which drives DxO Mark, and so much of the camera-chatter online, they are producing a template for discussion which is so fatally divorced from the creative process that it is functionally valueless.

There is one, and only one, meaningful measure of any digital camera's quality, and that is how the finished product looks in its finished form.  (Indeed, that form will itself in many instances alter the outcome of the evaluation).  

And that's the rub. A properly exposed (to the right) file looks awful out of the camera most of the time. The real road to finished image quality leads through the issue is how that data 'stretches' (my word again) over the tonal range in post.  The real finished quality can only be truly assessed in the finished product, which requires the extensive intercession of subjective human choices of creation, which cannot be quantified.

Files from certain cameras are vastly 'deeper' (my word) in the sense that, with equivalent exposures, they offer far greater maleability of the image in post-processing.  From recent experience in Antartica, I can say with certainty that files from a Phase back render tonal subtlety and richness at a level unachievable any 35mm dslr I have ever used. That same relationship appears to translate downwards, viz-a-vis even superior point and shoot cameras.  This is likely due to bit-depth, but the actual effect of it on finished image quality is only visible after the craft *not science* of post-processing has been applied, subjectively, by the artist in question.

The DxO model is somewhat akin to rating paintings based on a chemical analysis of the paints used. Interesting, but just not that useful in any serious conversation about aesthetics.  If we all photographed test targets for a living/hobby, such measures would mean more.  If we are trying to use technology to acheive self-expression, the serve little purpose at all.

- N.
Title: Eyes vs Numbers
Post by: NikosR on February 03, 2009, 12:05:27 pm
Could someone explain to me why having a high end dSLR measuring equal to or better in some respect to an MFDB should cause such a stir?

What's the inherent technical or engineering quality (apart from resolution and lack of AA) that would preclude a dSLR from measuring AND being actually better than an MFDB in things like DR?

Bit depth? Better analogue circuits? Better ADCs? American / European engineers being cleverer than Japanese ones? Access to resources? Being ridicoulously more expensive?

In the 'old' days we used to talk about pixel size. But nowdays this argument is gone and forgotten.

Just wondering...
Title: Eyes vs Numbers
Post by: fike on February 03, 2009, 12:25:52 pm
Bravo, Michael.  

I agree with earlier poster who said you were a bit too kind to DXO.  It is not that these measurements are worthless, but they can skew our perceptions.

Whenever you use metrics to measure quality of any sort, the result over time is that the product will mutate to meet the numerical goals--usually sacrificing some other important quality to get there.  One great example of this is when we allowed the megapixel to be the digital photo metric of quality, we got cameras that sacrificed many other attributes (noise, sharpness, file-sizes, speed of operation) to increase megapixels.

I will make one very strong statement that I believe must be accepted.  QUALITY is by DEFINITION SUBJECTIVE.

Things go wrong when metrics mean more than subjective quality...

like quality of service at a grocery store. They measure items scanned per minute.  If the cashier falls below a rate threshold, they are deficient.  So, to keep up, the cashier bags your groceries poorly, crushing your bread.  That wasn't the desired quality outcome, but the numbers don't lie do they?

The risk that we take as educated photographic consumers is to avoid another technology race like the megapixel race that has obsessed manufacturers recently.  They have pursued increased resolution often to the exclusion of more important qualities like ergonomics, features, color rendition, high ISO, etc...  I don't want DXO Mark to become the new bottom line of camera quality.  Then we will be stuck in a new purgatory of camera evolution.
Title: Eyes vs Numbers
Post by: fike on February 03, 2009, 12:32:05 pm
I wonder how a scanning back would rank on DXO tests?  Those things are a real bear to operate in the field, but their resolutions are insane.
Title: Eyes vs Numbers
Post by: douglasf13 on February 03, 2009, 12:50:04 pm
Quote from: NikosR
Could someone explain to me why having a high end dSLR measuring equal to or better in some respect to an MFDB should cause such a stir?

What's the inherent technical or engineering quality (apart from resolution and lack of AA) that would preclude a dSLR from measuring AND being actually better than an MFDB in things like DR?

Bit depth? Better analogue circuits? Better ADCs? American / European engineers being cleverer than Japanese ones? Access to resources? Being ridicoulously more expensive?

  I think bit depth and component quality are in the mix, as well as lens quality.
Title: Eyes vs Numbers
Post by: KSH on February 03, 2009, 12:55:06 pm
Quote from: ndevlin
Michael is dead-on with his assessment in this essay, but is perhaps a bit too kind to DxO.

Let me "put the cat amongst the pigeons" as much as DxO did with their MF chart, by asserting fairly baldly that their camera-measuring site and numbers are as close to useless nonsense as we need bother going. I'm not saying their data is wrong or that their intentions are anything but good, far from it.  

While I understand, and even struggle with myself, the measurbatory instinct which drives DxO Mark, and so much of the camera-chatter online, they are producing a template for discussion which is so fatally divorced from the creative process that it is functionally valueless.

There is one, and only one, meaningful measure of any digital camera's quality, and that is how the finished product looks in its finished form.  (Indeed, that form will itself in many instances alter the outcome of the evaluation).  

And that's the rub. A properly exposed (to the right) file looks awful out of the camera most of the time. The real road to finished image quality leads through the issue is how that data 'stretches' (my word again) over the tonal range in post.  The real finished quality can only be truly assessed in the finished product, which requires the extensive intercession of subjective human choices of creation, which cannot be quantified.

Files from certain cameras are vastly 'deeper' (my word) in the sense that, with equivalent exposures, they offer far greater maleability of the image in post-processing.  From recent experience in Antartica, I can say with certainty that files from a Phase back render tonal subtlety and richness at a level unachievable any 35mm dslr I have ever used. That same relationship appears to translate downwards, viz-a-vis even superior point and shoot cameras.  This is likely due to bit-depth, but the actual effect of it on finished image quality is only visible after the craft *not science* of post-processing has been applied, subjectively, by the artist in question.

The DxO model is somewhat akin to rating paintings based on a chemical analysis of the paints used. Interesting, but just not that useful in any serious conversation about aesthetics.  If we all photographed test targets for a living/hobby, such measures would mean more.  If we are trying to use technology to acheive self-expression, the serve little purpose at all.

- N.

It seems to me that DxO does not claim that their ranking shows which camera is best, or better than others, or which files are more malleable or "deeper." They are posting results of measurements they have made, award points on the basis of their measurements, and rank the cameras accordingly. They are open about their assumptions and limitations.

You claim that the results are "functionally useless" for the creative process, which I take to say: "Don't decide on a camera based on DxO marks alone!" Fair enough, good advice. But does that make the marks useless? And what do you have to offer instead, as measurement of "better" (and you ARE saying that the Phase gives better files)? You are introducing a number of variables into your assessement, which make that assessment deeply subjective, and therefore, in a sense, functionally useless for anyone who is not you. I am NOT saying that you are wrong, yours is the way to find what camera, or rather, ther files from which camera, suit your personal needs and preferences best. But I think there is room for both approaches, the subjective one, and the objective one. They are not mutually exclusive.

Let me be honest: To me, your post, and Michael's essay, come across as if you were offended by the DxO marks. Would Michael have produced the essay, would you have written your post if the results had been different, if they had ranked the medium format sensors ahead of the 35mm SLRs? Have you have worked on D3X files in the same manner as you have with the Phase One files? Have you done blind tests, working on files without knowing from which camera they came? If the answer is "yes" to these questions, that would make your assessment more valuable to me (who has worked with neither type of file).

Karsten
Title: Eyes vs Numbers
Post by: imagico on February 03, 2009, 01:00:48 pm
Quote from: michael
You've hit the nail on the head Bernard, but not likely in the way that you think.

My little experiment with G10 and P45+ files last year showed the core of the issue. On smallish prints the difference between good and very good becomes essentially invisible. That's one of DxO's flaws. They "normalize" to an 8X10" print. What's that about?
Michael

But OTOH when you look at the individual results (for example dynamic range) you can see the 'normalized' and non-normalized results and all the MFDBs they tested compare badly to the top DSLRs in both versions.  And as pointed out prominently on the DxO site elsewhere resolution compensates for noise in a way.  How much of this disadvantage is owed to internal raw processing is of course unclear.  Since the only way for DSLR makers to reduce noise by internal processing beyond good sensor design is by trading resolution the only 'fair' approach would be to measure actual resolution (not only pixel pitch) and to 'punish' the DSLR rating for lower resolutions (and also microcontrast).  The exact metrics of this would of course be a subjective decision.

From the engineering standpoint one also has to say that the design tricks used by the DSLR sensor makers are surely much harder to implement in a larger sensor even if the makers had equal expertise.
Title: Eyes vs Numbers
Post by: douglasf13 on February 03, 2009, 01:07:45 pm
Quote from: KSH
Have you have worked on D3X files in the same manner as you have with the Phase One files? Have you done blind tests, working on files without knowing from which camera they came? If the answer is "yes" to these questions, that would make your assessment more valuable to me (who has worked with neither type of file).

Karsten

  From the sounds of it, Michael has.  It'll be interesting what he says.
Title: Eyes vs Numbers
Post by: Tomcat on February 03, 2009, 02:11:27 pm
Maybe I'm just an old crotchety skeptic, but I think some key things are being missed with this Dxo thing:

1. Dxo clearly listed they only tested the sensors, not actual image quality.  They did not process the raw files.  There is no way to judge actual resolving power, final color capability, etc. without this.

2. They compared the latest DSLRs against 2006-era MF backs.  They point out how much better the DSLR sensors have gotten, but they don't allow for the MF sensors to have the same opportunity.  

3. I can't help but notice that all of Dxo's test pages have a big ad for their raw processing software, which only supports DSLRs.  Sorry, but I'm having a hard time accepting that the results aren't slanted towards DSLRs because of this and #2.  They are getting a lot of free press from these results, and a lot of people of now viewing their software ad...

Color me skeptical.
Title: Eyes vs Numbers
Post by: otbricki on February 03, 2009, 02:34:29 pm
Interesting article, and especially the comparison with high end audio.

DXO is off to a start, but clearly they have some refinement to their measurements before people will take them as clear measures of the capabilities of the camera. It is important to realize that these sorts of tests are very valuable to the development process of the technology. If you are just going to rely on subjective opinions then you are going to get into trouble because you won't have a repeatable way of determining the quality of the camera. Now some things of course will always be subjective; preferences in color rendering for example. But the color match with the subject can be measured.

The comparison with high end audio is important, but not the way the article poses it. High end audio has a very bad reputation in the engineering community because nobody has been able to reproduce the claims of these so-called 'golden ears' in any measurement or controlled test. I would hate to see cameras head in this direction because it allows anyone to sell anything with totally outlandish claims that are totally unreproducible. Fortunately I don't think is likely to happen because a photograph is frozen in time, making it possible for everyone to examine the same result of the reproduction process. Audio suffers because of the ephemeral nature of the reproduction.
Title: Eyes vs Numbers
Post by: vjbelle on February 03, 2009, 02:47:32 pm
Quote from: hcubell
I do find it interesting that Phase One itself, when asked why someone should consider upgrading from a P45 to a P65, states  that the resolution advantage is relatively minor and the "real" advantages with the P65 are cleaner higher ISO files of 15mp and the POSSIBILITY of future upgrades(no indication of what upgrades or when). Well, for about 1/3 the price of the upgrade you could buy a D3X with 24mp files at 1600 ISO that will probably blow away those 15mp 1600 ISO P65 files. Plus, you get world class AF and a gorgeous LCD with the D3X.
I could afford to upgrade from a 39mp back to a 60mp back, but there is no way I am going for it unless I was convinced that, aside from the relatively minor resolution advantage and vague promises about good  things in the future, the new back offered significant improvements in dynamic range, a world class LCD and the ability to quickly and easily pull 20 different "looks" from a file like I could get by loading 20 different films into my old Pentax 67.

AMEN!!!  Couldn't agree with you more!  It will be interesting to see how the dust settles at the end of the year.  

Victor
Title: Eyes vs Numbers
Post by: fike on February 03, 2009, 02:49:45 pm
Quote from: otbricki
Interesting article, and especially the comparison with high end audio.

DXO is off to a start, but clearly they have some refinement to their measurements before people will take them as clear measures of the capabilities of the camera. It is important to realize that these sorts of tests are very valuable to the development process of the technology. If you are just going to rely on subjective opinions then you are going to get into trouble because you won't have a repeatable way of determining the quality of the camera. Now some things of course will always be subjective; preferences in color rendering for example. But the color match with the subject can be measured.

The comparison with high end audio is important, but not the way the article poses it. High end audio has a very bad reputation in the engineering community because nobody has been able to reproduce the claims of these so-called 'golden ears' in any measurement or controlled test. I would hate to see cameras head in this direction because it allows anyone to sell anything with totally outlandish claims that are totally unreproducible. Fortunately I don't think is likely to happen because a photograph is frozen in time, making it possible for everyone to examine the same result of the reproduction process. Audio suffers because of the ephemeral nature of the reproduction.

You can't automate the measurement of every intangible property of image quality.  We lack the tools to do it.  Human evaluation is the only way to aggregate all the liminal factors with all the subliminal factors.
Title: Eyes vs Numbers
Post by: Panopeeper on February 03, 2009, 02:50:47 pm
I read following statements in Eyes vs Numbers:

Quote
I think though that the main problem is that DxO is analyzing raw camera data, which does not include any correction of fixed pattern noise. DSLR cameras remove these noise sources before storing their raw data. Medium format cameras do not. DSLRs also apply a lot of noise reduction algorithms before storing the data...Nikon, for example, does a lot of on-chip noise reduction to their D3 and D3x raw files

I feel following needs to be said, for Michael's statements may be taken by some readers too literally; this is not for or against any camera but for the better understanding of these issues.

1. All Nikon DSLR cameras cut off a portion of the pixel data, thereby removing pattern noise (one could not name this "noise reduction" without provocing the international Nikon community to violent responses). All Canon DSLRs keep the raw data "uncut". This difference may have caused the ridiculous assertion by DxO, that the D3X has two stops greater dynamic range than the 5D2.

2. Noise reduction in the sense most people understand it will be performed on raw data only by Sonys, at least by the A700 and A900. This step could not be disabled with an earlier firmware version of the A700, but now OFF means OFF, apparently.

3. On-chip (on-sensor) noise reduction occurs with all CMOS sensors, which are inherently more noisy than CCD. However, that is single-pixel noise reduction, which has absolute nothing to do with the noise reduction in the sense as it is done in raw processing software or by the mentioned Sony cameras, namely based on the context of pixels.

4. The raw data of the Phase One cameras is far from being really raw.

Finally, there is nothing wrong in analyzing raw data and making certain conclusions based on that; at least I am doing just that without feeling guilty. In fact, one can make certain findings only based on the raw data. However, one needs to be careful with those findings. I'm afraid, in the DxO evaluation of the P45+ they used ACR for the analysis; if so, that invalidates all of their findings.


Title: Eyes vs Numbers
Post by: John Camp on February 03, 2009, 03:10:04 pm
Quote from: otbricki
<snip> DXO is off to a start, but clearly they have some refinement to their measurements before people will take them as clear measures of the capabilities of the camera. <snip>

I honestly don't think this will ever happen. If you look at Sean Reid's review site [a pay site], you'll see that he often refers to a way a lens "draws." So in a review of high-end normal lenses, he gave excellent marks both to a Nikon and a Zeiss 50mm, said either one is capable of doing any kind of professional work, but he said that they draw differently and he prefers the Zeiss. So in this case, part of the "quality" equation is measuring Sean Reid's preferences, which may change from time to time. On the Online Photographer recently, Mike Johnston, who knows a lot about lenses, panned a lens because of its distortions, but actually liked the "look" it produced. In audio, you have large numbers of people who agree (apparently) that some amp/speaker systems sound better than others, despite measures to the contrary. It seems to me that the problem is with the engineers, not with the ears. Engineers have a bias toward measurement, which is a real bias, because if only 20% of a system is currently measurable and quantifiable, it's possible that the other 80% is more important in every way -- yet the engineers will tend to go with what they can measure, because that's their bias. It's the old "bumblebees can't fly" problem -- bumblebees CAN fly, the engineers just didn't have the ability to quantify the characteristics that allowed the bumblebees to do so.

The DXO problem is that they insist that they have a measure, but it's not a real measure. They have a whole bunch of measures, but those measures don't include the whole universe of possibilities, and without the whole universe, you can't reach a valid conclusion. And it may be inherently *impossible* to measure the entire universe of possibilities. So, if a lot of people with good, trained photographic eyes say that a large P65 print is better than a large D3 print, and a DXO measurement says it's not, then the eyes are right, just as they're right when they see a bumblebee fly.

JC
Title: Eyes vs Numbers
Post by: NikosR on February 03, 2009, 03:28:16 pm
Quote from: Panopeeper
I read following statements in Eyes vs Numbers:



I feel following needs to be said, for Michael's statements may be taken by some readers too literally; this is not for or against any camera but for the better understanding of these issues.

1. All Nikon DSLR cameras cut off a portion of the pixel data, thereby removing pattern noise (one could not name this "noise reduction" without provocing the international Nikon community to violent responses). All Canon DSLRs keep the raw data "uncut". This difference may have caused the ridiculous assertion by DxO, that the D3X has two stops greater dynamic range than the 5D2.

2. Noise reduction in the sense most people understand it will be performed on raw data only by Sonys, at least by the A700 and A900. This step could not be disabled with an earlier firmware version of the A700, but now OFF means OFF, apparently.

3. On-chip (on-sensor) noise reduction occurs with all CMOS sensors, which are inherently more noisy than CCD. However, that is single-pixel noise reduction, which has absolute nothing to do with the noise reduction in the sense as it is done in raw processing software or by the mentioned Sony cameras, namely based on the context of pixels.

4. The raw data of the Phase One cameras is far from being really raw.

Finally, there is nothing wrong in analyzing raw data and making certain conclusions based on that; at least I am doing just that without feeling guilty. In fact, one can make certain findings only based on the raw data. However, one needs to be careful with those findings. I'm afraid, in the DxO evaluation of the P45+ they used ACR for the analysis; if so, that invalidates all of their findings.


Gabor,

Thanks for your contribution. You confirmed much of what I was trying to say in this and other threads.

1. That issue of where Nikon are placing their zero point is well documented. It does affect pattern noise and is generally a good thing for all but the astrophotographers.

2. The Sonys, as you are saying, are the only cameras documented to perform real noise reduction (image processing in the digital domain for the purposes of reducing noise). I wonder where Michael got the feeling all dSLRs do that. And as you say, in the latest versions this can be switched off (and hopefully DxO have tested with this to off).

3. On chip 'noise reduction' cannot be called noise reduction.  A better word would be noise avoidance or measures to increase the S/N ratio. Any other noise avoidance technique (especially if performed in the analog domain) also cannot be called image processing or noise reduction. It is not a matter of semantics. These techniques only help to get cleaner digital data without compromising the image. If the MFDB manufacturers don't need, don't know or don't care to use such techniques has nothing to do with whether their raw data can be compared to dSLR raw.

4. I can't comment on that but if true it makes Michael's objection to comparing dSLR raw to MFDB raw even more irrelevant.One has to either accept that the concept of comparing raw is useful (for any camera) or not accept that and compare the output from the raw converters. Both approaches have caveats so both should be used IMO.
Title: Eyes vs Numbers
Post by: Misirlou on February 03, 2009, 03:32:16 pm
Quote from: ErikKaffehr
Having excess DR, resolution or whatever is of little utility, but can be nice the day you photograph black cats in a coal cellar or need to print really big.

I hate to admit this, but I'd actually love to see a technically excellent photo of some black cats in a coal cellar.

Hey, if nothing else, I'd like to use Bernard's snowscape photo, and the black cats in a coal cellar photo to test the abilities of my printers and monitors.
Title: Eyes vs Numbers
Post by: Panopeeper on February 03, 2009, 03:37:57 pm
Quote from: NikosR
3. On chip 'noise reduction' cannot be called noise reduction.  A better word would be noise avoidance or measures to increase the S/N ratio
I am usually cautious with this term, for most photographers understand "noise reduction" as the function performed by the raw converters or later. However, Canon is calling this "noise reduction"; following is from the Canon EOS 1D Mark III White Paper:

Additionally, the second-generation, on-chip noise removal circuit and the noise reduction technology explained below combine to remove noise effectively.
This has enabled the EOS-1D Mark III to be the first EOS DSLR camera to have ISO 3200 as part of its standard ISO speed range. Dynamic range at low ISO speeds is about the same with the 1D Mark III as it is with the EOS-1D Mark II and Mark II N despite the 1D Mark III’s increased resolution. A second-generation, on-chip, noise-reduction circuit is provided.

The problem is, that most photographers are thinking of the context sensitive function when reading "noise reduction". "Single pixel based noise reduction" would be a correct term, but who would use it, and how many would understand its meaning?
Title: Eyes vs Numbers
Post by: fike on February 03, 2009, 03:44:38 pm
Quote from: Panopeeper
I am usually cautious with this term, for most photographers understand "noise reduction" as the function performed by the raw converters or later. However, Canon is calling this "noise reduction"; following is from the Canon EOS 1D Mark III White Paper:

Additionally, the second-generation, on-chip noise removal circuit and the noise reduction technology explained below combine to remove noise effectively.
This has enabled the EOS-1D Mark III to be the first EOS DSLR camera to have ISO 3200 as part of its standard ISO speed range. Dynamic range at low ISO speeds is about the same with the 1D Mark III as it is with the EOS-1D Mark II and Mark II N despite the 1D Mark III’s increased resolution. A second-generation, on-chip, noise-reduction circuit is provided.

The problem is, that most photographers are thinking of the context sensitive function when reading "noise reduction". "Single pixel based noise reduction" would be a correct term, but who would use it, and how many would understand its meaning?

How is this different than capture sharpening versus print sharpening. Both are sharpening.  You could consider it capture noise reduction (or hardware noise reduction) and printing/display noise reduction.
Title: Eyes vs Numbers
Post by: NikosR on February 03, 2009, 03:54:13 pm
Quote from: Panopeeper
I am usually cautious with this term, for most photographers understand "noise reduction" as the function performed by the raw converters or later. However, Canon is calling this "noise reduction"; following is from the Canon EOS 1D Mark III White Paper:

Additionally, the second-generation, on-chip noise removal circuit and the noise reduction technology explained below combine to remove noise effectively.
This has enabled the EOS-1D Mark III to be the first EOS DSLR camera to have ISO 3200 as part of its standard ISO speed range. Dynamic range at low ISO speeds is about the same with the 1D Mark III as it is with the EOS-1D Mark II and Mark II N despite the 1D Mark III's increased resolution. A second-generation, on-chip, noise-reduction circuit is provided.

The problem is, that most photographers are thinking of the context sensitive function when reading "noise reduction". "Single pixel based noise reduction" would be a correct term, but who would use it, and how many would understand its meaning?

I think that Canon refer to what you're saying as 'noise removal' and by 'noise reduction' they refer to their normal noise reduction for long exposures (dark frame subtraction) and for High ISO (in the jpeg engine).

Regardless, the fact remains that these 'noise removal' techniques cannot be implemented (as far as I understand it) in the raw converter. Thus, any noise reduction that is performed there, be it at the pre or post demosaicing stage is a true noise reduction (context sensitive as you put it). If MFDB manufacturers insist on us performing such a function before looking at their results then one should compare their results with dSLR output having been also de-noised with comparable techniques in the raw converter.
Title: Eyes vs Numbers
Post by: NikosR on February 03, 2009, 03:57:44 pm
Quote from: fike
How is this different than capture sharpening versus print sharpening. Both are sharpening.  You could consider it capture noise reduction (or hardware noise reduction) and printing/display noise reduction.

This is not the correct analogy. What Gabor and I are referring to does not by definition have any adverse effect on the quality of the image data since they are noise removal or noise avoidance techniques at the pixel level. You don't remove useful data in the process of removing noise. You just make sure that the individual pixel level is 'correct'.
Title: Eyes vs Numbers
Post by: fike on February 03, 2009, 04:11:45 pm
Quote from: NikosR
This is not the correct analogy. What Gabor and I are referring to does not by definition have any adverse effect on the quality of the image data since they are noise removal or noise avoidance techniques at the pixel level. You don't remove useful data in the process of removing noise. You just make sure that the individual pixel level is 'correct'.


hmmm...okay.  I can accept that, but I don't understand how any algorithm can be so certain that it is only removing unwanted data--noise.  That is kind of the big problem with signal processing and noise.  It is very difficult to discern what is valid data and what is noise.  There are a lot of images that can really cause problems for noise reduction/removal, etc...:  astronomy, microscopy, macros of sand. Heck, even a highly detailed tile mosaic can conceivable cause significant problems to any noise reduction algorithm.  So, in the end, you are changing the image, although perhaps to make it more appealing to the human eye.  We all know the plastic look of excessive noise reduction as well as the crunchy look of too much sharpening.  Everything we and our cameras do changes the data.

What am I missing here?
Title: Eyes vs Numbers
Post by: NikosR on February 03, 2009, 04:17:28 pm
Quote from: fike
hmmm...okay.  I can accept that, but I don't understand how any algorithm can be so certain that it is only removing unwanted data--noise.  That is kind of the big problem with signal processing and noise.  It is very difficult to discern what is valid data and what is noise.  There are a lot of images that can really cause problems for noise reduction/removal, etc...:  astronomy, microscopy, macros of sand. Heck, even a highly detailed tile mosaic can conceivable cause significant problems to any noise reduction algorithm.  So, in the end, you are changing the image, although perhaps to make it more appealing to the human eye.  We all know the plastic look of excessive noise reduction as well as the crunchy look of too much sharpening.  Everything we and our cameras do changes the data.

What am I missing here?

You are missing the fact that we are not talking about noise reduction the way you understand it, that's why it's no good to call it noise reduction in the first place. We are talking about making sure there is minimum error in the reading of an individual pixel. It is the sum of the errors of individual pixels that make up 'noise' as you understand it. So, in effect, such techniques make sure there is less noise to deal with rather than trying to remove noise that already exists in the context of the image.


PS. FWIW I quote bobn2 from dpreview who gives a conscise description for what a CMOS on chip noise avoidance circuit can amount to (if you can bear reading it):

'The reason that CMOS needs 'NR' circuitry (actually misnamed) is that the random access CMOS cell can't provide a sample of the reset value, which is needed for correlated double sampling to reduced (sic) reset and pattern 'noises'. The NR circuitry is an additional transistor which allows the reset value to be read by the in-cell amplifier before the photodiode reading is made.'
Title: Eyes vs Numbers
Post by: fike on February 03, 2009, 04:49:06 pm
You know, I even catch myself slipping down into the bowels of discussing the camera sensor design and measurement(about which I no very little)  instead of focusing on the camera system.  I think the point of all of this is that it is the subjective results that matter, not abstract measurement.   I needn't care whether there is noise reduction or removal and where in the pipeline it is done.  I should care about the quality of the ouput--prints.

This stuff is interesting, but not really incredibly relevant to determining whether a camera is good or not.  Or, perhaps I should say, can you explain to me why this is useful or relevant information for the pro or amateur consumer to understand?
Title: Eyes vs Numbers
Post by: NikosR on February 03, 2009, 05:01:30 pm
Quote from: fike
You know, I even catch myself slipping down into the bowels of discussing the camera sensor design and measurement(about which I no very little)  instead of focusing on the camera system.  I think the point of all of this is that it is the subjective results that matter, not abstract measurement.   I needn't care whether there is noise reduction or removal and where in the pipeline it is done.  I should care about the quality of the ouput--prints.

This stuff is interesting, but not really incredibly relevant to determining whether a camera is good or not.  Or, perhaps I should say, can you explain to me why this is useful or relevant information for the pro or amateur consumer to understand?


If you're referring to the noise removal discussion, it is not. It is being discussed in the context of Michael's objection to comparing raw data from dSLRs to raw data from MFDBs due to dSLRs purportedly performing internal noise reduction before writing out the raw data while MFDBs do that in the raw converter.

If you are referring to DxO's test (or any other objective measurements) I believe they are relevant because a. may partially assist someone making decisions and b. may assist someone understand why he is seeing or not seeing things and what exactly it is he is seeing.
Title: Eyes vs Numbers
Post by: Mort54 on February 03, 2009, 05:17:05 pm
Quote from: Panopeeper
1. All Nikon DSLR cameras cut off a portion of the pixel data, thereby removing pattern noise (one could not name this "noise reduction" without provocing the international Nikon community to violent responses). All Canon DSLRs keep the raw data "uncut".
I find this comment hillarious, given that the D3X delivers more detail in deep shadows than either the 1DsIII or the 5DII at lower ISOs. So tell me again about cutting off a portion of the pixel data ???? Sounds like a rationalization to me.
Title: Eyes vs Numbers
Post by: BernardLanguillier on February 03, 2009, 05:43:09 pm
Quote from: michael
A lovely image Bernard!

I'd live to see a big print. The web hardly does it justice.

Thanks. The image just underwent a recall though, I found some small issues that need fixing.

I should be able to show you a large print next time you visit Tokyo.

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: Chris_Brown on February 03, 2009, 05:51:33 pm
Michael R.,

Great topic! Thanks for stirring the pot.

For me, a camera is more than a sensor and its supporting electronics. Even though the DxO evaluations are informative, they don't tell us the whole story. Considerations of body design, lenses, portability and software weigh significantly on our choices as photographers (pros & amateurs alike). Some of these factors are given more weight in our decisions than others.

One thing I'd like to point out is that, at present, all camera manufacturers don't seem to be listening to the global user base very well. They seem to be focusing on cost per pixel at the moment, which is quickly coming to a dead end (as Michael's topic illustrates).
Title: Eyes vs Numbers
Post by: jrmintz on February 03, 2009, 05:53:02 pm
Michael's analogy to high-end audio reproduction is interesting. As a producer of music and an amateur photographer I've found many similarities, and one huge difference: while photography is about capture and reproduction, audiophilia (is that a word?) is only about reproduction. There is no link between audio production gear (capture) and reproduction gear analogous to the common calibration of monitors and the profiling of printers by serious amateur and professional photographers. There are no standards like proRGB to calibrate to. You can create a listening area with reasonably flat frequency response through acoustic treatment and electronic EQ, like most good recording studio control rooms. There is still no basis on which to make a judgment of the 'accuracy' of playback equipment, high- or low-end, unless you were at the recording session. If you think your gear sounds accurate, great! I have often encountered great gear in lousy-sounding rooms. The job of the mastering engineer is to make a recording 'translate' to as many different playback situations as possible. The good ones make a bundle and are worth every penny.
Title: Eyes vs Numbers
Post by: Panopeeper on February 03, 2009, 06:02:38 pm
Quote from: fike
I don't understand how any algorithm can be so certain that it is only removing unwanted data--noise
Pls read the Canon White Paper (http://www.usa.canon.com/uploadedimages/FCK/Image/White%20Papers/EOS-1D_MarkIII_WP.pdf)

"Noise" is not a simple concept, though it sounds so. The on-chip noise reduction "cleanses" a single bit. The noise reduction you have to with affects ranges of pixels, based on their characteristics, for example on the irregularities where that is not expected (context sensitive noise reduction).

Quote
This stuff is interesting, but not really incredibly relevant to determining whether a camera is good or not.  Or, perhaps I should say, can you explain to me why this is useful or relevant information for the pro or amateur consumer to understand?

1. This stuff may appear to be irrelevant to you; it is very relevant to me (not the on-chip noise reduction).

2. Michael found it relevant enough to mention in "Eyes vs Numbers". *If* it is mentioned, then it should be correctly mentioned.
Title: Eyes vs Numbers
Post by: Panopeeper on February 03, 2009, 06:05:24 pm
Quote from: Mort54
I find this comment hillarious, given that the D3X delivers more detail in deep shadows than either the 1DsIII or the 5DII at lower ISOs. So tell me again about cutting off a portion of the pixel data ???? Sounds like a rationalization to me.
I am sure you can present raw images demonstrating, that the D3X has the same level (not better) details two stops deeper than the 5D2. Pls use yousendit.com and post here the URL for downloading.
Title: Eyes vs Numbers
Post by: Misirlou on February 03, 2009, 06:06:03 pm
Quote from: Chris_Brown
Michael R.,

Great topic! Thanks for stirring the pot.

For me, a camera is more than a sensor and its supporting electronics. Even though the DxO evaluations are informative, they don't tell us the whole story. Considerations of body design, lenses, portability and software weigh significantly on our choices as photographers (pros & amateurs alike). Some of these factors are given more weight in our decisions than others.

One thing I'd like to point out is that, at present, all camera manufacturers don't seem to be listening to the global user base very well. They seem to be focusing on cost per pixel at the moment, which is quickly coming to a dead end (as Michael's topic illustrates).

Can we say "Mirror lockup button"?
Title: Eyes vs Numbers
Post by: Mort54 on February 03, 2009, 06:18:43 pm
Quote from: Panopeeper
I am sure you can present raw images demonstrating, that the D3X has the same level (not better) details two stops deeper than the 5D2. Pls use yousendit.com and post here the URL for downloading.
I never said two stops better. I just said the D3X is demonstrating better detail in deep shadows at low ISOs than the 5DII is, and no pattern noise to boot. For examples, log onto Digilloyd's subscription site - lot's of real world test images and field shots from both cameras there that compare shadow details.

Sorry, but there's no data to support your contention that Nikon is cutting off pixel data to avoid things like pattern noise. Real world images say just the opposite.
Title: Eyes vs Numbers
Post by: Panopeeper on February 03, 2009, 06:58:38 pm
Quote from: Mort54
I never said two stops better
The DxO report said so. You need to decide if you are defending DxO's statements of you make your own conclusion (pls make this clear in your posts).

Quote
I just said the D3X is demonstrating better detail in deep shadows at low ISOs than the 5DII is, and no pattern noise to boot
You need to learn the very basic of this issue:

no raw file = no proof

You can claim whatever you want to, and I can give less than a fig for that.

Quote
For examples, log onto Digilloyd's subscription site
Only suckers are paying for that amateurish review (this opinion is based on the "free" information of that site).

Quote
lot's of real world test images and field shots from both cameras there that compare shadow details
I am sorry, but I don't have time for analysing lots of shots (made by people, who understand nothing of a review). I spend sometimes hours with a single pair of comparative raw images, if they are worth of analysis.

Quote
Sorry, but there's no data to support your contention that Nikon is cutting off pixel data to avoid things like pattern noise. Real world images say just the opposite
Honestly, I am becoming uneasy with this discussion. How do you come to make such a statement? As the matter of fact, I can prove any time what I stated, but I am a bit selective regarding the purpose of my demonstrations, for I doubt that you would understand anything of that.
Title: Eyes vs Numbers
Post by: ndevlin on February 03, 2009, 06:59:54 pm
I've found that, in digital, the ability to capture shadow detail is of way less importance than the ability to preserve highlights. The reason for this is linked to the 'expose to the right' principle. I want to get my shadow tones (the one's I've decided are the lowest in the scene I care to preserve) as far up the histogram as possible (ie: overexposing them).

A curve can then be applied in post (eg: very simply through the "Blacks" slider or shadow slider in LR). The 'higher' in the exposure these tones are, the better they will reproduce in the final image.  

Of course, the controlling factor is the burn-out of highlight detail. One can only overexpose the image to the point where critical highlight detail can be recovered, making a camera's highlight capture ability paramount to final image quality.

In practical applications, the really low tones get compressed even further in printing and viewing. Not that many images actually need gobs of low shadow detail (I'm talking zone 1 and the space bw zone 1 and zone 2). Nice solid blacks just look good. On the other hand, I find burnt highlights much more aesthetically troubling on a more regular basis.

- N.
Title: Eyes vs Numbers
Post by: douglasf13 on February 03, 2009, 07:02:42 pm
Quote from: Panopeeper
Only suckers are paying for that amateurish review (this opinion is based on the "free" information of that site).

  I am one of those suckers, and unfortunately, I agree  
Title: Eyes vs Numbers
Post by: BernardLanguillier on February 03, 2009, 07:18:45 pm
Quote from: ndevlin
I've found that, in digital, the ability to capture shadow detail is of way less importance than the ability to preserve highlights. The reason for this is linked to the 'expose to the right' principle. I want to get my shadow tones (the one's I've decided are the lowest in the scene I care to preserve) as far up the histogram as possible (ie: overexposing them).

I am not sure to understand what you are saying here.

The ability to preserve the highlights is a matter of correctly exposing. All the DSLRs can preserve highlights when exposed correctly, even a Canon G10 can do that.

What matters then is the amount of noise present in the 1/3 lower bits of the linear raw data corresponding to the tones of the scene that are 7 to 10 stops darker than those highlights you have avoided clipping through correct exposure.

If those bits are clean, then a suitable curve can be applied to transform the linear raw data into a good looking file. If they are not clean, then the curve that has to be applied will produce poor mid-tones and highlights transitions.

So in the end, the ability to produce noise free darker tones is the single most important aspect of a digital sensor.

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: Panopeeper on February 03, 2009, 07:38:13 pm
Quote from: BernardLanguillier
The ability to preserve the highlights is a matter of correctly exposing
LOL, isn't it strange that this has to be said again and again?
Title: Eyes vs Numbers
Post by: Ray on February 03, 2009, 07:38:51 pm
Quote from: Panopeeper
1. All Nikon DSLR cameras cut off a portion of the pixel data, thereby removing pattern noise (one could not name this "noise reduction" without provocing the international Nikon community to violent responses). All Canon DSLRs keep the raw data "uncut". This difference may have caused the ridiculous assertion by DxO, that the D3X has two stops greater dynamic range than the 5D2.

Gabor,
Are you implying that DXO has been able to access the D3X raw data before it's been cut off and that the DR results for the D3X are therefore inflated because they include very noisy lower stops of data that simply cannot be accessed in practice when converting raw D3X files?

Quote
I'm afraid, in the DxO evaluation of the P45+ they used ACR for the analysis; if so, that invalidates all of their findings.

Why do you think there may be a possibility that DXO are using ACR? They analyze the raw data before demosaicing and conversion, don't they?


Title: Eyes vs Numbers
Post by: Panopeeper on February 03, 2009, 07:51:51 pm
Quote from: Ray
Are you implying that DXO has been able to access the D3X raw data before it's been cut off and that the DR results for the D3X are therefore inflated because they include very noisy lower stops of data that simply cannot be accessed in practice when converting raw D3X files?
I am not implying anything regarding the DxO results, except that they had been achieved at incorrectly. If they would publish their sources (i.e. raw files, like Imaging Resources are doing) and if they had published their examt measurements, like I am doing, then and only then we could talk about that factually. Until then their "results" are nothing but opinions in my opinion.

Quote
Why do you think there may be a possibility that DXO are using ACR? They analyze the raw data before demosaicing and conversion, don't they?
I think you should ask this question from DxO. What I do know is, that their results re the P45+ DR and ISO characteristics are telling a tale of their ineptness.
Title: Eyes vs Numbers
Post by: Mosccol on February 03, 2009, 07:54:28 pm
Quote from: BernardLanguillier
I am not sure to understand what you are saying here.

The ability to preserve the highlights is a matter of correctly exposing. All the DSLRs can preserve highlights when exposed correctly, even a Canon G10 can do that.

Hi Bernard

I guess that what he meant related to the now well trodden 'expose to the right' approach: when shooting in RAW (and ONLY in RAW), then overexposing preserves more information in the highlights. It requires post processing (back to the 'left') but seems to be worth it.

Here is Michael's article on the subject:

http://www.luminous-landscape.com/tutorial...ose-right.shtml (http://www.luminous-landscape.com/tutorials/expose-right.shtml)

François
Title: Eyes vs Numbers
Post by: BernardLanguillier on February 03, 2009, 08:00:05 pm
Quote from: Mosccol
Hi Bernard

I guess that what he meant related to the now well trodden 'expose to the right' approach: when shooting in RAW (and ONLY in RAW), then overexposing preserves more information in the highlights. It requires post processing (back to the 'left') but seems to be worth it.

Sure, I am familiar with the ETTR dogma.

EETR can sometimes reduce the negative impact of noisy shadows, and is a way to optimize the available DR of a given sensor, but but the fact remains that what drives the quality of the file in the end is shadow noise.

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: BernardLanguillier on February 03, 2009, 08:07:28 pm
Quote from: Panopeeper
LOL, isn't it strange that this has to be said again and again?

Somehow it is understandable.

- The camera histogram does often not reflect the actual saturation of the photosites - which gives the illusion that highlight recovery is real,
  - Impact of WB,
  - Internal jpg vs raw

- The raw converters often provide the ability to reconstruct some blown channels from non blown ones and surrounding data  - which again gives the impression that highlight headroom is a reality,

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: Panopeeper on February 03, 2009, 08:13:02 pm
Quote from: Mosccol
when shooting in RAW (and ONLY in RAW), then overexposing preserves more information in the highlights. It requires post processing (back to the 'left') but seems to be worth it
The problem of a whole (ok, a half) world of digital photography is compressed in this statement:

what does overexposing mean?

Overexposing does not preserve but destroys information in the highlights, which may or may not be "recovered" (in cleartext: guessed). The question is: when does overexposure start?]/i]

Edit: Bernard, I started writing this post but did something else in the meantime, came back and posted it, then I saw your post; that's the reason of overlaps.
Title: Eyes vs Numbers
Post by: Ray on February 03, 2009, 08:14:12 pm
Quote from: Panopeeper
I think you should ask this question from DxO. What I do know is, that their results re the P45+ DR and ISO characteristics are telling a tale of their ineptness.
 

How do you know that if you don't have the cameras or the raw files to compare?

You probably recall that I went to some trouble in Bangkok comparing the Nikon D3 with my 5D at high ISO. I came to the conclusion that the high-ISO advantage of the D3 was in the order of 1/4 to 1/2 a stop, at various simulated ISOs above ISO 1600. This result seems to correspond with DXO's result which places the DR of the D3 at 1/2 a stop greater than the 5D at ISO 1600 and above.

What interest me is that DXO claims there's an even greater DR difference between the D3 (or D700) and the 5D at base ISO; more than one stop greater DR. This I did not test when I had the opportunity. But I now have the opportunity since I have both a D700 and a 5D.

If I were to repeat the tests in more ideal circumstances and demonstrate that the D700 really is 1/2 a stop better at ISO 1600 and above, and really is a whole stop (or more) better at base ISO, would you accept that DXO results are accurate?  
Title: Eyes vs Numbers
Post by: otbricki on February 03, 2009, 09:05:23 pm
Quote from: John Camp
I honestly don't think this will ever happen. If you look at Sean Reid's review site [a pay site], you'll see that he often refers to a way a lens "draws." So in a review of high-end normal lenses, he gave excellent marks both to a Nikon and a Zeiss 50mm, said either one is capable of doing any kind of professional work, but he said that they draw differently and he prefers the Zeiss. So in this case, part of the "quality" equation is measuring Sean Reid's preferences, which may change from time to time. On the Online Photographer recently, Mike Johnston, who knows a lot about lenses, panned a lens because of its distortions, but actually liked the "look" it produced. In audio, you have large numbers of people who agree (apparently) that some amp/speaker systems sound better than others, despite measures to the contrary. It seems to me that the problem is with the engineers, not with the ears. Engineers have a bias toward measurement, which is a real bias, because if only 20% of a system is currently measurable and quantifiable, it's possible that the other 80% is more important in every way -- yet the engineers will tend to go with what they can measure, because that's their bias. It's the old "bumblebees can't fly" problem -- bumblebees CAN fly, the engineers just didn't have the ability to quantify the characteristics that allowed the bumblebees to do so.

The DXO problem is that they insist that they have a measure, but it's not a real measure. They have a whole bunch of measures, but those measures don't include the whole universe of possibilities, and without the whole universe, you can't reach a valid conclusion. And it may be inherently *impossible* to measure the entire universe of possibilities. So, if a lot of people with good, trained photographic eyes say that a large P65 print is better than a large D3 print, and a DXO measurement says it's not, then the eyes are right, just as they're right when they see a bumblebee fly.

JC

The problem is not that a person "prefers" a certain look or not, or that they have some ability to achieve a holistic view of image quality, but rather when that person is shown images produced by that same process without telling him what the process was, can he reproducibly pick the images that were produced with a given process? Do you really think photographers can look at an image and reliably determine a priori what camera and lens produced that image under normal viewing conditions? My guess is right now the only way to do that would be under conditions where a camera's particular limitations are present - pattern noise in shadows and the like.

It is the same thing with amplifier/speaker preferences. Some people claim to have these preferences, but when they are confronted with a listening test between two measurably equivalent amplifiers where they do not know what the amplifier is they cannot identify which amplifier is in use when the amplifiers are exchanged.

It is only the placebo effect in play here; that any perceived differences are purely the result of psychological bias, rather than some mysterious paranormal sensory ability that transcends measurement science.

I don't think most people have a clear understanding how advanced measurement technology actually is. Today a chemist can detect a single molecule in a sample containing more molecules than there are stars in the universe, and determine its structure in detail at the level of individual atoms; a physicist can measure time to one second in 100 billion years. Mere human sensory ability is effectively infinitely crude in comparison.

In the audio field nobody has been able to hear audible differences in a controlled test that are not explained by electrically measurable phenomena for decades now. There have been offered prizes up to one million dollars for anyone who can do this. These prizes go unclaimed. Unfortunately the charade of the golden ear continues, but it is only that, a charade perpetuated by audio salesmen and high end companies that pander to these beliefs.

There is no reason whatsoever than photographic images should not be subject to exactly the same rules. In fact I think it should be far easier for photographs than audio because the reproduction is a static digital file rather than a transient collection of pressure waves in space.

DXO is off to a start, however there is work to do. I am optimistic that the day will come when measurements will provide a reliable guide to image quality.

Title: Eyes vs Numbers
Post by: Panopeeper on February 03, 2009, 09:18:50 pm
Quote from: Ray
How do you know that if you don't have the cameras or the raw files to compare?

There are tale tell signs.

Quote
If I were to repeat the tests in more ideal circumstances and demonstrate that the D700 really is 1/2 a stop better at ISO 1600 and above, and really is a whole stop (or more) better at base ISO, would you accept that DXO results are accurate?
I can not relate the results of your comparisons to anything; we are comparing the digital images in different dimensions. Your cat shot an ISO 156,800 on a 2310 Mpix 6x6cm camera illuminated by a 60W lamp in the kitchen looks ugly compared to my dog shot at ISO 100 on an APS-C camera in daylight if printed 20mmx30mm on an Epson 9900 printer.  (This was meant to illustrate how I see your comparisons.)

If you shoot a very clean gray scale card, *very* evenly illuminated, underexposed two or three stops (highlights are irrelevant), I can take measurements from those shots (if you throw in a color checker card too, even better). Equal exposure of the shots between the cameras is not relevant; low exposure is essential.

A really exciting shot for measurement looks like this Stouffer wedge (http://www.panopeeper.com/Download/Stouffer_Canon40D_Set1_ISO0200.CR2) (but it is not easy to make this shot correctly).
Title: Eyes vs Numbers
Post by: kwalsh on February 03, 2009, 09:56:16 pm
Quote from: otbricki
I don't think most people have a clear understanding how advanced measurement technology actually is. Today a chemist can detect a single molecule in a sample containing more molecules than there are stars in the universe, and determine its structure in detail at the level of individual atoms; a physicist can measure time to one second in 100 billion years. Mere human sensory ability is effectively infinitely crude in comparison.

In the audio field nobody has been able to hear audible differences in a controlled test that are not explained by electrically measurable phenomena for decades now. There have been offered prizes up to one million dollars for anyone who can do this. These prizes go unclaimed. Unfortunately the charade of the golden ear continues, but it is only that, a charade perpetuated by audio salesmen and high end companies that pander to these beliefs.

DXO is off to a start, however there is work to do. I am optimistic that the day will come when measurements will provide a reliable guide to image quality.

I had something similar to some of your comments to say, but I've got to play devil's advocate here first!

Based on the above snipped quote I think you might be missing a little bit of the point.  The article isn't saying that you can't measure image quality.  The point I believe is that when your chosen numerical metric deviates from what the community views as "good" it means you have failed to come up with a useful metric.  Doesn't matter how accurately you measure it, you are measuring the wrong thing.  Michael gives a number of reasons why he thinks the DXO metric fails the "smell" test from the community.

Similarly, I'll take issue with your comment on the audio field.  I also think the Golden Ears are mostly full of it (was going to be the point of my post) but your post again made me think you aren't getting the right message from Michael's article.  The point is that THD in fact is an awful metric for audio quality and this is well established for exactly the issues Michael outlines (odd vs. even harmonics). Blind studies clearly show that listeners easily prefer an amplifier with a high THD that has even harmonics to an amp with much lower THD that is composed of odd harmonics.  In fact, listeners will tolerate well over an order of magnitude more even harmonic distortion than odd.  These aren't idiots comparing thousand dollar AC power cables and claiming they can tell the difference, they are effects trivial to measure with a spectrum analyzer and easy for an average person to hear.  This issue is that the industry choose a metric that poorly correlated to perceived audio quality and it was obvious to everyone in the community.  There are two separate issues here.  One is crazy folks claiming they hear things that are placebo effects.  The other is engineers telling people one thing is better than another because of some numbers a spectrum analyzer spit out and not realizing their metric was totally screwed up because they had the wrong psycho-acoustic model.

Now - that all said - I was going to make a point about the audio analogy as well.  And that is much along what your comments are with the regards of the crazy Golden Ear set not understanding what a double blind study is and advocating people spend $500 on a wood knob for their volume potentiometer.  If measurements are useless when the metrics they produce aren't correlated to community perceptions then one must be equally careful with the methodology used to gather data from humans.  The Golden Ear's are world renown for their complete failure to understand human subject methodologies.  From all the things I've read from Michael over the years I don't think he suffers the same fault.  For example in his oft cited G10 vs. MF test he presented images blind to his subjects, and he clearly knew the limits of what they were comparing.

Ken

P.S. First post here, short intro.  Photographer, amateur, have shot 35mm and 4x5 film and now all digital.  Tend to do web and modest prints so I stick to entry level DSLRs but still am a sucker for high dollar lenses.  Have always liked the tone of conversions here so I finally registered.  That said, some of the discussion in this thread has started to tend towards the DPRish .
Title: Eyes vs Numbers
Post by: Leping on February 03, 2009, 10:04:32 pm
I tried not to jump to the discussion, since I am trying to finish an essay that in many way echoes Michael's words.  But it is not very often to see level of ignorance in some of the posts.  So, just a very few quick points.

We should all calm down and to try to understand what DxO really said.  There is a wealth of information in the DxOMark site, but poorly organized and presented.  How many bothered to read their technical definitions, formulations, testing methods, and the most important measure, the "3D" "Full SNR" charts, that was inproperly burried deep?

Nikon's clamping the black point at the mean read noise level is not an NR, and does not affect the noise in the shadows, as in the DxO SNR measurements.  The lower read noise of D3x is confirmed by eyes in many carefully conducted tests, when shadows are pushed up 2-4 stops, and the different is anything but subtle.  Weather the SNR over the 0.1% luminance region important is though totally a different question.  If you do not push the shadows up, if you do bracketing on every shots, then the answer is clearly no.  But the numbers and the pushed-up shadows do tell the differences the careful numerical analysis could reveal, even it is only an academia issue.  And the carefully measured and carefully tested did happn to agree almost perfectly.  

Are you aware that, despite of the common quick believe, actually DxOMark revealed even at low ISOs the Canon's noise is better than the D3x's from the midtone up, as noise performance can not be described as a single number even for a fixed ISO?  Did you see DxOMark actually said the 5DM2 is the least noisy camera at high ISO?  DxO does fail in several very important fronts, for example, to address the resolution issue (so the D3/D700 received overall scores improportional to their image quality, mostly the true resolution and the "looking"), to analysis not only the standard deviation but other noise measures such as luma/chroma noise spectrum and energy-spatial frequency distribution, and to intelligently construct sensible performance merit scores out of these raw numbers, which, exactly as MR said, by themselves can be valid while tend to grossly mis-lead (such as the 18% gray SNR).

When DxOMark said they used ACR for the Phase One P45+ analysis?  Calm down, read and think more, understand what the numbers really mean, and do they really help or harm your school of arts.

"Only suckers buy paid reviews", so all the magazine subscribers and buyers are in that category?

Also very interesting to me (but most did not agree), the MF lens quality is a very important contributor to the "MF look" -- my Pentax 67 prime and zoom lense raised my 5DM2 image quality one big step up.

I just want to stop here, as we seem all agree pretty well that DxOMark only provides one side of the story.

P.S.: Happen to be also an audiophile, I fully understand what MR said, and totally love LPs and tubes.  Many great photographers are musicians, Ansel Adams and Charlie Cramer included.  Photography is a form of art, not science about the numbers, although from time to time we need ways to justify the amount of money we ponder weather to spend, so we want to spend time to fully understand some numbers rather than treating them as total nonsense.

Quote from: Panopeeper
I read following statements in Eyes vs Numbers:

I feel following needs to be said, for Michael's statements may be taken by some readers too literally; this is not for or against any camera but for the better understanding of these issues.

1. All Nikon DSLR cameras cut off a portion of the pixel data, thereby removing pattern noise (one could not name this "noise reduction" without provocing the international Nikon community to violent responses). All Canon DSLRs keep the raw data "uncut". This difference may have caused the ridiculous assertion by DxO, that the D3X has two stops greater dynamic range than the 5D2.

2. Noise reduction in the sense most people understand it will be performed on raw data only by Sonys, at least by the A700 and A900. This step could not be disabled with an earlier firmware version of the A700, but now OFF means OFF, apparently.

3. On-chip (on-sensor) noise reduction occurs with all CMOS sensors, which are inherently more noisy than CCD. However, that is single-pixel noise reduction, which has absolute nothing to do with the noise reduction in the sense as it is done in raw processing software or by the mentioned Sony cameras, namely based on the context of pixels.

4. The raw data of the Phase One cameras is far from being really raw.

Finally, there is nothing wrong in analyzing raw data and making certain conclusions based on that; at least I am doing just that without feeling guilty. In fact, one can make certain findings only based on the raw data. However, one needs to be careful with those findings. I'm afraid, in the DxO evaluation of the P45+ they used ACR for the analysis; if so, that invalidates all of their findings.
Title: Eyes vs Numbers
Post by: Ray on February 03, 2009, 10:17:32 pm
Quote from: Panopeeper
There are tale tell signs.


I can not relate the results of your comparisons to anything; we are comparing the digital images in different dimensions. Your cat shot an ISO 156,800 on a 2310 Mpix 6x6cm camera illuminated by a 60W lamp in the kitchen looks ugly compared to my dog shot at ISO 100 on an APS-C camera in daylight if printed 20mmx30mm on an Epson 9900 printer.  (This was meant to illustrate how I see your comparisons.)

If you shoot a very clean gray scale card, *very* evenly illuminated, underexposed two or three stops (highlights are irrelevant), I can take measurements from those shots (if you throw in a color checker card too, even better). Equal exposure of the shots between the cameras is not relevant; low exposure is essential.

A really exciting shot for measurement looks like this Stouffer wedge (http://www.panopeeper.com/Download/Stouffer_Canon40D_Set1_ISO0200.CR2) (but it is not easy to make this shot correctly).

I wouldn't be shooting cats or moving subjects. I'd be shooting the sorts of subjects where, in the past, I've found the 5D to be a bit inadequate.

Here's one of the first shots I took with my 5D over 3 years ago. I was checking for flare from the 24-105 IS lens, which was very much in the news at the time, but instead found banding in the shadows. I believe the shot is a reasonably accurate ETTR, although your analysis would probably find it is slightly overexposed.

The crop is of the bottom right corner of the frame, from a conversion with linear tone curve. As you can see, the image is quite degraded in general, as well as exhibiting obvious banding. I would expect a similar shot using the D700 would show much improvement in those shadows. By comparing bracketed exposure of the same scene, I could work out if the D700 does indeed have a one stop DR advantage at ISO 100, thus confirming whether or not those DXO results have practical significance. I prefer this sort of subject to a Stouffer Wedge   .

[attachment=11356:Moogerah...overview.jpg]  [attachment=11357:Moogerah_5D_crop.jpg]
Title: Eyes vs Numbers
Post by: BernardLanguillier on February 03, 2009, 10:23:14 pm
Quote from: John Camp
Engineers have a bias toward measurement, which is a real bias, because if only 20% of a system is currently measurable and quantifiable, it's possible that the other 80% is more important in every way -- yet the engineers will tend to go with what they can measure, because that's their bias.

The design of a complex system, like a camera or lens, is a team work that involves an iterative process of assessement of achievement vs goals.

For this to work, there is a need to define metrics that are used to set the goal and to measure how close you are to that goal. Not all the metrics represent the preceived value of the customer well, and not all of them are public, but human kind does currently not know any other way to design complex products. This isn't a bias.

One of these metrics could be the average of the 1 to 5 rating that your 20 cousins gives to a sample picture taken with various iterations of a lens, one could be the actual weight vs target weight, the resolution in lines per mm,...

More often than not, some of the metrics used internally by companies are those they expect the customers to look at when trying to assess the value of a product (the famous benchmarks in the computer world). In the present case, the very existence of DxO will influence the camera companies in designing cameras that far well per the DxO metrics.

My personnal view has always been that the most important IP of a company is probably these metrics themselves, meaning knowing what has to be measured that can be measured and that correlates well with real performance as means to measure customer satisfaction. That's what you are talking about when you mention the examples of engineers only knowing 20% of the right metrics.

Back to the initial case, I am not sure that DxO is not representative of what my eyes see. For all the cameras I have had in my hands, I see an excellent correlations between their numbers and my experience understanding the fact that resolution is not a very important part of their work.

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: BernardLanguillier on February 03, 2009, 10:30:20 pm
Quote from: Panopeeper
Edit: Bernard, I started writing this post but did something else in the meantime, came back and posted it, then I saw your post; that's the reason of overlaps.

No problem.

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: ErikKaffehr on February 03, 2009, 10:50:45 pm
Hi,

My view is that we should not look at the DxO mark as a figure of merit but the individual measurements. If we do that and compare the Phase One P45+ with the Sony Alpha it is quite obvious that the P45 has an advantage of say 3-4 dB in all areas at minimum sensivity. The sole reason that the Sony Alpha rates higher is that it has higher usable sensivity, and who would doubt that?

One thing that DxO mark seems to indicate clearly that higher ISOs on the P45+ are not for real, no more amplification just underexposure.

The DxO mark quasi ignores that fact that the P45+ plus has about 60% more pixels, so if the A900 is good for A1 enlargements than the P45+ would be good enough for almost A0 and still offering a better image quality than the A900 at the pixel level.

Maybe DxO is a good thing, if we just ignore the single figure of merit and look at the real data.

If we compare the P45+ with Nikon D3x the Nikon is a little better on noise (1 dB at minimum ISO) but the pixel ratio is still the same.

Best regards
Erik


Quote from: BernardLanguillier
Hello Michael,

Interesting essay, thanks for posting it.

Resolution should indeed clearly be taken into account when rating cameras/backs. But even so, a number will indeed clearly never have the same meaning for different people with different needs and ends up being pretty useless (ideally the rating should be taking into account the print size).

The DxO mark is basically a measurement of sensor quality. Sensor quality doesn't tell the full story, but it is nonetheless relevant. I understand your point about the comparions being perhaps not 100% fair, but the superiority of the A900 along the metrics selected by DxO seems pretty logical when you consider that:

- The photosite size is about the same between 20+ MP DSLRs and 39MP backs (the D3/D700 photosites being significantly larger),
- The DSLRs use micro-lenses which contributes to better light efficiency,
- The DSLR technology is 3 years younger (nearly as long as a whole Bush mandate),
- Sony/Canon are much more talented semi-conductor companies than Kodak with a much longer history, a much more diverse range of application and a much higher number of knowledgeable engineers (and they work more...),
- They probably invest at least an order of magnitude more in fundamental semi-conductor research and tend to pool their resources between companies for activities of common interest.

So there are pretty good reasons to think that the DxO results provide a realistic view of what we have at hand.

As far as trusting one's eyes instead of measurement results, when a G10 is hard to disthinguish from a P45+ in a small print, wouldn't you say on the contrary that the DxO results are a pretty good match to what one actually sees?

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: BernardLanguillier on February 03, 2009, 11:06:36 pm
Quote from: ErikKaffehr
If we compare the P45+ with Nikon D3x the Nikon is a little better on noise (1 dB at minimum ISO) but the pixel ratio is still the same.

Erik,

Yes, true, the P45+ still has more pixels and is the better tool for some really large print applications.

I guess that MF manufacturers must be real glad that Nikon didn't go the MF route... if I were the Hasselblad CEO, I would volunteer 10% of my benefits to Nikon to make sure that they stick to that decision.

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: Ray on February 03, 2009, 11:29:24 pm
Quote from: ErikKaffehr
If we compare the P45+ with Nikon D3x the Nikon is a little better on noise (1 dB at minimum ISO) but the pixel ratio is still the same.

Best regards
Erik

That's right. However, when we look at the results for the normalised size of 8x12" at 300 ppi, which represents a downsampling of both images, the situation is reversed, and the P45+ has a 1dB SNR advantage over the D3X. However, 1dB either way is neither here nor there. For all parctical purposes, one could consider the SNR of the D3X and P45+ as being about the same. It's no wonder Edmund has got himself a D3X   .

I assume that on larger prints that involve a bit of interpolation for the D3X image, the P45+ would also have about the same, and equally insignificant, 1dB advantage, so we are left with basically just a resolution advantage for the P45+, which is not surprising. We don't need DXO to tell us that a 39mp camera is capable of greater resolution than a 24mp camera.

What's surprising is that the DR of the D3X seems to be better than that of the P45+, on both screen and at normalised 8x12" size. It would be interesting if someone could take some test shots with both cameras to confirm or deny this.
Title: Eyes vs Numbers
Post by: BernardLanguillier on February 03, 2009, 11:35:33 pm
Quote from: Ray
What's surprising is that the DR of the D3X seems to be better than that of the P45+, on both screen and at normalised 8x12" size. It would be interesting if someone could take some test shots with both cameras to confirm or deny this.

Quite a few people have done P25+ vs D3x tests, and all those I read reached the conclusion that the D3x had the better image quality overall.

Knowing that there are also many people who feel that the P25+ is better than the P45+ but for the resolution, I still fail to see how the DxO results are surprising and not correlated to real world results. As I have written several time, I have done an informal comparison against the ZD (considered to be in the very same ballpark as the other backs at base ISO) and I prefer the D3x files to the extend that I sold my ZD and full set of Mamiya lenses.

If I am not mistaken, Edmund has a P45+ on Mamiya and he would be in a position to do a comparison between both.

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: NashvilleMike on February 03, 2009, 11:42:06 pm
Quote from: John Camp
(snip...)
The DXO problem is that they insist that they have a measure, but it's not a real measure. They have a whole bunch of measures, but those measures don't include the whole universe of possibilities, and without the whole universe, you can't reach a valid conclusion. And it may be inherently *impossible* to measure the entire universe of possibilities. So, if a lot of people with good, trained photographic eyes say that a large P65 print is better than a large D3 print, and a DXO measurement says it's not, then the eyes are right, just as they're right when they see a bumblebee fly.

JC

I agree with John here, and I like the analogy with high end audio as well.  I'm an ex-audiophile, and was "in the business" earlier in my days, so I'm familiar with that "lifestyle" (or would that be "obsession" lol) as well as anyone and there are some definite parallels with this topic.

A few comments:

Today's society wants it easy and is used to getting things quickly. I remember (as a kid) the 70's, so I've seen the change.
Today we've got your instant coffee, a quick-start breakfast,the  morning snack from the row of vending machines down the hall, fast food lunch, and a microwave dinner, all eaten far too quickly as we accelerate to an early grave (or a dozen prescription medications at the least) as a result of our stress, lack of exercise, and poor nutrition.

Social commentary aside, the whole "what camera do I buy" / "what lens do I buy" process has changed as well. Today we've got the internet, the old school magazines, cameras being sold by typically less-than-knowledgeable clerks at big box retailers, a few camera stores here and there, and much, much, much more marketing and advertising being shoved at our ears and eyeballs. Most of that wasn't around in the "old days", you know, back when we had to trudge our 600mm lenses to school up hill both ways through the snow and all that.  

Salespeople in the 70's were usually trustworthy to some point while these days it's not common for me to hear in my local photo shop (said to a new buyer) "Oh, get brand XXXXXX, it's much better than brand YYYY" simple because there is a spiff (incentive payment to the salesperson) on that brand or product or a push of the sales clerks own bias, which may very well have been formulated by exposure to the marketing and a need to defend their own brand choice. I may be wrong, but my memories were that people weren't anywhere as defensive about their brand choice "back then" as they are now. Well, those days are gone, and the consumer is getting a lot more information from various sources aimed at them all at once, and understandably wants to make a "correct" purchasing decision. In 1970 you maybe talked to a pro you knew, talked to your salesperson, asked a friend, looked at an advert, and off you went. Now you've got half a dozen web sites and forums that discuss things subjectively, another half a dozen that do "objective" tests, plus the sales folks, friends, more advertising and marketing, and often it's hard to *quickly* get a clear message that screams "hey, Mister Consumer - obviously the Plektomatic Q45XDR-III 17.3 megapixel DSLR is the perfect camera, get it".

So in today's world, what's the holy grail?  The single all encompassing quality grade that covers all bases, is totally defensible, and can't possibly be wrong. I'd argue, as I think John would, that this is impossible, but it is the holy grail. If there could be one test site that totally was beyond any doubt the sole deity, the supreme being, and had "the grade", then the decision process would be really, really simple. However, real life isn't always simple, and due to, as John points out, the vast universe of possibilities that determine what quality is, I seriously doubt it's going to happen, ever. But people want this easy number, because it relieves them of the far more time consuming option that I think today provides a better answer: a combined analysis of multiple subjective reviewers, a reading to understand each reviewers biases and tastes, an overview of the objective sites (plural, not just DXO or photozone or whatever in isolation, but a look at the results across a group of multiple test sites) and then some discussion with folks who own them, and if possible, renting of the proposed purchase to evaluate it. This takes time, and it takes effort, and it takes admitting that something that measures well might not actually shoot well in some situations, or perhaps, the things the photographer values highly in the complex interrelated object called "image quality" may be weighted differently versus something that is made to shine on a test chart.

Now, does this mean measurements have no place? Certainly not. I think what I'd personally like to see in the future is an *exploration* of how to quantify what we might see in terms of "image quality" beyond the current test methods. We could, for example, look at lens performance. Many folks only look at resolution (via photozone or slrgear or whatever) or an MTF graph for "the answer". Yet lens designers themselves (like the fellow from Zeiss in his MTF white paper and Dr. Caldwell, he of the famed Coastal 60mm Macro) have said MTF doesn't provide all the answers. So perhaps more research could be done in these areas - trying to correlate measurements with visual performance (not just lenses, but camera bodies too). Offhand, I don't think only evaluating sensor data exclusive to the raw conversion really helps us much here, but that's must my opinion.

I don't think we'll ever be able to fully quantify image quality, maybe not in my lifetime or the next generations, but I think it would still be valuable to at least try and make some progress. To circle around back to the high end audio analogy, Stereophile always has measurements and commentary on them along with the subjective review of the gear itself, and I get the distinct impression that the crew at Stereophile is very much aware of the differences between good measurement and good sound and tries to figure out why this is so. And perhaps that's what is needed "down the road" with photographic gear - more discussion of both subjective and objective together, with some dialog as to *why* something that scores .00002 geek-o-units of noise may not produce visible image quality as well as something that scores .000034 geek-o-units, or why a lens that knocks one out of the ballpark on photozone just doesn't draw that well when faced with a three dimensional object in the real world.

Like others, I do worry, that in the same manner that so many people use one and only one "objective" lens test site to judge lenses (even when research indicates divergent opinions to that site) that the DXOmark becomes the one and only factor, simply because it's an easy to digest number. Like I stated earlier, careful analysis of the wider spectrum of opinions, always taking into account reviewer bias as well as an admittance that "objective" tests may not be perfect, is a better method at this time.

As an aside, I'd also feel that a lot of people who make proclamations about image quality or lens sharpness (or whatever) need to include the *magnitude* of the difference. A lot of times I think people end up chasing the wrong thing - a guy who drops 8 grand on a D3X to shoot landscape, but then uses a 24-120 average-at-best lens on gitzo 1 series tripod is going to get creamed by a guy with a D2X, 100/2 Makro Planar and a stable support system. My own approach has always been lenses first, then bodies, and I've of late come to realize the importance of the support system as well. Bodies come and go, as will the arguments in the forums as to which one is better, but if one acquires a set of lenses that perform well and (importantly) have a "drawing" style that one feels suits their subjective tastes, they're off to a heck of a good start.

-m




Title: Eyes vs Numbers
Post by: ErikKaffehr on February 03, 2009, 11:43:59 pm
Bernard,

Sorry, my reply was not really a reply to your posting. I just choose to reply to your posting while it was the first one the thread. I decided to repost it as a new topic.

Mostly I agree with your postings and you make great pictures.

Best regards
Erik


Quote from: BernardLanguillier
Erik,

Yes, true, the P45+ still has more pixels and is the better tool for some really large print applications.

I guess that MF manufacturers must be real glad that Nikon didn't go the MF route... if I were the Hasselblad CEO, I would volunteer 10% of my benefits to Nikon to make sure that they stick to that decision.

Cheers,
Bernard
Title: Eyes vs Numbers
Post by: NikosR on February 04, 2009, 01:13:02 am
Quote from: Panopeeper
I think you should ask this question from DxO. What I do know is, that their results re the P45+ DR and ISO characteristics are telling a tale of their ineptness.

I think that DxO state clearly on their site that they test raw data and analyze them with their proprietary (but commercially available) DxO Analyze tool. How did ACR come into this discussion?
Title: Eyes vs Numbers
Post by: Panopeeper on February 04, 2009, 01:41:35 am
Quote from: NikosR
I think that DxO state clearly on their site that they test raw data and analyze them with their proprietary (but commercially available) DxO Analyze tool. How did ACR come into this discussion?
I was only speculating about the reason their analysis of the P45+ is so thoroughly wrong. Someone mentioned, that DxO (the raw processor) does not support MFDBs; but how would they feed their DxO Analyzer?
Title: Eyes vs Numbers
Post by: NikosR on February 04, 2009, 02:05:07 am
Quote from: Panopeeper
I was only speculating about the reason their analysis of the P45+ is so thoroughly wrong. Someone mentioned, that DxO (the raw processor) does not support MFDBs; but how would they feed their DxO Analyzer?

If you trust their description of DxO Analyzer it includes 'Automatic high-speed processing of RAW and/or RGB data for either camera or component level analysis'. Now, it's anybody's guess what is the relationship between this and their DxO Pro converter software.

So, sort of engaging with DxO in a direct discussion we will never know exactly what they're doing.

Gabor, you seem to have some partially documented objections (valid or not) regarding their results. Have you tried to directly contact them and challenge them? This might be more constructive (or not) than moaning in internet forums with any chance you're given. You should be able the speak the same tech language with them so they might engage to some extent. I would be interested to know their response.
Title: Eyes vs Numbers
Post by: BernardLanguillier on February 04, 2009, 02:12:38 am
contact@dxomark.com
Title: Eyes vs Numbers
Post by: Ben Rubinstein on February 04, 2009, 04:50:16 am
I think that this discussion and the two essays are perhaps interesting on an amatuer level or to rich hobbyists. For a pro given comparable image quality (though if you've needed FF or good high iso noise that means that the Nikon only just got into the game) then first and foremost as a choice will be system. If one works for you or provides for you then the feel of a camera or the availability of specific lenses is going to be far more important than any of this nonsense. Secondly that the 5Dmk II or Sony provide equivalent image quality is as irrelevant to a working pro as it was when the playing field was level for all cameras in the film days. No one suggested that as you could use an Elan 7, an EOS 1v was then a niche product! The difference is just as striking if not more so today. If you need weather sealing, dual card slots, pro AF and 20+ megapixels then there is only two choices and the difference is one of system not some pathetic wrangling over points of a percent. More than that, the 5D mkII and Sony are then utterly irrelevant cameras. This discussion and the assertion that these new cameras in some way are game changers is only relative to rich hobbyists and not to pro photographers!

That anyone can justify a P65+ without having a full economic business case to justify why the upgrade would be necessary is an idea that is horrific to me but more than that, if it's just hobbyists then their decision is irrelevant to the rest of the world. They can go back to fondling their uber expensive gear while working pro's will buy equipment that is both relevant and justifiable. If MR is right and the playing field has been leveled, then for a pro the only considerations now are what the pro choices are. Expensive toys or super megapixel amatuer cameras are now as irrelevant as they always should have been but unfortunately have not to date due to the ongoing digital odyssey forcing amatuer cameras into relevance.

As to the numbers, when the numbers are that close you have to make the decision based on what you want from a file. The megapixels from a 5D and D700 are the same but I find the 5D files to be much sharper out of camera. For all the talk about the Sony, the RAW files I've worked on show noise comparable to my original 1Ds at iso 200 and that is very bad. Could a different workflow show other results? Yes. Is that relevant to what I want? Not in the slightest. If I don't like what I see then screw what anyone else says, screw the test results, I'll keep to what I like and am certainly mature enough not to bother arguing with someone else about if my taste is better than theirs.
Title: Eyes vs Numbers
Post by: 32BT on February 04, 2009, 05:52:01 am
Quote from: pom
I think that this discussion and the two essays are perhaps interesting on an amatuer level or to rich hobbyists. For a pro given comparable image quality (though if you've needed FF or good high iso noise that means that the Nikon only just got into the game) then first and foremost as a choice will be system.

...

 If I don't like what I see then screw what anyone else says, screw the test results, I'll keep to what I like and am certainly mature enough not to bother arguing with someone else about if my taste is better than theirs.

While I mostly agree with what you write, I am not entirely sure whether a working professional can actually avoid that arguing. Considering that most professional photography is commissioned by an art-director or by a client, both of which I believe have a steadily degrading sense of quality, I'm convinced that part of the problem is in the justification of the diminishing returns of "higher quality" systems.

We live in a world where both the speed and the quantity of information is steadily increasing. Since the quality of information is inversely proportional to quantity and speed, we will be confronted with new generations that won't care less. I believe this is one of the reasons of the demise of the music industry: eventually the amount of music produced and its fleeting character will result in people not caring to pay for this productivity. They will not consider "ripping" to be a crime etc...

I also see this in television production. We now are gradually transitioning to HD television (in Europe), yet the picture quality of even reputable screenings is sometimes appalling. You can clearly see that some el-cheapo camcorder was used by the extreme chromatic aberration visible, but because of the increased number of scenes per second, viewers likely won't care less. (Also noticed that it is all the rage these days to move the camera? And if the camera didn't move, they will move the picture digitally?).

Point is: while the reproduction media steadily increases in quality, the production of content seems to steadily decrease in quality, primarily triggered by an oversaturated consumer who does (or eventually will) not care. And in between you, the working professional, and the consumer, stands an art-director who is either him/herself unable to properly judge quality, or is simply hired to represent a consumer that doesn't require quality.

Note that I mean this in the general sense: a wedding jockey could suggest that they try to capture a special event and abide by their own standards of quality, but I would tongue-in-cheek like to counter that with the argument that these days people will likely mary more than once in their lifetime and value their uncle who fools around with the video capability of his 5d2 more than the stills shot by the wedding photographer and his mfdb...

And by the way, this is probably the reason for the possible success of integrated video capability in photocameras which are obviously ergonomically disastrous devices to try and shoot video. But the consumer wants, nay, expects the scene to shake, move, and rattle these days, so there you go. You would think that in-camera shake reduction a-la Sony would be a great tool coupled to video capability, and a clear advantage over Canon, but alas...
Title: Eyes vs Numbers
Post by: Ben Rubinstein on February 04, 2009, 06:43:11 am
I agree with every single sentence you've written! However I feel that it further drives the argument that the time to compromise on the camera for the sake of the chip is over given a certain plateau reached in image quality as driven by client expectations and as such pro AF is much more important than a possible 1/2 stop gain in DR or whatever. I do not believe that definitive statements about the industry can be made based on the requirements/wishes of the rich hobbyist sector for whom rationality in purchases has always had a very loose connection with the real world. Arguments about degrees of IQ when cameras already have sufficient for the chosen client expectations is indeed only the realm of those who have the free time to argue such irrelevancies while pros are choosing ergonomics and lenses. The realm of those fondling their cameras but not having to get a paid job done right if they want food on their table the next month.  A pro will choose an amount of megapixels needed for the job (subjective but that's fair) whereas the hobbyist will always want more independent of necessity, the 5D mkII is a perfect example. A crippled abortion of an upgrade which is still sold out most places...

Again, not to belittle what is most certainly the most profitable sector of the camera market however saying that pro level cameras are now being pushed towards niche is ignoring the entire pro sector for whom they were made. As there is no rationality to hobbyist purchases I don't think that a study of the same can in any way be definitive to anyone but marketing departments.
Title: Eyes vs Numbers
Post by: Chris_Brown on February 04, 2009, 09:19:07 am
Quote
One thing I'd like to point out is that, at present, all camera manufacturers don't seem to be listening to the global user base very well . . .
Quote from: Misirlou
Can we say "Mirror lockup button"?
Can we say "WiFi connectivity using 802.11n protocols"?
Title: Eyes vs Numbers
Post by: ndevlin on February 04, 2009, 11:34:12 am
QUOTE (BernardLanguillier @ Feb 3 2009, 04:18 PM)
The ability to preserve the highlights is a matter of correctly exposing

<LOL, isn't it strange that this has to be said again and again?>



My point is that this is true, but doesn't answer the question. If you accept that ETTR is correct (and frankly, those who practically invented digital photography think it's elementary), then the "best" exposure is the highest exposure you can have without loosing highlight detail.  This expsoure is best because it puts the most real data in *all* tonal zones, but for purposes of this discussion, especially in the shadow tones.

In a perfect world, my desired zone 2 data would be two thirds of the way up the histogram! I can then put it in its place, according to my creative vision of the image.  Would I could do this, there would be no noise to speak of in what ends up being the low shadows of the finished image.

Of course, this isn't possible with anything but static subjects and multiple exposures.  In most real world situations, the limiting factor for how much data I can get in my desired shadow tones is how high I can push the highlight zones without losing them.   Therefore, the functional limiting factor of image quality is actually highlight preservation.

Similarly, none of this applies to jpegs (none of this conversation is really relevant to jpeg applications anyway, given the crudity of in-camera post), and to dynamic, lower-light situations. There, I agree, clean shadow detail is king.  But that's not really the landscape context at all, but rather the terrain of photojournalism and wedding photography.

We may well be saying the same thing, just in different ways.

- N.
Title: Eyes vs Numbers
Post by: sreidvt on February 04, 2009, 12:21:54 pm
Quote from: John Camp
I honestly don't think this will ever happen. If you look at Sean Reid's review site [a pay site], you'll see that he often refers to a way a lens "draws." So in a review of high-end normal lenses, he gave excellent marks both to a Nikon and a Zeiss 50mm, said either one is capable of doing any kind of professional work, but he said that they draw differently and he prefers the Zeiss. So in this case, part of the "quality" equation is measuring Sean Reid's preferences, which may change from time to time. On the Online Photographer recently, Mike Johnston, who knows a lot about lenses, panned a lens because of its distortions, but actually liked the "look" it produced. In audio, you have large numbers of people who agree (apparently) that some amp/speaker systems sound better than others, despite measures to the contrary. It seems to me that the problem is with the engineers, not with the ears. Engineers have a bias toward measurement, which is a real bias, because if only 20% of a system is currently measurable and quantifiable, it's possible that the other 80% is more important in every way -- yet the engineers will tend to go with what they can measure, because that's their bias. It's the old "bumblebees can't fly" problem -- bumblebees CAN fly, the engineers just didn't have the ability to quantify the characteristics that allowed the bumblebees to do so.

The DXO problem is that they insist that they have a measure, but it's not a real measure. They have a whole bunch of measures, but those measures don't include the whole universe of possibilities, and without the whole universe, you can't reach a valid conclusion. And it may be inherently *impossible* to measure the entire universe of possibilities. So, if a lot of people with good, trained photographic eyes say that a large P65 print is better than a large D3 print, and a DXO measurement says it's not, then the eyes are right, just as they're right when they see a bumblebee fly.

JC

Hi John,

Of course, the least important aspects of my reviews are my opinions, per se. The most important aspects are the observations and examples. From those, one should be able to draw some of his or her own conclusions. So, for example, in the recent review of the ZE 50/1.4 and Canon 50/1.2 L the core elements of the review were the examples and observations about how both of those lenses draw. Whether I personally prefer one or the other is almost incidental. Frankly, I think that distinction applies to all reviews. For example, it's worth knowing if a lens is prone to veiling flare and shows lower contrast as a result. Whether or not I personally might want to use that lens for certain work is mostly anecdotal.

Our job, as reviewers, is, first and foremost, to describe. That's also why rankings of cameras and lenses are always suspect. How can a reviewer know what the exact needs and priorities of a given reader-photographer are? And lacking that knowledge, how can he or she say what is "best"?

Responding to DXO's approach overall would take more time than I can give right now but I think that it is important to remember that data presented in numbers is not necessarily any more objective or accurate than data presented in other ways. DXO has to make certain decisions about what they want to measure and how. Measuring high ISO noise, for example, is certainly not cut and dry because luminance noise affects picture differently than chrominance noise, shadow noise can look different than mid-tone noise, subject lighting type can affect noise, etc. The potential dangers, of describing picture quality in numbers, are over-simplification and over-generalization. A given testing path and set of decisions leads to a certain number. But what that number tells us, really, is just something about what's at the end of that specific path. So some people's experiences with a given camera may jibe with DXOs results and some may not.

I think that the real measure of a digital file is not only what it looks like (after a RAW conversion for example) but also how malleable and resilient it is during post-processing. Some digital files fall apart quickly as one tries to change them from their initial form to the form they take as a source for a final print.

Some other thoughts, none of which is likely to surprise you.

One is that I believe that the way a camera works as a tool and, especially, the way it allows one to see the subject can be as or more important than "file quality" per se.

The second is that, in the digital age, we are often overly concerned with technical performance. As I think over the history of photography and the work that has affected me the most, very little of it has been done with cameras that were (in their time) at the apex of technical quality. There's Strand's, Evans' and Atget's work with view cameras but none of those pictures is great because of the camera, lens or film quality. When I worked with large format, which I did for a long time, it was as much for the seeing/working process it provided as for any technical advantages it had.

Most of the photography that has affected and influenced me personally has been made with small format cameras using whatever film the photographer needed for the lighting conditions. Very little of it was made with low ASA film (except for work done when low ASA was all there was).

So I guess my feelings about this can be summarized in this way.

1) DXO tests show us certain kinds of measures of certain aspects of camera performance. They're a set of specific paths followed to their natural ends. Generalizing those results beyond that is problematic.  

2) Once a camera's technical output is sufficient for what a photographer wants to make (and we all have different needs and tastes in that respect) technical performance differences can become red herrings that distract us from the much more important and difficult process of making a strong picture.  I report on various kinds of technical camera and lens performance in reviews because a professional review essentially requires that. But I try to never confuse resolution, contrast, CA, etc. differences with photography itself.

Cheers,

Sean
Title: Eyes vs Numbers
Post by: Mort54 on February 04, 2009, 09:09:38 pm
Quote from: Panopeeper
The DxO report said so. You need to decide if you are defending DxO's statements of you make your own conclusion (pls make this clear in your posts).
I never said I was defending DxO's statement. I could care less about DxO's statement. I never once even brought up DxO, or mentioned it in any way. You're the one who's bringing DxO into this. Let's TRY to stick to the facts here, shall we.

Quote from: Panopeeper
You can claim whatever you want to, and I can give less than a fig for that.
You've offered not a shred of proof to back up your statement re Nikon chopping off bits, and YOU expect anybody to give a fig about what you say? Again, let's try to stick to the facts here.

Quote from: Panopeeper
Only suckers are paying for that amateurish review (this opinion is based on the "free" information of that site).
At least he's tested both the D3X and 5DII side by side, and provided the RAW files for download so subscribers can draw their own conclusions, which apparently is a lot more than you've done. Have you even touched either camera? The free information is a tiny summary of all the work he's done. For you to dismiss his review without actually looking at his review tells me all I need to know.

Quote from: Panopeeper
I am sorry, but I don't have time for analysing lots of shots (made by people, who understand nothing of a review). I spend sometimes hours with a single pair of comparative raw images, if they are worth of analysis.
If you don't have time, then why did you bring it up :-)

Quote from: Panopeeper
Honestly, I am becoming uneasy with this discussion.
What discussion. You're the one who's ranting and raving here. I'm just trying to educate you.

Quote from: Panopeeper
How do you come to make such a statement? As the matter of fact, I can prove any time what I stated, but I am a bit selective regarding the purpose of my demonstrations, for I doubt that you would understand anything of that.
Really, then prove it. I've told you where you can find proof regarding my statement. You've done nothing but hand wave and blow smoke. Regarding what I do and don't understand, I do understand when someone is full of it.
Title: Eyes vs Numbers
Post by: JRSmit on February 06, 2009, 06:26:20 am
Sean,

I think you make a good statement, it comes down to what a person perceived with the word quality. In Europe ISO9000 forced involved to define the meaning of quality. I got involved in the period when the Berlin wall opened up, and all of a sudden the trabant car became on of the icons of that period in our history. We had an discussion on quality and someone asked: which car is the best quality: mercedes or trabant. Invariably most if not all would initally say without hesitation: Mercedes. However after discussion and agree on what is quality, the outcome is quality is a measure of fullfilment of one's need. Then a trabant can be of better quality than a mercedes.

Regards,


Jan R.