Luminous Landscape Forum
Equipment & Techniques => Cameras, Lenses and Shooting gear => Topic started by: ErikKaffehr on May 09, 2014, 12:29:35 am
-
Hi,
DxO mark takes a lot of bashing, which is seldom justified. DxO makes a commercial raw converter and they essentially share some of the data they acquire for each sensor. Sharing information is normally regarded a good thing, except in China and some other countries.
There are a few things to keep in mind regarding DxO-mark for sensors:
- It is based on raw images
- It is mostly about image noise
- DxO mark essentially ignores pixel count (data are normally normalised to 8 MP, but this is largely irrelevant)
- DxO mark measures colour depth but not colour fidelity (well, they measure SMI but it is not a part of DxO-mark)
The DxO-mark is based on three factors:
- Dynamic range, which weakly favors large pixels
- Colour depth, which is the number of colours the sensor can reproduce
- Low light ISO, the highest ISO setting that gives a good image, this also weakly favors large pixels
A key difference of the ISO rating is that it favors cameras with variable pre amps, as it ignores DR. This is the reasons Canons shine in this area.
Sensor area influences DxO-mark heavily, doubling the sensor area probably adds something like 10-12 points to the total DxO-mark.
None of the top eight cameras are "high ISO cameras", one of the top eight is the Phase One IQ 180. The others have all Sony Exmoor sensors and deserve their position mostly due to DR (dynamic range).
In the colour depth category the Phase One cameras dominate, all first three place go to Phase One. This is more due to the sensor size than CMOS vs. CCD or CFA design.
Once DxO-mark tests the IQ-250 it will very probably go to the top, by a wide margin. It is using a larger version of the Sony Exmoor sensor. The new RED Dragon sensor may be higher in rating, though, but it is probably not regarded to be a still camera.
Best regards
Erik
-
Erik
You make a huge amount of posts on the technicalities of this great user of our time and money, but there are as many different types of photographer as their are manufacturers, resources and styles. Bashing a resource is as absurd as defending it in my opinion, it's an organisation that runs a very specific set of tests to produce a very specific set of results that are important to a specific set of people.
My opinion, for what it's worth, is that photography in the main is a creative process, the look, message, style of any image produced is down to the vision of the photographer using the equipment at his or her disposal. The results from DXO testing give a very very small amount of important data to help photographers who value that information make decisions on what they want to buy, to others it means nothing. A lens is not great because of a high DXO mark, a camera is not great because it has a high DXO mark, they just have specific qualities that meet DXO's testing standards, there is so much that is not shown by these sorts of tests.
So, don't take it to heart that people don't take the same amount from these tests that you do, it's important to the people it's important to, you may think people are crazy for bashing the information, others may think you're crazy for showing charts when talking about a lens rather than beautiful images made with it, it's all good just different people with different requirements.
Enjoy your photography!
Mat
-
DxO need to explain how they test more so people can understand it.
As with any test there is a margin for error.
My view is that is has limited use in the field for photographers so I don't really take it that seriously.
It's a bit like a computer benchmark..that's not always a great indicator of real world performance.
-
Thanks for the sum up. In my (simple) mind, what I take from this is that larger sensors naturally "win" in terms of dynamic range and colour depth. This is important to me, as I shoot a lot of landscapes-seascapes. And that is why I still use a FF DSLR for such subjects.
Rgds
-
Eric, I use the DxO software as the my basic processing software however their testing means very little to me as I rate colour fidelity and MP as two of my basic starting criteria. After all what the camera can produce is the limits of its performance and it is all down hill from there.That is why Canon produce better results in the tests than in my world.
I am using these as the starting points along with micro contrast for the inter-reaction between my cameras and lenses and the real world conditions that I have to actually take photographs in.
The DxO way to me is the same sort of technical mumbo jumbo as the ISO criteria for film speed.
-
Thanks for the sum up. In my (simple) mind, what I take from this is that larger sensors naturally "win" in terms of dynamic range and colour depth. This is important to me, as I shoot a lot of landscapes-seascapes. And that is why I still use a FF DSLR for such subjects.
Rgds
Depends which full frame you use. If it's Canon then there are plenty of APS-C DSLR's with better dynamic range.
The problem with DxO is putting into perspective. numbers rarely mean much.
For example how to they come to their low light ratings? Image quality is subjective and most of us would have a print size target (at least I do) for high ISO images. But quality can vary here, IQ degrades how do DxO come up with these numbers?
It's all a bit vague for my liking.
-
I think DxOMark is one of many tools used to make a "smart" purchase
Before buying a camera or lens one reads dpreview, photozone etc to be informed.
Just one more intelligent metric to make an intelligent purchase decision.
Marc
PS: I'm going to purchase a new car, I read technical reviews, I read driving and functional reviews then I go to the dealer for a test drive and make a "smart" decision
So I decide on a Mazada Miata with skinny tires because it is fun to push on back roads at rational speeds over a Porsche 911 GTB. The Porsche came out high in the technical reviews, the Miata high in the functional reviews but at least I read the reviews and test drove the cars before writing the check!
-
Depends which full frame you use. If it's Canon then there are plenty of APS-C DSLR's with better dynamic range.
The problem with DxO is putting into perspective. numbers rarely mean much.
I use a Canon 6D and I am quite happy with it. Previously, I have used a 5DMKII. Never had a shot ruined because of lack of DR, but I do take some time to optimise exposure on the field, even sometimes using GND filters to balance "sky and land", or "sky and sea". I am currently working on some photos I took in the SW Portugal coast, these are sunsets over the sea, with some coastal cliffs as foreground. I had to use a GND to hold down the sky a bit, I suppose I am "old school" in that regard. Never had to up the shadows in post by much, so no problems there.
I have experience with good APS-C sensors, like Pentax K5 and Fuji XE-1 or X-Pro1, or X100, I have used them in the past too (still use the X100 for travel). What I can say is that with these cameras, I had to resort to highlight recovery sooner and more often, in Lightroom. The Canon 6D files have more room to play with. Are there better sensors out there for FF? I believe so, for sure, but the Canon FF is "good enough to me".
Rgds
-
IMO it's important not to succumb to simpletonistic interpretations of the info provided by any testing process. Personally I'd prefer that DxO not provide a single-number score for people to glom onto. But since they do you've gotta go past that and look at the more in-depth analysis. I consider their data useful but not determinative...as Marc observes, after the reading comes the test drive.
Sometimes I'll end up prefering a less technically capable camera over a more capable one due to ergonomic issues. Or maybe a "lesser" camera effortlessly delivers a pleasing or distinctive look that I have to wrestle with a "better" camera to achieve. A single number isn't gonna tell you about that stuff. DxO themselves are surely aware of this...sometimes I wonder, though, about segments of their readership.
-Dave-
-
I think DXOMark provides some useful information for those who are interested althought I too am not keen on their scoring systems. I'd prefer to make my own mind up based on the measurements. I like their sensor information, not so keen on their lens data. I can't fathom their Perceptual Mpix concept and would prefer to see basic mtf50/lwph info across the frame - just like the way dpreview used to present their data.
Dave
-
Hi,
I pretty much agree with your view. I like the data but I feel the skeptical about the scoring system.
Best regards
Erik
I think DXOMark provides some useful information for those who are interested althought I too am not keen on their scoring systems. I'd prefer to make my own mind up based on the measurements. I like their sensor information, not so keen on their lens data. I can't fathom their Perceptual Mpix concept and would prefer to see basic mtf50/lwph info across the frame - just like the way dpreview used to present their data.
Dave
-
The scoring system is exactly the problem - guided, as it is, by some arbitrary and undisclosed weighting of the average of the three sub-scores it produces, a factor which unquestionably introduces an (unintentional? maybe) bias in favour of cameras that do well in one particular metric.
The tests themselves are inherently skewed in favour or specific use-cases which might well be entirely unrepresentative of the use to which many of us put our cameras - the very fact that many of their tests and their sensor scores are weighted towards 100 ISO, when many of us never use our cameras at 100 ISO, is a bias.
And as to lenses: did you know that the Canon 50mm f/1.8 II is a "better" lens than the Canon 600mm f/4 II based on DxO's scoring?
It's also true that DxO has made verifiable mistakes with its tests, but instead of publicly admitting to these errors, have quietly edited/fixed them unannounced, which is not very "scientific" for a company with the phrase "image science" in its logo tagline...
But the real problem with DxO's scores of course, is the use to which they're put by trolls trying to "prove" that their choice of camera is superior to another's, because DxO has scored its sensor higher.
Thanks for the sum up. In my (simple) mind, what I take from this is that larger sensors naturally "win" in terms of dynamic range...
No.
As Barry has pointed out, this is not intrinsically true, because it ignores improvements in sensor technology's "state of the art" - in the Real World, recent crop cameras often significantly outperform older full frame sensors in terms of dynamic range (shadow DR, anyway, which is all that anyone seems to be interested in these days) and high ISO noise handling.
-
- Low light ISO, the highest ISO setting that gives a good image
Huge problem right there, Erik: a "good" image by what criteria? To what standard?
A perfect example of the arbitrariness of the whole DxO process; and given that you can't have a good - or bad - high ISO image until the Raw data are actually converted into an image (and the conversion stage is vital to the end result, as we'd all agree), a completely misleading and disingenuous false equivalence - you cannot establish, purely from analysing Raw data, whether there's a good high ISO image in there.
I find this point particularly ironic, given just how damn' good DxO's Optics Pro is at converting high ISO images - DxO knows it's not telling the whole "low light" story here.
-
Hi,
Raw images have measurable noise, just open a raw file with raw digger mark an area and it will measure the noise. It is measuring ISO that requires a calibrated light source.
A good description of DxO mark is published here: http://www.luminous-landscape.com/essays/dxomark_sensor_for_benchmarking_cameras.shtml
it is worth reading.
The formal definition of the "low light maximum ISO" is:
Low-Light ISO is then the highest ISO setting for the camera such that the Signal-to-Noise ratio reaches this 30dB value [32:1 ratio at 18% middle grey] while keeping a good Dynamic Range of 9 EVs [512:1 ratio] and a Color Depth of 18 bits [roughly 64×64×64 colors].
and
Low-Light ISO is normalized to an arbitrary reference resolution of 8 MPixels
The values may be arbitrary, but they are equal to all. Normalising the data to 8 MPixels means that resolution doesn't affect the rating.
DxO makes all these measurements, so they can make a good raw converter. They feel that this is a relevant way to present data. It is anyway more relevant than manufacturers data which may say 25000 ISO but say nothing about the resulting image quality.
Personally I almost never look at the DxO mark but look at the individual measurements.
A good sensor will rate well in all categories. A sensor with high readout noise will be hurt in the landscape rating. Canon's have high readout noise at low ISO, but it improves at high ISO because the signal from the sensor is amplified more before raw conversion.
Regarding the lens comparison, the 600/4 is sharper than the 50/1.8II according to DxO data, so I don't understand the rating.
I always check DxO data on lenses I want to buy, if available. It has saved me some money and a lot of frustration, I guess.
Best regards
Erik
Huge problem right there, Erik: a "good" image by what criteria? To what standard?
A perfect example of the arbitrariness of the whole DxO process; and given that you can't have a good - or bad - high ISO image until the Raw data are actually converted into an image (and the conversion stage is vital to the end result, as we'd all agree), a completely misleading and disingenuous false equivalence - you cannot establish, purely from analysing Raw data, whether there's a good high ISO image in there.
I find this point particularly ironic, given just how damn' good DxO's Optics Pro is at converting high ISO images - DxO knows it's not telling the whole "low light" story here.
-
One of the good points about these discussions is that it can bring out your generally unspoken assumptions about your photography. So whether you agree with the points raised or not is often not that important.
Keith Reeder states that he doesn't use ISO 100, that is my default EI setting. Some people will use Auto ISO with or without a manually adjusted limits, I usually don't.And even when I do, I will limit its upper speed to EI 400 unless it is a sport in demanding circumstances. Doesn't make me right or wrong just different.
But when words such as equalized or normalized or adjusted for a standardized output are used, it all becomes BS to me. Whether its DxO or DPR or anyone else.
-
Hi,
Normalisation is a pretty smart thing. It says how things will look in print or for that part on screen at normal sizes.
Let's say that we happen to have same generation technology, but we have large pixels in one camera like the Sony A7s, while a camera like the Sony A7r has three times as many pixels. Pixel for pixel the A7s will blow the A7r away in noise, DR and ISO. On the other hand when printed the 36 MP Sony A7r image will be interpolated down to same size as the 12 MP A7s image, three times that many pixels will be used, and almost the entire per pixel advantage of the A7s will be compensated. DR gains a bit less when downsampling.
So the normalisation stuff makes it possible to compare sensors of different pixel count, while the unnormalised values correspond essentially looking at the image at actual pixels.
Best regards
Erik
But when words such as equalized or normalized or adjusted for a standardized output are used, it all becomes BS to me. Whether its DxO or DPR or anyone else.
-
No.
As Barry has pointed out, this is not intrinsically true, because it ignores improvements in sensor technology's "state of the art" - in the Real World, recent crop cameras often significantly outperform older full frame sensors in terms of dynamic range (shadow DR, anyway, which is all that anyone seems to be interested in these days) and high ISO noise handling.
Well, of course comparisons only make sense if we talk about concurrent sensors. Of course recent crop cameras have better performance than FF cameras of the past.
-
Well, of course comparisons only make sense if we talk about concurrent sensors. Of course recent crop cameras have better performance than FF cameras of the past.
Yeah, but people don't, Paulo: there's a routine presumption that any FF sensor is inherently "better" than any crop sensor, purely because its a FF sensor.
So it's not a matter of "of course..." - some folk really don't get this.
-
Hi Erik,
yeah, I understand completely that "noise" is objectively measurable: but what about subjective notions like noise "quality"?
It's entirely possible to have an image with a higher measurable noise value that still results in a "good" image (possibly a subjectively better image than from a camera with a smaller measured noise level) because the noise itself is (say) small-grained and unobjectionable, or "film-like": so I still balk at the idea of objectively quantifying something like the amount of noise and then using that metric alone - the quantity - as a determinant of high ISO image quality.
Again, until you're looking at the converted image, it's not indicative of anything meaningful to the end result.
A good converter can be a tremendous equaliser, and different converters render noise in very different ways, even if the amount of noise is objectively the same in all of them...
My basic point is that you cannot say "there's noise xx at ISO yyyy, therefore the image quality will be zz..."
-
Erik
thank you for your insights "under the hood" on these test paradigms. I have come to learn much from you
But would never buy a lens or system based on these narrow tests
Rent the lens and body you like. If you like the look, buy it. Otherwise send it back to Roger. Once I settle on a body.. rinse and repeat for the lenses
As noted above by another 6D user, I have never felt inhibited by it's DR. In my 240 as well. It's about how the image appeals to me. I really don't care about "tests"
When I raced cars we never did road tests. We decided what class, got a vehicle and optimized it in real time… nothing was theoretical. All real results. Works for cameras too to some extent
-
Eric's comments would lead me to conclude that the DXO "bashing" is justified. There are so many compromises made in their methodology as to make the end results not ones I would rely on to make a lens or camera purchase any more than I would use the feds fuel economy numbers when selecting a vehicle to buy.
-
That is of course the save yourself a lot of money question.
Best regards
Erik
Yeah, but people don't, Paulo: there's a routine presumption that any FF sensor is inherently "better" than any crop sensor, purely because its a FF sensor.
So it's not a matter of "of course..." - some folk really don't get this.
-
Eric's comments would lead me to conclude that the DXO "bashing" is justified. There are so many compromises made in their methodology as to make the end results not ones I would rely on to make a lens or camera purchase any more than I would use the feds fuel economy numbers when selecting a vehicle to buy.
What I like about DxO is that their actual measurements eg. DR usually agree with what you get if you take a raw file from your own camera. This has been confirmed time over time by the geeks on this forum. Their data is real. For the rest .... if you believe that a single number can sum up a camera or a musical instrument, then I guess you should marry a shoe-size ;)
Edmund
-
Maybe someone who has a real beef with their process can post a sample from the maligned camera or lens in question that shows the problem.
Nyself I dont have a problem with what they recommend. I have put my money there with good results. I do feel some lenses they rate poorly are better than presented. I'm talking about some old stuff. Overall DxO seems to know their stuff.
-
What I like about DxO is that their actual measurements eg. DR usually agree with what you get if you take a raw file from your own camera. This has been confirmed time over time by the geeks on this forum. Their data is real. For the rest .... if you believe that a single number can sum up a camera or a musical instrument, then I guess you should marry a shoe-size ;)
Edmund
I see nothing wrong with marrying a "10" !
Marc :)
-
Hi,
The usual tests for ladies don't measure bitchness.
Best regards
Erik
I see nothing wrong with marrying a "10" !
Marc :)
-
Hi,
The usual tests for ladies don't measure bitchness.
Best regards
Erik
How can one measure something that doesn't exist? ;)
We're skating on thin ice here - there may some ladies reading their husbands posts ...
Let's go back to bashing DxO and cameras :)
Edmund
-
How can one measure something that doesn't exist?
Well, the British think tank DEMOS recently tried ...
http://takimag.com/article/misogyny_among_the_fairer_sex_jim_goad/print#axzz32GcPsljO
-
The problem with DxO is that those are lab measurements that do not conform with field conditions.
Also, a great lens has an artistic quality, that cannot be measured, only experienced.
And the human eye has its own way of seeing things, different from a machine, and the qualities there can't always pinned down by simple numbers.
The reason these lab results are so popular: everybody can easily compare lenses and cameras, like you can compare the horse power of cars.
There is a difference between criticism and bashing that needs to be respected.
Camera developers have let lab results rule for a long time - and with them, their marketing departments. First we had the Megapixel wars, then the High ISO wars.
I wish camera quality would be tested by those who actually use them on a high level - not by lab technicians.
We would probably get cameras that are not as much designed for specs, but for artistic potential in image making.
-
The problem with DxO is that those are lab measurements that do not conform with field conditions.
Do you have an actual example of these supposed gaps?
My view is that the correlation is excellent.
Cheers,
Bernard
-
I wish camera quality would be tested by those who actually use them on a high level - not by lab technicians.
We're all waiting for you to do it :)
Edmund
-
I wish camera quality would be tested by those who actually use them on a high level - not by lab technicians.
They frequently are.
And then these Real World tests/reviews are summarily dismissed (often in support of some specific agenda) in favour of "but DxO says...", as if the Real World experience - and perhaps the hard-won skill of the user in maximising the image potential of the camera in question - are irrelevances.
-
They frequently are.
And then these Real World tests/reviews are summarily dismissed (often in support of some specific agenda) in favour of "but DxO says...", as if the Real World experience - and perhaps the hard-won skill of the user in maximising the image potential of the camera in question - are irrelevances.
The problem with such real-world reports is that they often point in orthogonal directions. If tester A says that camera 1 is better than camera 2, while tester B says the opposite and neither provide sufficient background for themselves or their testing methology, it is really hard to conclude based on the tests.
I am a big fan of such real-world tests for the stuff that cannot possibly be measured. Such as ergonomy and user-interface.
-h
-
The problem with such real-world reports is that they often point in orthogonal directions. If tester A says that camera 1 is better than camera 2, while tester B says the opposite and neither provide sufficient background for themselves or their testing methology, it is really hard to conclude based on the tests.
I am a big fan of such real-world tests for the stuff that cannot possibly be measured. Such as ergonomy and user-interface.
-h
It's not only that. Some perceptual aspects can be quantified eg. distorsion, some with great difficulty eg eg. color can be quantified initially by colorimetry; so testers ignore those hard aspects until they are quantified, rather than saying: This thing looks good/bad. A typical example is perceived "upscalability" of an image from a camera - some upscale well, some fall apart.
Edmund
-
It's not only that. Some perceptual aspects can be quantified eg. distorsion, some with great difficulty eg eg. color can be quantified initially by colorimetry; so testers ignore those hard aspects until they are quantified, rather than saying: This thing looks good/bad. A typical example is perceived "upscalability" of an image from a camera - some upscale well, some fall apart.
Edmund
I think that a more significant source of confusion is :
1) Some (possibly artistically good) photographer test the "DR" of a camera and finds it to be good/poor
2) Some "lab technicians" tests the "DR" of the same camera and finds it to be poor/good
Chances are, the two are not really testing the same thing using equivalent methods, so no wonder that they come to diverging conclusions. There is the very real possibility that the "lab technicians" are reporting something true but irrelevant. But there is also the possibility that the "seasoned veteran" plainly does not understand the image processing (how many concert pianists understands how a grand piano works?), or is using a processing workflow that is relevant to him and nobody else.
I do believe in reporting thoroughly _what_ you did, and _why_ you concluded like you did. Ideally, someone ought to be able to repeat your experiment and if they share taste and choice of subjects with you should come to the same conclusions.
-h
-
It's almost a cliche to mention it, but the lessons learned from the good (bad) old days of high-end audio are salutary.
Stereo Review magazine did in depth technical reviews of amps and such down to four decimal points. Their reviews were so influential that manufacturers actually designed some products to test well, with 0.005 THD. The problem was that there was little if any correlation between test performance and how an amp sounded.
The Absolute Sound magazine took the other approach, eschewing lab tests and just relying on the judgement of experienced listeners. In this case most observers noted that this was much more closely correlated with the user's listening experience that the lab tests. The take-away was that people listen to music not numbers. The reasons are now well understood, though they wern't at the time.
Michael
-
It's almost a cliche to mention it, but the lessons learned from the good (bad) old days of high-end audio are salutary.
Stereo Review magazine did in depth technical reviews of amps and such down to four decimal points. Their reviews were so influential that manufacturers actually designed some products to test well, with 0.005 THD. The problem was that there was little if any correlation between test performance and how an amp sounded.
The Absolute Sound magazine took the other approach, eschewing lab tests and just relying on the judgement of experienced listeners. In this case most observers noted that this was much more closely correlated with the user's listening experience that the lab tests. The take-away was that people listen to music not numbers. The reasons are now well understood, though they wern't at the time.
Michael
The lesson to be learned here, Michael, is that the results of any lab tests, whatever the nature of the subject or object that is under study, have to be interpreted.
Such interpretation requires an understanding of the subjective significance of any numerical results, and how that subjective significance might be affected by other factors, such as the type of music, the loudness of the music, the frequency of the sound, the acoustic properties of the listening environment, and perhaps most significantly the hearing acuity of the listener.
Those who are interested in the technical aspects of hi fi soon learn that differences of 0.005 THD and 0.05 THD are irrelevant to their listening experience whatever the circumstances, just as those who are interested in the technical aspects of sensor performance soon learn that differences in Dynamic Range of 0.1 EV are irrelevant to their visual experience.
I suspect that those who attach little or no credence to the DXOMark test results are mostly jpeg shooters, or those who simply don't have the photographic skills to confirm for themselves the significance of the DXO results.
-
Hi,
For the audio comparison, there is a lot of hype in that area.
Regarding the DxO tests, they present a lot of info, but all of that info relates to noise. DxO mark essentially ignores resolution, for instance. I am pretty sure that the measurements are accurate.
Now, I would say noise is an important characteristic of cameras, but other factors may be more important, especially if noise related data is good enough. Lenses, AF and handling may have a significant role.
Best regards
Erik
The lesson to be learned here, Michael, is that the results of any lab tests, whatever the nature of the subject or object that is under study, have to be interpreted.
Such interpretation requires an understanding of the subjective significance of any numerical results, and how that subjective significance might be affected by other factors, such as the type of music, the loudness of the music, the frequency of the sound, the acoustic properties of the listening environment, and perhaps most significantly the hearing acuity of the listener.
Those who are interested in the technical aspects of hi fi soon learn that differences of 0.005 THD and 0.05 THD are irrelevant to their listening experience whatever the circumstances, just as those who are interested in the technical aspects of sensor performance soon learn that differences in Dynamic Range of 0.1 EV are irrelevant to their visual experience.
I suspect that those who attach little or no credence to the DXOMark test results are mostly jpeg shooters, or those who simply don't have the photographic skills to confirm for themselves the significance of the DXO results.
-
The lesson to be learned here, Michael, is that the results of any lab tests, whatever the nature of the subject or object that is under study, have to be interpreted.
Such interpretation requires an understanding of the subjective significance of any numerical results, and how that subjective significance might be affected by other factors, such as the type of music, the loudness of the music, the frequency of the sound, the acoustic properties of the listening environment, and perhaps most significantly the hearing acuity of the listener.
Those who are interested in the technical aspects of hi fi soon learn that differences of 0.005 THD and 0.05 THD are irrelevant to their listening experience whatever the circumstances, just as those who are interested in the technical aspects of sensor performance soon learn that differences in Dynamic Range of 0.1 EV are irrelevant to their visual experience.
I suspect that those who attach little or no credence to the DXOMark test results are mostly jpeg shooters, or those who simply don't have the photographic skills to confirm for themselves the significance of the DXO results.
More likely they feel they do not need to use numerical lab results. Most top chefs probably dont feel the need to use timers and temperature probes. They see, feel, taste.
The science is important, nobody credible would say we would have the current level of tools without the science. DxO tries to use a repeatable scientific methodology for comparisons. The deciding factor for everyone is judgement. Some people have it based on a variety of things. Some people lack judgement in a variety of things.
IMO DxO numbers are better at telling you what is good equipment than what is bad equipment.
-
It's almost a cliche to mention it, but the lessons learned from the good (bad) old days of high-end audio are salutary.
Stereo Review magazine did in depth technical reviews of amps and such down to four decimal points. Their reviews were so influential that manufacturers actually designed some products to test well, with 0.005 THD. The problem was that there was little if any correlation between test performance and how an amp sounded.
The Absolute Sound magazine took the other approach, eschewing lab tests and just relying on the judgement of experienced listeners. In this case most observers noted that this was much more closely correlated with the user's listening experience that the lab tests. The take-away was that people listen to music not numbers. The reasons are now well understood, though they wern't at the time.
Michael
There are many problems with high end audio, measurements is not one of them.
The reluctance to use blind testing and/or consulting the scientific literature comes to mind as the biggest reason why people are laughing at the industry, magazines and enthusiasts (such as myself). High end audio is (by the majority) seen as males in their 50s with more money than critical sense. I think that is sad.
If we did not have the "subjective revolution" in the late 70s/early 80s (before I started working part-time at a hifi shop), I claim that we would probably have better sound in our homes today. Right now, the hifi industry gets a better return-on-investement by marketing idiotic cables or other bling for the cult, rather than researching and developing the stuff that, you know, affect sound.
More likely they feel they do not need to use numerical lab results. Most top chefs probably dont feel the need to use timers and temperature probes. They see, feel, taste.
I believe that there are different lines of thought in cooking as well. Heston Blumenthal has a decidedly "scientific" approach to cooking.
http://en.wikipedia.org/wiki/Heston_Blumenthal
"Molecular Gastronomy" gives some great examples of issues where expert cooks have come to the right _conclusions_, but their arguments are all wrong. I believe that this can be relevant to photography: a great photographer with a sound testing methology might conclude that doing so and so gives her better images. When she tries to _explain_ why doing this leads to better images, she may well be totally wrong, and when this "knowledge" works its way into photography books, websites and curriculum, people will inevitably apply her explanations in areas where they leads to bad practice. Depending on how much testing these people do on their own (and/or how much of a guru status the initial reporter have in the field), people might end up doing counter-productive things.
http://en.wikipedia.org/wiki/Molecular_gastronomy#Example_myths_debunked_or_explained
In (classical) physics, there can only be one "right", all other must be wrong. In cooking or photography art, there can be several "rights", and the degree of "rightness" may be cultural/social. This may explain why so many great photographers have such strong opinions about the analysis and measurement of technical facets of their cameras.
-h
-
Hi,
For the audio comparison, there is a lot of hype in that area.
Regarding the DxO tests, they present a lot of info, but all of that info relates to noise. DxO mark essentially ignores resolution, for instance. I am pretty sure that the measurements are accurate.
Now, I would say noise is an important characteristic of cameras, but other factors may be more important, especially if noise related data is good enough. Lenses, AF and handling may have a significant role.
Best regards
Erik
Hi Erik,
Noise (of various types) has always been a huge nuisance in both photographic and audio reproduction. It's been one of the major factors that contributes either directly or indirectly to a clear sense of the difference between a live performance of music (without amplification and loudspeakers, such as a classical concert) and a recording of the same music.
The ideal situation for me, when listening to music through a well-adjusted hi fi system, is to be able to close my eyes and imagine, without distraction, that I have the best seat in the Concert Hall or Opera House.
It's misleading to claim that the DXO tests ignore resolution. Sensors have no resolution in themselves. The megapixel count describes the potential resolution of the sensor. Surely one doesn't need to be told that a 36mp sensor has a greater potential resolution than a 16mp sensor!
The actual resolution will always be determined in conjunction with a specific lens, and this is what DXOMark also test, producing the new metric 'Perceptual Megapixel' or P-Mpix.
Now one might sometimes disagree with the quantitative significance of those P-Mix results, but one can't sensibly disagree that the resolution of a recorded image is always a product of both sensor and lens. If one increases the potential resolution of either one of them, the resolution of the recorded result will also increase.
-
It's almost a cliche to mention it, but the lessons learned from the good (bad) old days of high-end audio are salutary.
Stereo Review magazine did in depth technical reviews of amps and such down to four decimal points. Their reviews were so influential that manufacturers actually designed some products to test well, with 0.005 THD. The problem was that there was little if any correlation between test performance and how an amp sounded.
The Absolute Sound magazine took the other approach, eschewing lab tests and just relying on the judgement of experienced listeners. In this case most observers noted that this was much more closely correlated with the user's listening experience that the lab tests. The take-away was that people listen to music not numbers. The reasons are now well understood, though they wern't at the time.
Michael
Strangely enough, people also choose which classical concert they go to by personal sampling and by word of mouth or reviews, not numbers :)
Edmund
-
More likely they feel they do not need to use numerical lab results. Most top chefs probably dont feel the need to use timers and temperature probes. They see, feel, taste.
The science is important, nobody credible would say we would have the current level of tools without the science. DxO tries to use a repeatable scientific methodology for comparisons. The deciding factor for everyone is judgement. Some people have it based on a variety of things. Some people lack judgement in a variety of things.
Hi,
This analogy doesn't quite make sense. If you wish to compare photography with cooking, I think the camera, lenses, other photographic equipment and editing programs would be analogous to ovens, grills, choppers, blenders and mixers etc. The raw, unprocessed food would be analogous to the RAW files of the scenes or subjects that have been photographed, and the preparation, mixing, addition of various spices, and the cooking, would be analogous to the processing of the RAW images in Lightroom and/or Photoshop.
IMO DxO numbers are better at telling you what is good equipment than what is bad equipment.
Don't understand this. If a DR of 14EV is good, then by the same reasoning a DR of 7EV is bad.
-
Don't understand this. If a DR of 14EV is good, then by the same reasoning a DR of 7EV is bad.
Good and Bad are the wrong terms IMO. Too absolutist. More DR is More, less is Less. There are plenty of photographic situations where seven stops of DR is sufficient. Or maybe my photographic style features high contrast with crushed shadows and/or blown highlights. In such cases having 14 stops available is a luxury—great if I like starting with spatial/tonal-rich source data and then decimating it in post—but not a necessity.
When I look at a DxO evaluation it's to see whether or not a particular piece of gear will do what I want it to, or maybe if it has extra capabilities I might want to take advantage of. This by itself doesn't tell me whether or not I'll actually like the gear in use. To find this out I've gotta put it through its paces.
Also, sometimes gear will behave in interesting ways when pushed beyond its intended limits. Guitar amp distortion, anyone? This happens with electronic cameras too. I like the tonal distortion my Leica M8.2s produce when overexposed. I suspect this causes the sensor to record enough infrared to affect chroma response, as in the attached photo (which was taken two days ago, not in autumn...those leaves, in the visible spectrum, are intensely green).
-Dave-
-
Hi,
This analogy doesn't quite make sense. If you wish to compare photography with cooking, I think the camera, lenses, other photographic equipment and editing programs would be analogous to ovens, grills, choppers, blenders and mixers etc. The raw, unprocessed food would be analogous to the RAW files of the scenes or subjects that have been photographed, and the preparation, mixing, addition of various spices, and the cooking, would be analogous to the processing of the RAW images in Lightroom and/or Photoshop.
Don't understand this. If a DR of 14EV is good, then by the same reasoning a DR of 7EV is bad.
To me the light entering the lens would be raw food. The lens is processing, like cutting, peeling, cooking. The sensor is arranging for plating, the software is spices. I agree the analogy is not great.
AFAIK DxO does its measurements on one copy they get. That is not scientific. I know for a fact lenses they say are poor resolve well to 14MP APS-C, they also have good OOF rendering characteristics. Therefore I think DxO may get a bum copy of a lens. If they rate something well it is unlikely the copy is far above other samples. If they rate something badly they may have a defective product.
-
To me the light entering the lens would be raw food. The lens is processing, like cutting, peeling, cooking. The sensor is arranging for plating, the software is spices. I agree the analogy is not great.
AFAIK DxO does its measurements on one copy they get. That is not scientific. I know for a fact lenses they say are poor resolve well to 14MP APS-C, they also have good OOF rendering characteristics. Therefore I think DxO may get a bum copy of a lens. If they rate something well it is unlikely the copy is far above other samples. If they rate something badly they may have a defective product.
My guess is production line pass/fail diagnosis is now probably DxO's main line of business, except it concerns camera phones. I am not sure that Nikon, Canon and co. enjoy the idea of end users being able to test lens samples in the same way.
The phone industry is probably now the driving factor behind the ISO standard test procedures and sensor development.
I participated in a camera phone quality meeting at their headquarters, and feel reluctant to bash them too hard because they are earnest people who genuinely care about image quality, and their catering and coffee is superb. Their developers get unlimited free Nespresso Pro!
Edmund
-
Good and Bad are the wrong terms IMO. Too absolutist. More DR is More, less is Less. There are plenty of photographic situations where seven stops of DR is sufficient. Or maybe my photographic style features high contrast with crushed shadows and/or blown highlights. In such cases having 14 stops available is a luxury—great if I like starting with spatial/tonal-rich source data and then decimating it in post—but not a necessity.
When I look at a DxO evaluation it's to see whether or not a particular piece of gear will do what I want it to, or maybe if it has extra capabilities I might want to take advantage of. This by itself doesn't tell me whether or not I'll actually like the gear in use. To find this out I've gotta put it through its paces.
Also, sometimes gear will behave in interesting ways when pushed beyond its intended limits. Guitar amp distortion, anyone? This happens with electronic cameras too. I like the tonal distortion my Leica M8.2s produce when overexposed. I suspect this causes the sensor to record enough infrared to affect chroma response, as in the attached photo (which was taken two days ago, not in autumn...those leaves, in the visible spectrum, are intensely green).
-Dave-
Of course, Dave. I agree. This is why numbers are so much more useful than ordinary words when attempting to describe such characteristics in an objective manner. If the DR of a P&S camera is found to be 7EV at ISO 80, and the methodology is sound, and the term DR is well defined, then it's an indisputable fact that the DR is 7EV. Whether that level of performance is good or bad, or good enough, or not too bad, is always a matter of opinion and can be the subject of endless debate.
Without the benefit of DXO test results, or similarly objective results, one could waste an awful lot of time and money testing cameras for oneself. I'll always remember the exaggerated opinions expressed about the performance of the Nikon D3 when it was first released, at a time before DXO had made its test results available. The hype was ridiculous. Because the D3 had an unprecedented high ISO setting of 25,600, it was assumed that the high-ISO performance of the camera was streets ahead of any other camera on the market. I recall claims of up to 2 stops better high-ISO performance than any other DSLR then available.
I was using a Canon 5D at the time and took the trouble to visit a Nikon store to do a few test comparisons. Underexposing the Canon at ISO 3200 to simulate the higher ISO settings up to 25,600, my results indicated that between ISO 1600 and ISO 25,600, the D3 had no more than 0.5EV better DR. Now that's worth something, and is a noticeable improvement, but is not particularly significant, and a long way from the hype of 2 stops improvement.
When I first reported these findings on this forum, the debate and disputes continued because DXO had still not made its test results available. A couple of years later the DXOMark site appeared, showing comparisons between the Canon 5D and the Nikon D3. I admit I was surprised to discover how closely my own results, using relatively crude methods, matched those of DXO.
Making use of the performance limitations of equipment to create an unusual artistic effect might have some merit. The banding that's very noticeable in the deep shadows of shots from the 5D can be used to create a tapestry effect, by severely underexposing the entire scene. ;D
-
Hi,
In general, I feel that test are far more reliable than user experience. The main reason is controlled conditions and the possibilty to go back and redo if in doubt. My own experience is that the more factors are equalised the more the equipment performs as expected. Say, shooting a white balance card and use it to for white balance makes colours much more similar, shoot a color checker and equalize black and white fields, global contrast will be similar. Create a DNG colour profile for both cameras, even less difference will be visible.
Regarding DxO, I feel their data is usable. For me it is mostly about the lens tests, as I am essentially happy with my cameras regarding noise characteristics. DxO says that it takes five points on DxO mark for a visible difference, so I can see if a new generation of sensor gives a significant improvement or not by a glance.
Regarding the DR stuff, I feel that it is not the more important stuff in life, and it is quite hard to measure in real life, as it is often reduced by lens flare. In practical photography I see that there is a significant advantage in reproduction of darks between my late generation Sony cameras and my P45+ back that is using a Kodak sensor developed around 2005. This corresponds to DxO data.
Getting back to lenses, I look at the curves. I feel it is a good info, and it is useful to decide if a lens is an useful upgrade. For instance, I have Sony 24-70/2.8 lens that has weak corners at 24 mm and drops of axis resolution at 70 mm. The new Sigma 24-105/4 "Art" semt to be an attractive alternative, but looking at DxO I found out that it is probably not better than the Sony lens.
DxO makes their measurements to be able to support sensor/lens combinations with their quite reespected raw converter. So their results go into their raw converter, which does quite a few lens corrections.
Personally, I much enjoy Roger Ciala's writing over at LensRentals. They test tens of thousands lenses each year anf thus also see statistical variations. I also like Rogers explorative mind, insights on optics and scientific/statistical approach. LensRentals has probably the most extensive lens testing facility outside the lens manufacturing business and they are glad to share their findings.
A problem with formal tests is that they are focused on in focus sharpness, but most images will have significant areas more or less out of focus, as plane of absolute focus is very thin. So in many cases, out of focus rendition may be even more important than rendition in absolute focus. This is an area there many new lenses can shine, with circular aperture for instance, while many designs show weakness with longitudional chromatic aberration and double contours caused by over corrected spherical aberration. Photozone covers the bokeh stuff pretty well in their tests.
Best regards
Erik
Of course, Dave. I agree. This is why numbers are so much more useful than ordinary words when attempting to describe such characteristics in an objective manner. If the DR of a P&S camera is found to be 7EV at ISO 80, and the methodology is sound, and the term DR is well defined, then it's an indisputable fact that the DR is 7EV. Whether that level of performance is good or bad, or good enough, or not too bad, is always a matter of opinion and can be the subject of endless debate.
Without the benefit of DXO test results, or similarly objective results, one could waste an awful lot of time and money testing cameras for oneself. I'll always remember the exaggerated opinions expressed about the performance of the Nikon D3 when it was first released, at a time before DXO had made its test results available. The hype was ridiculous. Because the D3 had an unprecedented high ISO setting of 25,600, it was assumed that the high-ISO performance of the camera was streets ahead of any other camera on the market. I recall claims of up to 2 stops better high-ISO performance than any other DSLR then available.
I was using a Canon 5D at the time and took the trouble to visit a Nikon store to do a few test comparisons. Underexposing the Canon at ISO 3200 to simulate the higher ISO settings up to 25,600, my results indicated that between ISO 1600 and ISO 25,600, the D3 had no more than 0.5EV better DR. Now that's worth something, and is a noticeable improvement, but is not particularly significant, and a long way from the hype of 2 stops improvement.
When I first reported these findings on this forum, the debate and disputes continued because DXO had still not made its test results available. A couple of years later the DXOMark site appeared, showing comparisons between the Canon 5D and the Nikon D3. I admit I was surprised to discover how closely my own results, using relatively crude methods, matched those of DXO.
Making use of the performance limitations of equipment to create an unusual artistic effect might have some merit. The banding that's very noticeable in the deep shadows of shots from the 5D can be used to create a tapestry effect, by severely underexposing the entire scene. ;D
-
DR is significant. I pay close attention to how the shaded undersides of trees look in bright sun. You can see detail easily that may be hard to capture on camera.
Try this simple test: Walk with your colorchecker passport from a well lit room to a very dark room at night. Can you still see the colors? Yes. Can you see the small shifts of warmer/ cooler white tone? Yes, very easily. If all this detail vanishes in your images the camera, or maybe the output medium, needs better tonality in darks.
-
AFAIK DxO does its measurements on one copy they get. That is not scientific.
If they are being accurate about _what_ they are doing, I am reluctant to claim that they are "not scientific" based on them reviewing only one sample. You would have a much stronger case for calling them "not scientific" on the basis that their reports are not peer-reviewed and published in a reputable journal. But then, how much scientific literature is being discussed here?
I agree that it is valid to question how representative a single sample of a lens is. It seems that the measurement that cause most irritation on discussion forums, though, is the camera DR measurement (especially among Canon owners). Do you also think that there is sample variation in interchangeable-lens cameras?
-h
-
I agree that it is valid to question how representative a single sample of a lens is.
While true, it still is a sample of the larger population (like the one you might buy). Besides, I believe that DxO have in the past have updated their database, when it became obvious (the benefit of publishing it for comparison to other tests) that they had tested an outlier.
It seems that the measurement that cause most irritation on discussion forums, though, is the camera DR measurement (especially among Canon owners).
It is what it is, and it's almost exactly the same as what I measure on the models that I tested.
Do you also think that there is sample variation in interchangeable-lens cameras?
There is always sample variation, but the real question is how large is it, and where do we draw the line of calling it still acceptable (as it will drive cost, up or down). Then, should we be inclined to purchase it, we can still try and find a copy that scores relatively high, by conducting proper tests if one knows how. Of two copies, there is most likely one better than the other (maybe the DxO copy), but maybe not on all aspects, so we also need to make personal trade-offs.
Cheers,
Bart
-
No, I don't think DR varies like lenses. The quantity of lens variability is shown by the need for focus adjust on high end cameras. What governs DR? Full well capacity? That would be based on the volume of silicon, the purity of the chip? Probably not a lot of variability. Manufacturer process quality differences yes.
-
Hi,
Full well capacity and readout noise are the major factors.
Best regards
Erik
No, I don't think DR varies like lenses. The quantity of lens variability is shown by the need for focus adjust on high end cameras. What governs DR? Full well capacity? That would be based on the volume of silicon, the purity of the chip? Probably not a lot of variability. Manufacturer process quality differences yes.
-
For me, there is a huge gap between scoring or charting cameras and photography in general.
Scoring and charting a camera (or lens) is scientific purpose. I agree that the photographer need to know his gear specificity but hell ... that will not make him a better photographer, at all.
DxO imho is aiming at a specific new age species of photoconsumers who stay more time on internet than shooting. The capacity of a camera is subjective as bokeh, in a certain limit. Old cameras still run properly, old mfdb backs still run properly.
So DxO is a sort of marketing tool, after all. They even might have some brand preferences so they are not that impartial.
Gear technology is a deviance in photography. The photographer have so much infos coming trough his head that he is always sort of stressed.
"Have I made the good choice ??" ... "So my camera is not that good after all ??" ... "should I return it and jump on this another brand ??" ... "DxO is saying that my lens is crap but my photos are selling well; what should I do ??" ...
In this world, a lot of ppl do not feel secure; the world want you to feel insecure. Brands try to make you feel secure by many sneaky and clever ways (some ppl are paid for that). Those kind of sites just put even more doubts in their heads and it result in a form of stress and frustration, endless debate on whatever is good or not.
Meanwhile you may notice that the most rewarded photographers on this planet do not give a flying buck at all of this, unless they are paid to do so.
For me, Michael reviews are completely sufficient because it come from an experimented photographer. Photographers who have a load of flight hours also review things with wisdom.
DxO just look like a freakin bot compared to real and uninterested photographer review.
So, at the end, it is amazing how internet and ppl who know his power can remote a whole population of humans. Even if the world is getting more and more comfortable in developed country, this intra-species behaviour is sort of alarming. You have the shepherds and the goats.
P.S: the not so funny fact is that some humans like being goats and know they are goats. So that lead to a big question: Are humans prone to obedience and submission, after all ?
-
Scoring and charting a camera (or lens) is scientific purpose. I agree that the photographer need to know his gear specificity but hell ... that will not make him a better photographer, at all.
This notion doesn't make much sense to me. You agree on the one hand that the photographer needs to know the specifications of his gear, yet claim on the other hand that knowing such specifications will not make him a better photographer at all.
One of these statements must be false.
Gear technology is a deviance in photography. The photographer have so much infos coming trough his head that he is always sort of stressed.
"Have I made the good choice ??" ... "So my camera is not that good after all ??" ... "should I return it and jump on this another brand ??" ... "DxO is saying that my lens is crap but my photos are selling well; what should I do ??" ...
On the contrary! DXO doesn't use such terms as 'crap'. They use numbers. Once the photographer understands the significance of the numbers, he is more able to make an informed choice and free himself from all stress regarding the performance of his equipment, and whether or not he has made a good choice. If a new camera appears on the market which, according to DXO, has higher specifications with regard to DR, for example, than the camera one is currently using, and one knows from experience that the DR of one's current camera is always acceptable, or at least very rarely unacceptable, then one should able to resist any hype and sales pitch from any source praising the new camera for its excellent DR capabilities.
Furthermore, if one is actually dissatisfied with the DR of one's current camera, or any other specification which DXO test, one has the opportunity to check for oneself, from the DXOMark comparisons, whether or not the new camera has a worthwhile increase in DR, or other specification, which would justify the purchase of a new camera. Worthwhile improvements might be Color Sensitivity greater than 1 bit higher; DR greater than 0.5EV higher, and SNR greater than 6dB higher.
-
This notion doesn't make much sense to me. You agree on the one hand that the photographer needs to know the specifications of his gear, yet claim on the other hand that knowing such specifications will not make him a better photographer at all.
One of these statements must be false.
On the contrary! DXO doesn't use such terms as 'crap'. They use numbers. Once the photographer understands the significance of the numbers, he is more able to make an informed choice and free himself from all stress regarding the performance of his equipment, and whether or not he has made a good choice. If a new camera appears on the market which, according to DXO, has higher specifications with regard to DR, for example, than the camera one is currently using, and one knows from experience that the DR of one's current camera is always acceptable, or at least very rarely unacceptable, then one should able to resist any hype and sales pitch from any source praising the new camera for its excellent DR capabilities.
Furthermore, if one is actually dissatisfied with the DR of one's current camera, or any other specification which DXO test, one has the opportunity to check for oneself, from the DXOMark comparisons, whether or not the new camera has a worthwhile increase in DR, or other specification, which would justify the purchase of a new camera. Worthwhile improvements might be Color Sensitivity greater than 1 bit higher; DR greater than 0.5EV higher, and SNR greater than 6dB higher.
There is no false statement.
All FF cameras since 2008 have enough DR for all major professional work (weeding, products, landscape...). The thing is to understand light and lightning and reading the camera's user manual. Knowing the specification of his gear mean knowing how to use it at the maximum, and knowing very well how to PP specifically behind. If consumers was focussing on those two very points, trust me, ppl would not buy that much gear. Market just want us to buy more, that the only reality. The reality is also that gear become better and better even in other sectors of this market. I finished a weeding one half hour ago and I did it with the Fuji XT-1 only and some good lenses. The D700 and 800 stayed in the car and I KNOW that this little inexpensive camera did the job extremely well.
I think that photographers, if they focus enough, can output outstanding results out of "obsolete" cameras. The camera is just a dead tool. the photographer's skills are the most important. It is why your numbers tell me nothing. Now I have a check to bring to my bank ;)
-
There is no false statement.
All FF cameras since 2008 have enough DR for all major professional work (weeding, products, landscape...). The thing is to understand light and lightning and reading the camera's user manual. Knowing the specification of his gear mean knowing how to use it at the maximum, and knowing very well how to PP specifically behind. If consumers was focussing on those two very points, trust me, ppl would not buy that much gear. Market just want us to buy more, that the only reality. The reality is also that gear become better and better even in other sectors of this market. I finished a weeding one half hour ago and I did it with the Fuji XT-1 only and some good lenses. The D700 and 800 stayed in the car and I KNOW that this little inexpensive camera did the job extremely well.
I think that photographers, if they focus enough, can output outstanding results out of "obsolete" cameras. The camera is just a dead tool. the photographer's skills are the most important. It is why your numbers tell me nothing. Now I have a check to bring to my bank ;)
Well there is sufficient and not quite enough as far as DR goes. I personally detested the early digital era where we got a similar effect to the early days of chemical photography..white out skies with nothing there. Situation has improved a lot, but not if you're shooting Canon who are lagging behind on DR, you can get by with what's on offer, but they are not up to other makers in that areas.
I could spend a long time counting Wedding photographers who have struggled with DR issues by using Canon. Technique can help but it is one of the reasons I didn't move to Canon (the other being personal handling preferences) None of Canon's FF bodies have DR even up to the better APS-C models, and the margin is quite notable too. You don't always need big DR, but there are times it is very useful.
-
There is no false statement.
Okay! Let's examine this statement of yours, "the photographer needs to know his gear specificity". The immediate question that follows from such an assertion is, "Why does the photographer need to know his gear specificity ?" I presume we're not talking here about basic procedures in the manual such as how to turn the camera on and off, and how to insert the batteries.
Now according to you, it doesn't make him a better photographer in any way. If this is also true, then one can't help wondering, "Why would anyone, whose goal is to become a better photographer, waste his time learning his gear specifications?" Surely there must be more important things to do.
Your answer to this appears to be, "Knowing the specification of his gear mean knowing how to use it at the maximum, and knowing very well how to PP specifically behind."(I'm presuming that last part of your sentence means, "specifically, knowing how to post-process very well afterwards".)
Does knowing how to use the gear to its maximum, that is knowing how to use the gear to its full potential, not make one a better photographer? Are you really claiming that an understanding of how to get the most from one's equipment has nothing to do with being a better photographer? If so, perhaps we should define more clearly what constitutes being a 'better' photographer.
As regards post processing, surely that is another, separate, skill which has to be learned in addition to the purely photographic skills of selection of subject, lighting, and the taking of the shot. Apparently, Henri Cartier Bresson never did any post processing. He was a good photographer, wasn't he? I get the impression that most people using cameras rely upon the post processing built into their cameras. However, I don't.
All FF cameras since 2008 have enough DR for all major professional work (wedding, products, landscape...).
Well, I don't consider myself to be a professional photographer, so perhaps I'm not qualified to refute that statement. Also, I'm so modest that I would hesitate to claim that my standards must be higher than those required for major professional work.
However, I find such a comment very strange when you include categories such as landscape. I can understand that DR specifically may not be not a concern when taking photos of products in a studio, or in any situation where the photographer can create his own lighting and pre-arrange the composition.
I can also appreciate that portraits of people in general, such as those taken at a wedding, may not require the camera to have an exceptional DR. What's usually more important in those circumstances is SNR at 18% grey.
By contrast, the real dynamic range of many landscape scenes can be huge; far, far greater than the capabilities of the Nikon D800E, for example. We might think that 13 or 14 stops of DR in a camera is very impressive, and it is impressive compared with other, lesser cameras, but not when compared with the real scene. Those last 3 or 4 stops of DR from the D800E, say between the 10th and the 14th stop, will appear quite degraded compared with what the eye saw in the real scene.
The real, practical advantage of a camera with a DR rating of 14 EV, compared to one with a DR rating of say 11 EV, is that the 14 EV camera has a usable DR of maybe 10 or 11 EV, whereas the 11 EV camera, using the same standards of noise-free acceptability, may have a usable DR of only 7 or 8 EV.
I finished a weeding one half hour ago and I did it with the Fuji XT-1 only and some good lenses. The D700 and 800 stayed in the car and I KNOW that this little inexpensive camera did the job extremely well.
Can't comment on the performance of the Fuji XT-1. DXOmark haven't shown their test results yet. Also, I'm not sure a camera is a suitable tool for weeding purposes. I prefer to put on some rubber gloves and just pull the weeds out. (Sorry! Couldn't resist. ;D )
I think that photographers, if they focus enough, can output outstanding results out of "obsolete" cameras.
Are you implying that a similar degree of focus and dedication using a modern camera will produce results that are no more outstanding? Since you've already agreed that a photographer needs to know the specifications of his camera in order to be able to use it to its full potential, do you not think that a modern camera used to its full potential will produce, at least some of the time, even more outstanding results than obsolete equipment in the hands of the same photographer?
The camera is just a dead tool. The photographer's skills are the most important. It is why your numbers tell me nothing.
I'm afraid not, unless perhaps you believe in magic. I haven't yet come across any stories on the internet of any photographer being able to take a photograph with his skills alone, without a camera. I do believe that having a camera is the first essential, wouldn't you agree? ;)
Also, I see a contradiction in your statement that the numbers tell you nothing. Didn't you previously state that the photographer needs to know the specifications of his camera? Perhaps you do not consider numbers such as '30dB SNR at ISO 3200' a specification. Such numbers seem quite specific to me.
I'll give you an example of how useful and meaningful the DXO numbers can be. Let's say I'm considering purchasing a Sony RX100 MkII. I'm a bit undecided because I've been disappointed in the past when using P&S cameras, because the noise, DR and high-ISO performance leave much to be desired. How will this RX100-II compare with some of the DSLRs I've used in the past? What can I expect?
The last Canon DSLR I bought was the 15mp 50D, with which I was reasonably pleased. How would the basic, fundamental, RAW image quality of the RX100 II compare with that of the Canon 50D with a roughly equivalent lens attached, such as the EF-S 17-55/F2.8? DXOMark is able to tell me. Checking their charts, I find the following information which I've been able to interpret from the numbers, because I'm such a skilled photographer. ;)
(1) Color Sensitivity of the RX100 is noticeably better than that of the 50D at base ISO, but slightly worse, to an insignificant degree, at ISO 3200.
(2) Tonal Range of the RX100 at all ISOs is very slightly worse, but to an insignificant degree.
(3) SNR at 18% grey is also very marginally worse, but to an insignificant degree.
(4) Dynamic Range of the RX100 II is actually significantly better than the that of the 50D at base ISO, by one whole stop. However, at ISO 3200 the DR of the RX100 is marginally worse by a less significant 1/3rd of a stop.
(5) The built-in zoom lens of the RX100 II gets a P-Mpix rating for sharpness of 6, whereas the 17-55/F2.8 on the 50D gets a slightly higher P-Mpix rating of 7. Distortion and chromatic aberration is also better in the Canon lens.
Conclusion. The images from the RX100 II, on balance, will be marginally worse, technically, than those from the Canon 50D used with EF-S 17-55/F2.8 lens (a lens I used to use with my 50D), but not to a significant degree and not as much as I expected. I've gone to the trouble of making this comparison because I'm considering getting the latest Mk3 version. However, I shall wait for the DXO tests of the RX100 Mk3 before I make a decision. The improvements in the Mk3 might result in basic image quality being at least the equal of the Canon 50D with 17-55/F2.8 lens, in all respects, and the wider aperture of F1.8-F2.8, of the Mk3, could result in lower noise as a result of sometimes being able to use a lower ISO than I would in the same circumstances with the 50D.
Now I have a check to bring to my bank
Ah! Now I understand your definition of a good photographer. The more money you earn, the better the photographer you are. ;)
-
No, I don't think DR varies like lenses. The quantity of lens variability is shown by the need for focus adjust on high end cameras.
The need for micro-focus adjustment just shows that there is lens variation wrgt off-sensor PDAF. It does not (in my view) prove that there is lens variation when doing manual focus.
Of course, we all (?) agree that there is some variation in lens properties (even when manually focussed).
-h