Pages: 1 2 3 [4] 5   Go Down

Author Topic: Eyes vs Numbers  (Read 30906 times)

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Eyes vs Numbers
« Reply #60 on: February 03, 2009, 10:17:32 pm »

Quote from: Panopeeper
There are tale tell signs.


I can not relate the results of your comparisons to anything; we are comparing the digital images in different dimensions. Your cat shot an ISO 156,800 on a 2310 Mpix 6x6cm camera illuminated by a 60W lamp in the kitchen looks ugly compared to my dog shot at ISO 100 on an APS-C camera in daylight if printed 20mmx30mm on an Epson 9900 printer.  (This was meant to illustrate how I see your comparisons.)

If you shoot a very clean gray scale card, *very* evenly illuminated, underexposed two or three stops (highlights are irrelevant), I can take measurements from those shots (if you throw in a color checker card too, even better). Equal exposure of the shots between the cameras is not relevant; low exposure is essential.

A really exciting shot for measurement looks like this Stouffer wedge (but it is not easy to make this shot correctly).

I wouldn't be shooting cats or moving subjects. I'd be shooting the sorts of subjects where, in the past, I've found the 5D to be a bit inadequate.

Here's one of the first shots I took with my 5D over 3 years ago. I was checking for flare from the 24-105 IS lens, which was very much in the news at the time, but instead found banding in the shadows. I believe the shot is a reasonably accurate ETTR, although your analysis would probably find it is slightly overexposed.

The crop is of the bottom right corner of the frame, from a conversion with linear tone curve. As you can see, the image is quite degraded in general, as well as exhibiting obvious banding. I would expect a similar shot using the D700 would show much improvement in those shadows. By comparing bracketed exposure of the same scene, I could work out if the D700 does indeed have a one stop DR advantage at ISO 100, thus confirming whether or not those DXO results have practical significance. I prefer this sort of subject to a Stouffer Wedge   .

[attachment=11356:Moogerah...overview.jpg]  [attachment=11357:Moogerah_5D_crop.jpg]
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Eyes vs Numbers
« Reply #61 on: February 03, 2009, 10:23:14 pm »

Quote from: John Camp
Engineers have a bias toward measurement, which is a real bias, because if only 20% of a system is currently measurable and quantifiable, it's possible that the other 80% is more important in every way -- yet the engineers will tend to go with what they can measure, because that's their bias.

The design of a complex system, like a camera or lens, is a team work that involves an iterative process of assessement of achievement vs goals.

For this to work, there is a need to define metrics that are used to set the goal and to measure how close you are to that goal. Not all the metrics represent the preceived value of the customer well, and not all of them are public, but human kind does currently not know any other way to design complex products. This isn't a bias.

One of these metrics could be the average of the 1 to 5 rating that your 20 cousins gives to a sample picture taken with various iterations of a lens, one could be the actual weight vs target weight, the resolution in lines per mm,...

More often than not, some of the metrics used internally by companies are those they expect the customers to look at when trying to assess the value of a product (the famous benchmarks in the computer world). In the present case, the very existence of DxO will influence the camera companies in designing cameras that far well per the DxO metrics.

My personnal view has always been that the most important IP of a company is probably these metrics themselves, meaning knowing what has to be measured that can be measured and that correlates well with real performance as means to measure customer satisfaction. That's what you are talking about when you mention the examples of engineers only knowing 20% of the right metrics.

Back to the initial case, I am not sure that DxO is not representative of what my eyes see. For all the cameras I have had in my hands, I see an excellent correlations between their numbers and my experience understanding the fact that resolution is not a very important part of their work.

Cheers,
Bernard

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Eyes vs Numbers
« Reply #62 on: February 03, 2009, 10:30:20 pm »

Quote from: Panopeeper
Edit: Bernard, I started writing this post but did something else in the meantime, came back and posted it, then I saw your post; that's the reason of overlaps.

No problem.

Cheers,
Bernard

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Eyes vs Numbers
« Reply #63 on: February 03, 2009, 10:50:45 pm »

Hi,

My view is that we should not look at the DxO mark as a figure of merit but the individual measurements. If we do that and compare the Phase One P45+ with the Sony Alpha it is quite obvious that the P45 has an advantage of say 3-4 dB in all areas at minimum sensivity. The sole reason that the Sony Alpha rates higher is that it has higher usable sensivity, and who would doubt that?

One thing that DxO mark seems to indicate clearly that higher ISOs on the P45+ are not for real, no more amplification just underexposure.

The DxO mark quasi ignores that fact that the P45+ plus has about 60% more pixels, so if the A900 is good for A1 enlargements than the P45+ would be good enough for almost A0 and still offering a better image quality than the A900 at the pixel level.

Maybe DxO is a good thing, if we just ignore the single figure of merit and look at the real data.

If we compare the P45+ with Nikon D3x the Nikon is a little better on noise (1 dB at minimum ISO) but the pixel ratio is still the same.

Best regards
Erik


Quote from: BernardLanguillier
Hello Michael,

Interesting essay, thanks for posting it.

Resolution should indeed clearly be taken into account when rating cameras/backs. But even so, a number will indeed clearly never have the same meaning for different people with different needs and ends up being pretty useless (ideally the rating should be taking into account the print size).

The DxO mark is basically a measurement of sensor quality. Sensor quality doesn't tell the full story, but it is nonetheless relevant. I understand your point about the comparions being perhaps not 100% fair, but the superiority of the A900 along the metrics selected by DxO seems pretty logical when you consider that:

- The photosite size is about the same between 20+ MP DSLRs and 39MP backs (the D3/D700 photosites being significantly larger),
- The DSLRs use micro-lenses which contributes to better light efficiency,
- The DSLR technology is 3 years younger (nearly as long as a whole Bush mandate),
- Sony/Canon are much more talented semi-conductor companies than Kodak with a much longer history, a much more diverse range of application and a much higher number of knowledgeable engineers (and they work more...),
- They probably invest at least an order of magnitude more in fundamental semi-conductor research and tend to pool their resources between companies for activities of common interest.

So there are pretty good reasons to think that the DxO results provide a realistic view of what we have at hand.

As far as trusting one's eyes instead of measurement results, when a G10 is hard to disthinguish from a P45+ in a small print, wouldn't you say on the contrary that the DxO results are a pretty good match to what one actually sees?

Cheers,
Bernard
« Last Edit: February 03, 2009, 10:52:45 pm by ErikKaffehr »
Logged
Erik Kaffehr
 

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Eyes vs Numbers
« Reply #64 on: February 03, 2009, 11:06:36 pm »

Quote from: ErikKaffehr
If we compare the P45+ with Nikon D3x the Nikon is a little better on noise (1 dB at minimum ISO) but the pixel ratio is still the same.

Erik,

Yes, true, the P45+ still has more pixels and is the better tool for some really large print applications.

I guess that MF manufacturers must be real glad that Nikon didn't go the MF route... if I were the Hasselblad CEO, I would volunteer 10% of my benefits to Nikon to make sure that they stick to that decision.

Cheers,
Bernard

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Eyes vs Numbers
« Reply #65 on: February 03, 2009, 11:29:24 pm »

Quote from: ErikKaffehr
If we compare the P45+ with Nikon D3x the Nikon is a little better on noise (1 dB at minimum ISO) but the pixel ratio is still the same.

Best regards
Erik

That's right. However, when we look at the results for the normalised size of 8x12" at 300 ppi, which represents a downsampling of both images, the situation is reversed, and the P45+ has a 1dB SNR advantage over the D3X. However, 1dB either way is neither here nor there. For all parctical purposes, one could consider the SNR of the D3X and P45+ as being about the same. It's no wonder Edmund has got himself a D3X   .

I assume that on larger prints that involve a bit of interpolation for the D3X image, the P45+ would also have about the same, and equally insignificant, 1dB advantage, so we are left with basically just a resolution advantage for the P45+, which is not surprising. We don't need DXO to tell us that a 39mp camera is capable of greater resolution than a 24mp camera.

What's surprising is that the DR of the D3X seems to be better than that of the P45+, on both screen and at normalised 8x12" size. It would be interesting if someone could take some test shots with both cameras to confirm or deny this.
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Eyes vs Numbers
« Reply #66 on: February 03, 2009, 11:35:33 pm »

Quote from: Ray
What's surprising is that the DR of the D3X seems to be better than that of the P45+, on both screen and at normalised 8x12" size. It would be interesting if someone could take some test shots with both cameras to confirm or deny this.

Quite a few people have done P25+ vs D3x tests, and all those I read reached the conclusion that the D3x had the better image quality overall.

Knowing that there are also many people who feel that the P25+ is better than the P45+ but for the resolution, I still fail to see how the DxO results are surprising and not correlated to real world results. As I have written several time, I have done an informal comparison against the ZD (considered to be in the very same ballpark as the other backs at base ISO) and I prefer the D3x files to the extend that I sold my ZD and full set of Mamiya lenses.

If I am not mistaken, Edmund has a P45+ on Mamiya and he would be in a position to do a comparison between both.

Cheers,
Bernard

NashvilleMike

  • Guest
Eyes vs Numbers
« Reply #67 on: February 03, 2009, 11:42:06 pm »

Quote from: John Camp
(snip...)
The DXO problem is that they insist that they have a measure, but it's not a real measure. They have a whole bunch of measures, but those measures don't include the whole universe of possibilities, and without the whole universe, you can't reach a valid conclusion. And it may be inherently *impossible* to measure the entire universe of possibilities. So, if a lot of people with good, trained photographic eyes say that a large P65 print is better than a large D3 print, and a DXO measurement says it's not, then the eyes are right, just as they're right when they see a bumblebee fly.

JC

I agree with John here, and I like the analogy with high end audio as well.  I'm an ex-audiophile, and was "in the business" earlier in my days, so I'm familiar with that "lifestyle" (or would that be "obsession" lol) as well as anyone and there are some definite parallels with this topic.

A few comments:

Today's society wants it easy and is used to getting things quickly. I remember (as a kid) the 70's, so I've seen the change.
Today we've got your instant coffee, a quick-start breakfast,the  morning snack from the row of vending machines down the hall, fast food lunch, and a microwave dinner, all eaten far too quickly as we accelerate to an early grave (or a dozen prescription medications at the least) as a result of our stress, lack of exercise, and poor nutrition.

Social commentary aside, the whole "what camera do I buy" / "what lens do I buy" process has changed as well. Today we've got the internet, the old school magazines, cameras being sold by typically less-than-knowledgeable clerks at big box retailers, a few camera stores here and there, and much, much, much more marketing and advertising being shoved at our ears and eyeballs. Most of that wasn't around in the "old days", you know, back when we had to trudge our 600mm lenses to school up hill both ways through the snow and all that.  

Salespeople in the 70's were usually trustworthy to some point while these days it's not common for me to hear in my local photo shop (said to a new buyer) "Oh, get brand XXXXXX, it's much better than brand YYYY" simple because there is a spiff (incentive payment to the salesperson) on that brand or product or a push of the sales clerks own bias, which may very well have been formulated by exposure to the marketing and a need to defend their own brand choice. I may be wrong, but my memories were that people weren't anywhere as defensive about their brand choice "back then" as they are now. Well, those days are gone, and the consumer is getting a lot more information from various sources aimed at them all at once, and understandably wants to make a "correct" purchasing decision. In 1970 you maybe talked to a pro you knew, talked to your salesperson, asked a friend, looked at an advert, and off you went. Now you've got half a dozen web sites and forums that discuss things subjectively, another half a dozen that do "objective" tests, plus the sales folks, friends, more advertising and marketing, and often it's hard to *quickly* get a clear message that screams "hey, Mister Consumer - obviously the Plektomatic Q45XDR-III 17.3 megapixel DSLR is the perfect camera, get it".

So in today's world, what's the holy grail?  The single all encompassing quality grade that covers all bases, is totally defensible, and can't possibly be wrong. I'd argue, as I think John would, that this is impossible, but it is the holy grail. If there could be one test site that totally was beyond any doubt the sole deity, the supreme being, and had "the grade", then the decision process would be really, really simple. However, real life isn't always simple, and due to, as John points out, the vast universe of possibilities that determine what quality is, I seriously doubt it's going to happen, ever. But people want this easy number, because it relieves them of the far more time consuming option that I think today provides a better answer: a combined analysis of multiple subjective reviewers, a reading to understand each reviewers biases and tastes, an overview of the objective sites (plural, not just DXO or photozone or whatever in isolation, but a look at the results across a group of multiple test sites) and then some discussion with folks who own them, and if possible, renting of the proposed purchase to evaluate it. This takes time, and it takes effort, and it takes admitting that something that measures well might not actually shoot well in some situations, or perhaps, the things the photographer values highly in the complex interrelated object called "image quality" may be weighted differently versus something that is made to shine on a test chart.

Now, does this mean measurements have no place? Certainly not. I think what I'd personally like to see in the future is an *exploration* of how to quantify what we might see in terms of "image quality" beyond the current test methods. We could, for example, look at lens performance. Many folks only look at resolution (via photozone or slrgear or whatever) or an MTF graph for "the answer". Yet lens designers themselves (like the fellow from Zeiss in his MTF white paper and Dr. Caldwell, he of the famed Coastal 60mm Macro) have said MTF doesn't provide all the answers. So perhaps more research could be done in these areas - trying to correlate measurements with visual performance (not just lenses, but camera bodies too). Offhand, I don't think only evaluating sensor data exclusive to the raw conversion really helps us much here, but that's must my opinion.

I don't think we'll ever be able to fully quantify image quality, maybe not in my lifetime or the next generations, but I think it would still be valuable to at least try and make some progress. To circle around back to the high end audio analogy, Stereophile always has measurements and commentary on them along with the subjective review of the gear itself, and I get the distinct impression that the crew at Stereophile is very much aware of the differences between good measurement and good sound and tries to figure out why this is so. And perhaps that's what is needed "down the road" with photographic gear - more discussion of both subjective and objective together, with some dialog as to *why* something that scores .00002 geek-o-units of noise may not produce visible image quality as well as something that scores .000034 geek-o-units, or why a lens that knocks one out of the ballpark on photozone just doesn't draw that well when faced with a three dimensional object in the real world.

Like others, I do worry, that in the same manner that so many people use one and only one "objective" lens test site to judge lenses (even when research indicates divergent opinions to that site) that the DXOmark becomes the one and only factor, simply because it's an easy to digest number. Like I stated earlier, careful analysis of the wider spectrum of opinions, always taking into account reviewer bias as well as an admittance that "objective" tests may not be perfect, is a better method at this time.

As an aside, I'd also feel that a lot of people who make proclamations about image quality or lens sharpness (or whatever) need to include the *magnitude* of the difference. A lot of times I think people end up chasing the wrong thing - a guy who drops 8 grand on a D3X to shoot landscape, but then uses a 24-120 average-at-best lens on gitzo 1 series tripod is going to get creamed by a guy with a D2X, 100/2 Makro Planar and a stable support system. My own approach has always been lenses first, then bodies, and I've of late come to realize the importance of the support system as well. Bodies come and go, as will the arguments in the forums as to which one is better, but if one acquires a set of lenses that perform well and (importantly) have a "drawing" style that one feels suits their subjective tastes, they're off to a heck of a good start.

-m




Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Eyes vs Numbers
« Reply #68 on: February 03, 2009, 11:43:59 pm »

Bernard,

Sorry, my reply was not really a reply to your posting. I just choose to reply to your posting while it was the first one the thread. I decided to repost it as a new topic.

Mostly I agree with your postings and you make great pictures.

Best regards
Erik


Quote from: BernardLanguillier
Erik,

Yes, true, the P45+ still has more pixels and is the better tool for some really large print applications.

I guess that MF manufacturers must be real glad that Nikon didn't go the MF route... if I were the Hasselblad CEO, I would volunteer 10% of my benefits to Nikon to make sure that they stick to that decision.

Cheers,
Bernard
Logged
Erik Kaffehr
 

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
Eyes vs Numbers
« Reply #69 on: February 04, 2009, 01:13:02 am »

Quote from: Panopeeper
I think you should ask this question from DxO. What I do know is, that their results re the P45+ DR and ISO characteristics are telling a tale of their ineptness.

I think that DxO state clearly on their site that they test raw data and analyze them with their proprietary (but commercially available) DxO Analyze tool. How did ACR come into this discussion?
Logged
Nikos

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Eyes vs Numbers
« Reply #70 on: February 04, 2009, 01:41:35 am »

Quote from: NikosR
I think that DxO state clearly on their site that they test raw data and analyze them with their proprietary (but commercially available) DxO Analyze tool. How did ACR come into this discussion?
I was only speculating about the reason their analysis of the P45+ is so thoroughly wrong. Someone mentioned, that DxO (the raw processor) does not support MFDBs; but how would they feed their DxO Analyzer?
Logged
Gabor

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
Eyes vs Numbers
« Reply #71 on: February 04, 2009, 02:05:07 am »

Quote from: Panopeeper
I was only speculating about the reason their analysis of the P45+ is so thoroughly wrong. Someone mentioned, that DxO (the raw processor) does not support MFDBs; but how would they feed their DxO Analyzer?

If you trust their description of DxO Analyzer it includes 'Automatic high-speed processing of RAW and/or RGB data for either camera or component level analysis'. Now, it's anybody's guess what is the relationship between this and their DxO Pro converter software.

So, sort of engaging with DxO in a direct discussion we will never know exactly what they're doing.

Gabor, you seem to have some partially documented objections (valid or not) regarding their results. Have you tried to directly contact them and challenge them? This might be more constructive (or not) than moaning in internet forums with any chance you're given. You should be able the speak the same tech language with them so they might engage to some extent. I would be interested to know their response.
« Last Edit: February 04, 2009, 02:06:19 am by NikosR »
Logged
Nikos

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Eyes vs Numbers
« Reply #72 on: February 04, 2009, 02:12:38 am »

Ben Rubinstein

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1822
Eyes vs Numbers
« Reply #73 on: February 04, 2009, 04:50:16 am »

I think that this discussion and the two essays are perhaps interesting on an amatuer level or to rich hobbyists. For a pro given comparable image quality (though if you've needed FF or good high iso noise that means that the Nikon only just got into the game) then first and foremost as a choice will be system. If one works for you or provides for you then the feel of a camera or the availability of specific lenses is going to be far more important than any of this nonsense. Secondly that the 5Dmk II or Sony provide equivalent image quality is as irrelevant to a working pro as it was when the playing field was level for all cameras in the film days. No one suggested that as you could use an Elan 7, an EOS 1v was then a niche product! The difference is just as striking if not more so today. If you need weather sealing, dual card slots, pro AF and 20+ megapixels then there is only two choices and the difference is one of system not some pathetic wrangling over points of a percent. More than that, the 5D mkII and Sony are then utterly irrelevant cameras. This discussion and the assertion that these new cameras in some way are game changers is only relative to rich hobbyists and not to pro photographers!

That anyone can justify a P65+ without having a full economic business case to justify why the upgrade would be necessary is an idea that is horrific to me but more than that, if it's just hobbyists then their decision is irrelevant to the rest of the world. They can go back to fondling their uber expensive gear while working pro's will buy equipment that is both relevant and justifiable. If MR is right and the playing field has been leveled, then for a pro the only considerations now are what the pro choices are. Expensive toys or super megapixel amatuer cameras are now as irrelevant as they always should have been but unfortunately have not to date due to the ongoing digital odyssey forcing amatuer cameras into relevance.

As to the numbers, when the numbers are that close you have to make the decision based on what you want from a file. The megapixels from a 5D and D700 are the same but I find the 5D files to be much sharper out of camera. For all the talk about the Sony, the RAW files I've worked on show noise comparable to my original 1Ds at iso 200 and that is very bad. Could a different workflow show other results? Yes. Is that relevant to what I want? Not in the slightest. If I don't like what I see then screw what anyone else says, screw the test results, I'll keep to what I like and am certainly mature enough not to bother arguing with someone else about if my taste is better than theirs.
« Last Edit: February 04, 2009, 04:56:52 am by pom »
Logged

32BT

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3095
    • Pictures
Eyes vs Numbers
« Reply #74 on: February 04, 2009, 05:52:01 am »

Quote from: pom
I think that this discussion and the two essays are perhaps interesting on an amatuer level or to rich hobbyists. For a pro given comparable image quality (though if you've needed FF or good high iso noise that means that the Nikon only just got into the game) then first and foremost as a choice will be system.

...

 If I don't like what I see then screw what anyone else says, screw the test results, I'll keep to what I like and am certainly mature enough not to bother arguing with someone else about if my taste is better than theirs.

While I mostly agree with what you write, I am not entirely sure whether a working professional can actually avoid that arguing. Considering that most professional photography is commissioned by an art-director or by a client, both of which I believe have a steadily degrading sense of quality, I'm convinced that part of the problem is in the justification of the diminishing returns of "higher quality" systems.

We live in a world where both the speed and the quantity of information is steadily increasing. Since the quality of information is inversely proportional to quantity and speed, we will be confronted with new generations that won't care less. I believe this is one of the reasons of the demise of the music industry: eventually the amount of music produced and its fleeting character will result in people not caring to pay for this productivity. They will not consider "ripping" to be a crime etc...

I also see this in television production. We now are gradually transitioning to HD television (in Europe), yet the picture quality of even reputable screenings is sometimes appalling. You can clearly see that some el-cheapo camcorder was used by the extreme chromatic aberration visible, but because of the increased number of scenes per second, viewers likely won't care less. (Also noticed that it is all the rage these days to move the camera? And if the camera didn't move, they will move the picture digitally?).

Point is: while the reproduction media steadily increases in quality, the production of content seems to steadily decrease in quality, primarily triggered by an oversaturated consumer who does (or eventually will) not care. And in between you, the working professional, and the consumer, stands an art-director who is either him/herself unable to properly judge quality, or is simply hired to represent a consumer that doesn't require quality.

Note that I mean this in the general sense: a wedding jockey could suggest that they try to capture a special event and abide by their own standards of quality, but I would tongue-in-cheek like to counter that with the argument that these days people will likely mary more than once in their lifetime and value their uncle who fools around with the video capability of his 5d2 more than the stills shot by the wedding photographer and his mfdb...

And by the way, this is probably the reason for the possible success of integrated video capability in photocameras which are obviously ergonomically disastrous devices to try and shoot video. But the consumer wants, nay, expects the scene to shake, move, and rattle these days, so there you go. You would think that in-camera shake reduction a-la Sony would be a great tool coupled to video capability, and a clear advantage over Canon, but alas...
Logged
Regards,
~ O ~
If you can stomach it: pictures

Ben Rubinstein

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1822
Eyes vs Numbers
« Reply #75 on: February 04, 2009, 06:43:11 am »

I agree with every single sentence you've written! However I feel that it further drives the argument that the time to compromise on the camera for the sake of the chip is over given a certain plateau reached in image quality as driven by client expectations and as such pro AF is much more important than a possible 1/2 stop gain in DR or whatever. I do not believe that definitive statements about the industry can be made based on the requirements/wishes of the rich hobbyist sector for whom rationality in purchases has always had a very loose connection with the real world. Arguments about degrees of IQ when cameras already have sufficient for the chosen client expectations is indeed only the realm of those who have the free time to argue such irrelevancies while pros are choosing ergonomics and lenses. The realm of those fondling their cameras but not having to get a paid job done right if they want food on their table the next month.  A pro will choose an amount of megapixels needed for the job (subjective but that's fair) whereas the hobbyist will always want more independent of necessity, the 5D mkII is a perfect example. A crippled abortion of an upgrade which is still sold out most places...

Again, not to belittle what is most certainly the most profitable sector of the camera market however saying that pro level cameras are now being pushed towards niche is ignoring the entire pro sector for whom they were made. As there is no rationality to hobbyist purchases I don't think that a study of the same can in any way be definitive to anyone but marketing departments.
« Last Edit: February 04, 2009, 06:46:25 am by pom »
Logged

Chris_Brown

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 974
  • Smile dammit!
    • Chris Brown Photography
Eyes vs Numbers
« Reply #76 on: February 04, 2009, 09:19:07 am »

Quote
One thing I'd like to point out is that, at present, all camera manufacturers don't seem to be listening to the global user base very well . . .
Quote from: Misirlou
Can we say "Mirror lockup button"?
Can we say "WiFi connectivity using 802.11n protocols"?
Logged
~ CB

ndevlin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 679
    • Follow me on Twitter
Eyes vs Numbers
« Reply #77 on: February 04, 2009, 11:34:12 am »

QUOTE (BernardLanguillier @ Feb 3 2009, 04:18 PM)
The ability to preserve the highlights is a matter of correctly exposing

<LOL, isn't it strange that this has to be said again and again?>



My point is that this is true, but doesn't answer the question. If you accept that ETTR is correct (and frankly, those who practically invented digital photography think it's elementary), then the "best" exposure is the highest exposure you can have without loosing highlight detail.  This expsoure is best because it puts the most real data in *all* tonal zones, but for purposes of this discussion, especially in the shadow tones.

In a perfect world, my desired zone 2 data would be two thirds of the way up the histogram! I can then put it in its place, according to my creative vision of the image.  Would I could do this, there would be no noise to speak of in what ends up being the low shadows of the finished image.

Of course, this isn't possible with anything but static subjects and multiple exposures.  In most real world situations, the limiting factor for how much data I can get in my desired shadow tones is how high I can push the highlight zones without losing them.   Therefore, the functional limiting factor of image quality is actually highlight preservation.

Similarly, none of this applies to jpegs (none of this conversation is really relevant to jpeg applications anyway, given the crudity of in-camera post), and to dynamic, lower-light situations. There, I agree, clean shadow detail is king.  But that's not really the landscape context at all, but rather the terrain of photojournalism and wedding photography.

We may well be saying the same thing, just in different ways.

- N.
« Last Edit: February 05, 2009, 10:55:18 am by ndevlin »
Logged
Nick Devlin   @onelittlecamera        ww

sreidvt

  • Newbie
  • *
  • Offline Offline
  • Posts: 14
Eyes vs Numbers
« Reply #78 on: February 04, 2009, 12:21:54 pm »

Quote from: John Camp
I honestly don't think this will ever happen. If you look at Sean Reid's review site [a pay site], you'll see that he often refers to a way a lens "draws." So in a review of high-end normal lenses, he gave excellent marks both to a Nikon and a Zeiss 50mm, said either one is capable of doing any kind of professional work, but he said that they draw differently and he prefers the Zeiss. So in this case, part of the "quality" equation is measuring Sean Reid's preferences, which may change from time to time. On the Online Photographer recently, Mike Johnston, who knows a lot about lenses, panned a lens because of its distortions, but actually liked the "look" it produced. In audio, you have large numbers of people who agree (apparently) that some amp/speaker systems sound better than others, despite measures to the contrary. It seems to me that the problem is with the engineers, not with the ears. Engineers have a bias toward measurement, which is a real bias, because if only 20% of a system is currently measurable and quantifiable, it's possible that the other 80% is more important in every way -- yet the engineers will tend to go with what they can measure, because that's their bias. It's the old "bumblebees can't fly" problem -- bumblebees CAN fly, the engineers just didn't have the ability to quantify the characteristics that allowed the bumblebees to do so.

The DXO problem is that they insist that they have a measure, but it's not a real measure. They have a whole bunch of measures, but those measures don't include the whole universe of possibilities, and without the whole universe, you can't reach a valid conclusion. And it may be inherently *impossible* to measure the entire universe of possibilities. So, if a lot of people with good, trained photographic eyes say that a large P65 print is better than a large D3 print, and a DXO measurement says it's not, then the eyes are right, just as they're right when they see a bumblebee fly.

JC

Hi John,

Of course, the least important aspects of my reviews are my opinions, per se. The most important aspects are the observations and examples. From those, one should be able to draw some of his or her own conclusions. So, for example, in the recent review of the ZE 50/1.4 and Canon 50/1.2 L the core elements of the review were the examples and observations about how both of those lenses draw. Whether I personally prefer one or the other is almost incidental. Frankly, I think that distinction applies to all reviews. For example, it's worth knowing if a lens is prone to veiling flare and shows lower contrast as a result. Whether or not I personally might want to use that lens for certain work is mostly anecdotal.

Our job, as reviewers, is, first and foremost, to describe. That's also why rankings of cameras and lenses are always suspect. How can a reviewer know what the exact needs and priorities of a given reader-photographer are? And lacking that knowledge, how can he or she say what is "best"?

Responding to DXO's approach overall would take more time than I can give right now but I think that it is important to remember that data presented in numbers is not necessarily any more objective or accurate than data presented in other ways. DXO has to make certain decisions about what they want to measure and how. Measuring high ISO noise, for example, is certainly not cut and dry because luminance noise affects picture differently than chrominance noise, shadow noise can look different than mid-tone noise, subject lighting type can affect noise, etc. The potential dangers, of describing picture quality in numbers, are over-simplification and over-generalization. A given testing path and set of decisions leads to a certain number. But what that number tells us, really, is just something about what's at the end of that specific path. So some people's experiences with a given camera may jibe with DXOs results and some may not.

I think that the real measure of a digital file is not only what it looks like (after a RAW conversion for example) but also how malleable and resilient it is during post-processing. Some digital files fall apart quickly as one tries to change them from their initial form to the form they take as a source for a final print.

Some other thoughts, none of which is likely to surprise you.

One is that I believe that the way a camera works as a tool and, especially, the way it allows one to see the subject can be as or more important than "file quality" per se.

The second is that, in the digital age, we are often overly concerned with technical performance. As I think over the history of photography and the work that has affected me the most, very little of it has been done with cameras that were (in their time) at the apex of technical quality. There's Strand's, Evans' and Atget's work with view cameras but none of those pictures is great because of the camera, lens or film quality. When I worked with large format, which I did for a long time, it was as much for the seeing/working process it provided as for any technical advantages it had.

Most of the photography that has affected and influenced me personally has been made with small format cameras using whatever film the photographer needed for the lighting conditions. Very little of it was made with low ASA film (except for work done when low ASA was all there was).

So I guess my feelings about this can be summarized in this way.

1) DXO tests show us certain kinds of measures of certain aspects of camera performance. They're a set of specific paths followed to their natural ends. Generalizing those results beyond that is problematic.  

2) Once a camera's technical output is sufficient for what a photographer wants to make (and we all have different needs and tastes in that respect) technical performance differences can become red herrings that distract us from the much more important and difficult process of making a strong picture.  I report on various kinds of technical camera and lens performance in reviews because a professional review essentially requires that. But I try to never confuse resolution, contrast, CA, etc. differences with photography itself.

Cheers,

Sean
« Last Edit: February 04, 2009, 12:24:06 pm by sreidvt »
Logged

Mort54

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 590
    • http://
Eyes vs Numbers
« Reply #79 on: February 04, 2009, 09:09:38 pm »

Quote from: Panopeeper
The DxO report said so. You need to decide if you are defending DxO's statements of you make your own conclusion (pls make this clear in your posts).
I never said I was defending DxO's statement. I could care less about DxO's statement. I never once even brought up DxO, or mentioned it in any way. You're the one who's bringing DxO into this. Let's TRY to stick to the facts here, shall we.

Quote from: Panopeeper
You can claim whatever you want to, and I can give less than a fig for that.
You've offered not a shred of proof to back up your statement re Nikon chopping off bits, and YOU expect anybody to give a fig about what you say? Again, let's try to stick to the facts here.

Quote from: Panopeeper
Only suckers are paying for that amateurish review (this opinion is based on the "free" information of that site).
At least he's tested both the D3X and 5DII side by side, and provided the RAW files for download so subscribers can draw their own conclusions, which apparently is a lot more than you've done. Have you even touched either camera? The free information is a tiny summary of all the work he's done. For you to dismiss his review without actually looking at his review tells me all I need to know.

Quote from: Panopeeper
I am sorry, but I don't have time for analysing lots of shots (made by people, who understand nothing of a review). I spend sometimes hours with a single pair of comparative raw images, if they are worth of analysis.
If you don't have time, then why did you bring it up :-)

Quote from: Panopeeper
Honestly, I am becoming uneasy with this discussion.
What discussion. You're the one who's ranting and raving here. I'm just trying to educate you.

Quote from: Panopeeper
How do you come to make such a statement? As the matter of fact, I can prove any time what I stated, but I am a bit selective regarding the purpose of my demonstrations, for I doubt that you would understand anything of that.
Really, then prove it. I've told you where you can find proof regarding my statement. You've done nothing but hand wave and blow smoke. Regarding what I do and don't understand, I do understand when someone is full of it.
Logged
I Reject Your Reality And Substitute My
Pages: 1 2 3 [4] 5   Go Up