Luminous Landscape Forum

Equipment & Techniques => Cameras, Lenses and Shooting gear => Topic started by: Ray on February 20, 2008, 08:52:48 pm

Title: The truth about the E-3 Image Quality
Post by: Ray on February 20, 2008, 08:52:48 pm
It always amazes me how quick people are to champion the superior attributes of a new camera they have just bought. They seem to become like salseman for their new equipment providing free advertising for the manufacturer. To the more rational amongst us, their claims often seem to defy credibility.

I'll just mention a few recent claims which I viewed with great skepticism.

1. The image quality from the 40D is virtually as good as the 5D, perhaps better in some respects.

2. The 40D image quality is a definite improvement over the 20D and 30D.

3. The E-3 definitely produces sharper images than the 40D.

4. The E-3 produces better and sharper images than the 5D. (And I know because I'm a working photographer who publishes in magazines and I've used both cameras, but I'm too busy to carry out tests.)

5. The Nikon D3 has about a 2 stop noise improvement at high ISO, compared with anything else on the market.

6. The Phase P21 DB definitely produces a more sold looking image with a greater sense of 3-dimensionality than the 1Ds3 can produce. You either see it or you don't.

Often when one questions such claims, the people making the claim are unable to provide the evidence for whatever reason, usually because they are too busy perhaps, but sometimes I think because they want to let the myth ride, whilst it can.

I now see that Dpreview have just completed their review of the Olympus E-3. There are no surprises for me, expect perhaps one. The E-3 actually produces image which are less sharp than the 40D when shooting RAW. This is apparently because the E-3 uses a slightly stronger than average AA filter which limits the potential to capture sharp detail. The lens used on the E-3 during the tests is considered by Dpreview to be one of the sharpest optics they have ever used, the Olympus 50/F2, so I don't think that claims that the lens used on the E-3 might be substandard will hold.

The other image quality characteristics are no surprise. The E-3 is definitely noisier than the 40D at ISO 1600 and above. Highlight headroom is slightly less so the camera is more prone to clipping of highlights. Jpeg output is slightly sharper and punchier (more contrast and saturation) which is no doubt the reason why some people got the impression the E-3 produced even better results than the 5D.

Of course, needless to say, there are more features to a camera than fundamental image quality. That's not in question. Here I'm only addressing fundamental image quality, not range of lenses available, speed of focussing or wieght and waterproofing advantages etc etc.
Title: The truth about the E-3 Image Quality
Post by: DarkPenguin on February 20, 2008, 09:45:10 pm
I don't see much difference between any of these cameras.
Title: The truth about the E-3 Image Quality
Post by: Ray on February 20, 2008, 10:19:08 pm
Quote
I don't see much difference between any of these cameras.
[a href=\"index.php?act=findpost&pid=176317\"][{POST_SNAPBACK}][/a]

Any of which cameras? The one's that Dpreview compared or the examples I mentioned in my preamble?

I also don't see much difference in fundamental resolution between the cameras that Dpreview compared, but they didn't include the 5D or other FF cameras. The most obvious differences between the E-3 and the 40D, D300 etc is noise at high ISO, but that can often be at least partially compensated for by use of a wider aperture for the same DoF, except when you need to use the widest aperture available regardless of DoF consequences. A 40D with 50/1.4 at f2 and ISO 1600 is likely to produce noticeably better image quality than an E-3 with 50/2 at f2, if you need a fast shutter speed. If you don't need a fast shutter speed, the sensor's anti-shake in the E-3 will provide some benefits since the Canon 50/1.4 does not have IS.

My point here is not that the E-3 image quality is significantly worse than it's immediate competitors. It's not. My point is, despite being very marginally worse, how is it that some people came to a conclusion that it was noticeably better?
Title: The truth about the E-3 Image Quality
Post by: DarkPenguin on February 20, 2008, 11:27:10 pm
Follow the lenses.  Although resolution might not be better the better Oly lenses are generally more consistent across the frame.

Man, I'm getting about $3k more in a tax return than I expected.  That 40D purchase will not be enough to quench my thirst for camera/lens blood.  I think I need to buy a new lens.
Title: The truth about the E-3 Image Quality
Post by: Ray on February 21, 2008, 12:08:17 am
Quote
Follow the lenses.  Although resolution might not be better the better Oly lenses are generally more consistent across the frame.
[a href=\"index.php?act=findpost&pid=176333\"][{POST_SNAPBACK}][/a]

Lenses are fundamental to image quality. The lens used on the E-3 in the dpreview test is described as one of the finest dpreview have ever used. The Canon 50/1.4 is designed for FF 35mm. It's unlikely the Oly would be more consistent across the frame, although it is possible that some Oly lenses are more consistent across the frame than some Canon EF-S lenses.
Title: The truth about the E-3 Image Quality
Post by: espressogeek on February 21, 2008, 12:09:15 am
I would love to see the e-3 eat the 40d's lunch, and I own a 40d. Its just not in the cards with the smaller sensor. I agree the glass is more consistent, and that panny/leica glass is sweet, but its not good enough to compensate for the smaller image captured.

That being said I would love to have the panny/leica lens that came stock on the dmc-l1 and the 25mm summilux. I really enjoy those lenses signatures.
Title: The truth about the E-3 Image Quality
Post by: Ken R on February 21, 2008, 12:14:16 am
Just saw the E3 test on dpreview and they compare it to the 40d. Basically IQ is a tosup at iso 800 and below.
Title: The truth about the E-3 Image Quality
Post by: Misirlou on February 21, 2008, 01:11:37 am
The answer to your question is that different people use different criteria for making their choices. Wow. How surprising.

The problem with this whole line of reasoning is that one party or another may set the terms of the debate in a different way. Some posters like to use words such as "quality" and "definite" rather indiscriminately. There are an infinite number of potential individual opinions on such matters, and one will make up one's mind based on the criteria one chooses.

Just look at the replies here so far. One person reads the DPreview article and thinks it says the E-3 is less sharp than the 40D, and another poster reads it to say that it's a tossup below ISO 800. Is one of you wrong? Is one of you a fool? I don't know. Why do we need to get so worked up over it?

Ray, a couple months ago, I made some RAW shots with my 20D and my 40D. When I looked at those shots, I was convinced that the 40D versions were better. You were not. I don't agree with the logic you applied to the situation. Is one of us wrong? I don't know how we could ever come to an agreement. You will not convince me that I am wrong, and I will not convince you. There is nothing more to say. What else do you expect?

Digital imaging, especially in artistic considerations, is subject to so many personal variables that it is impossible to account for things in any "definite" way. So we can all pixel peep till the cows come home, and still come to different conclusions.  

Some people keep buying Canons, some people keep buying Nikons, and some are buying Olympus cameras. Heck, a friend of mine who has some top of the line DSLRs just bought a Holga. All of those people have reasons for their choices that they consider perfectly valid. Would you suggest that most all of them are wrong, since there can theoretically be only one camera that produces the "best image quality" at any given time?

No matter what equipment I select, I never forget that most of the photos that really inspire me were made with equipment far, far inferior to what I own now. Almost all of St. Ansel's work comes to mind. All of his early work was done with terrible lenses, by current standards. When looking through my work over the years, I can identify maybe 4% or so that would have come out substantially "better" had I used "better" equipment.

I bet I will take out my 50 year old Rolleiflex's a few times and make some great photos that print better at large sizes than some other shots I get out of the 40D. I'm also willing to bet there are a whole slew of people that are going to buy E-3s and produce better photos than I will with my 40D. What of it?
Title: The truth about the E-3 Image Quality
Post by: Ray on February 21, 2008, 04:54:58 am
Quote
Digital imaging, especially in artistic considerations, is subject to so many personal variables that it is impossible to account for things in any "definite" way. So we can all pixel peep till the cows come home, and still come to different conclusions. 

Some people keep buying Canons, some people keep buying Nikons, and some are buying Olympus cameras. Heck, a friend of mine who has some top of the line DSLRs just bought a Holga. All of those people have reasons for their choices that they consider perfectly valid. Would you suggest that most all of them are wrong, since there can theoretically be only one camera that produces the "best image quality" at any given time?

No matter what equipment I select, I never forget that most of the photos that really inspire me were made with equipment far, far inferior to what I own now. Almost all of St. Ansel's work comes to mind. All of his early work was done with terrible lenses, by current standards. When looking through my work over the years, I can identify maybe 4% or so that would have come out substantially "better" had I used "better" equipment.

I bet I will take out my 50 year old Rolleiflex's a few times and make some great photos that print better at large sizes than some other shots I get out of the 40D. I'm also willing to bet there are a whole slew of people that are going to buy E-3s and produce better photos than I will with my 40D. What of it?
[a href=\"index.php?act=findpost&pid=176350\"][{POST_SNAPBACK}][/a]

Misirlou,
You've gone way off tangent here. This has got nothing to do with one person's ability to take an artistic shot with a Holga. Nor has it got anything to do with reasons other than absolute 'camera' image quality for making a decision to purchase a particular camera. We know there are all sorts of considerations other than absolute image quality for using or preferring one camera over another. A few weeks ago I bought a 40D myself, in Bangkok, simply because, whilst in the process of checking out the 17-55/2.8 IS lens, I discovered I could pick up a 40D body for less than US$1,000, which was much less than I had imagined the price would be. Also, I've always recognised the 40D has some useful features, in particular LiveView with manual focussing and 6.5 frames per second.

I've got no objection at all to different people using different criteria for making their choices. As you say, that is not surprising. In fact it is to be expected and it's a good thing. It's healthy and it's sensible. The concept of the best tool for the job should prevail in the decision making process.

However, when people comment on the performance of a camera they should always make a clear reference to whatever criterion they are using, otherwise their comments are not helpful and confusion reigns.

Quote
Some posters like to use words such as "quality" and "definite" rather indiscriminately.

You can say that again   .

Quote
One person reads the DPreview article and thinks it says the E-3 is less sharp than the 40D, and another poster reads it to say that it's a tossup below ISO 800. Is one of you wrong? Is one of you a fool? I don't know. Why do we need to get so worked up over it?

Worked up? Why do we bother at all? I agree that everything has to be interpreted, but some interpretations can be less accurate than others. Should we adopt the policy of keeping silent about such differences of interpretation?

Quote
Ray, a couple months ago, I made some RAW shots with my 20D and my 40D. When I looked at those shots, I was convinced that the 40D versions were better. You were not. I don't agree with the logic you applied to the situation. Is one of us wrong? I don't know how we could ever come to an agreement. You will not convince me that I am wrong, and I will not convince you. There is nothing more to say. What else do you expect?

Misirlou, my preamble to this thread was not directed only at you. There are a number of people who might think the 40D produces 'definitely" better image quality than the 20D, but your tests appear to have been done as an after-thought, (after you'd passed on the 20D to your mother). The tests were not conclusive. There was some ambiguity about the focussing. There are many other comparison tests that have been conducted and any opinion that I have is not that there is no image quality improvement whatsoever but that the image quality improvement is more marginal than one might expect from 14 bit processing plus 25% more pixels.

These are matters that can be discussed rationally without getting worked up about it, surely.  
Title: The truth about the E-3 Image Quality
Post by: Moynihan on February 21, 2008, 07:33:53 am
I have noticed that the IQ emphasis in reviews seems to be noise performance at high ISO, a subject that is of little interest to me.
I assume that these cameras (being electric) have a "base" noise level, that would even effect quality at their lowest ISO setting? I Assume this may also vary with ambient temperature, and other factors?
Question: Is good comparative performance at high ISO re noise, mean good comparative performance at low ISO re noise? Or are they un-related?
Title: The truth about the E-3 Image Quality
Post by: BJL on February 21, 2008, 12:12:16 pm
A rough reading of the tests at DPReview and elsewhere suggests that
- when each camera's maker handles the conversion to JPEG (i.e. in-camera), the E-3 does about as well as the 40D or maybe slightly better for sharpness.
- when a common third party Adobe RAW converter software is used, the Olympus JPEG's seem even a bit worse than in-camera conversion!
[added] - some reviews have indicated that raw conversions done with Olympus software (Studio, Master) offer better resolution than those done with Adobe's software. This a been reported for previous Olympus models also.

One possible interpretation is that Adobe's RAW conversion is not (yet) as well tuned for E-3 raw files as they are for Canon's 40D raw files. (Good raw conversion seems model specific, not just brand and raw file format specific.)

[edited] Maybe those seeking the ''truth'' about inherent IQ should study RAW conversions done by the camera maker's own software? Or by seeking the best raw converter for each camera, rather than trusting Adobe to handle all cameras equally well.
Title: The truth about the E-3 Image Quality
Post by: DarkPenguin on February 21, 2008, 12:26:31 pm
I really wish they would print their standard test scene at a variety of sizes.  Judge it from there.
Title: The truth about the E-3 Image Quality
Post by: Misirlou on February 21, 2008, 06:35:33 pm
Quote
Misirlou,
You've gone way off tangent here. This has got nothing to do with one person's ability to take an artistic shot with a Holga. Nor has it got anything to do with reasons other than absolute 'camera' image quality for making a decision to purchase a particular camera. We know there are all sorts of considerations other than absolute image quality for using or preferring one camera over another. A few weeks ago I bought a 40D myself, in Bangkok, simply because, whilst in the process of checking out the 17-55/2.8 IS lens, I discovered I could pick up a 40D body for less than US$1,000, which was much less than I had imagined the price would be. Also, I've always recognised the 40D has some useful features, in particular LiveView with manual focussing and 6.5 frames per second.

I've got no objection at all to different people using different criteria for making their choices. As you say, that is not surprising. In fact it is to be expected and it's a good thing. It's healthy and it's sensible. The concept of the best tool for the job should prevail in the decision making process.

However, when people comment on the performance of a camera they should always make a clear reference to whatever criterion they are using, otherwise their comments are not helpful and confusion reigns.
You can say that again   .
Worked up? Why do we bother at all? I agree that everything has to be interpreted, but some interpretations can be less accurate than others. Should we adopt the policy of keeping silent about such differences of interpretation?
Misirlou, my preamble to this thread was not directed only at you. There are a number of people who might think the 40D produces 'definitely" better image quality than the 20D, but your tests appear to have been done as an after-thought, (after you'd passed on the 20D to your mother). The tests were not conclusive. There was some ambiguity about the focussing. There are many other comparison tests that have been conducted and any opinion that I have is not that there is no image quality improvement whatsoever but that the image quality improvement is more marginal than one might expect from 14 bit processing plus 25% more pixels.

These are matters that can be discussed rationally without getting worked up about it, surely. 
[a href=\"index.php?act=findpost&pid=176371\"][{POST_SNAPBACK}][/a]

I'm not sure they can. My point with going off on the "tangent" was that the incredible proliferation in equipment and technique that people use points to a very real possibility that you'll never get very many photographers to agree on any standardized measure of image quality, "camera" or otherwise. You seem to be suggesting that it's unfortunate that people might reach a different conclusion than you when they look at the same data, and I'm saying it's not surprising, or even notable.

If there are people satisfied with Holgas, you can bet that there are people out there who will be satisfied with the high ISO performance of a 14MP point and shoot. [Some people actually bought those disc cameras, remember those?] I've read comments from testers stating that they prefer the noisy sensors in their cameras because their particular noise is "more like film grain" than other sensors. How can you argue with that? How could you quantify it? Can you prove somehow that they're wrong?

Consider MTF curves. I understand them, look at them from time to time, but have never once used one to guide the purchase of any lens over any other. Is that wrong? Foolish? I don't know. If someone derived a measure like that for sensor performance, would I use it? Maybe not. It's just not that important to me, at least when compared with all the other variables. If you come up with one, I'll follow along, but I'll only give the results a small amount of weight in the overall choice.

I agree that it would be great if testers were more specific when describing their procedures and their judgement criteria, but at some point, the burden of that specificity becomes so excessive that nobody will bother to go through with it. Even Mr. Reichman curtailed his DXO testing process.

My own experience is instructive. You're darn right my 20D/40D test was an afterthought. I had no intention of exhaustively testing anything; I was merely providing some data to those who might be interested, since I actually had both cameras on hand for a brief time. I have neither the time, the patience or the incliniation to do precision testing work. And after the blowback from this one, I'm not likely to ever bother with anything similar in the future.

I would like something like DarkPenguin's approach: Let the testing sites publish some unmanipulated RAW shots of repeatable scenes from new cameras, at assorted sizes (including at least one full rsolution capture), and leave it at that.
Title: The truth about the E-3 Image Quality
Post by: panoak on February 21, 2008, 08:03:35 pm
Just thru the process of reading the new E-3 test, and then some field testing, and then more remarks about the Pentax K-20,  I finally understood....  Pixel peeping is a secondary hobby, not to be confused or allowed to intrude on the primary interest.  For the overwhelming majority of "important" imaging requirements, we came into that realm when Oly produced the E-1 and everyone else adopted the 6 Mp. Sony DSLR sensor.  There have been gains in higher ISO performance, but almost nothing in any other field of IQ.
     At this point, it matters almost not at all which modern DSLR you prefer.  If you don't intend to print big, there is still no urgent need to look askew at a Nikon D-70, or anything in that generation.  Post processing nullifies ALL distinction, and defines the true intent of the photographer.  It doesn't matter whether your writing slants to the right or left.  The important thing is to engage in the telling of your story.
Title: The truth about the E-3 Image Quality
Post by: Ray on February 21, 2008, 08:21:37 pm
Quote
A rough reading of the tests at DPReview and elsewhere suggests that
- when each camera's maker handles the conversion to JPEG (i.e. in-camera), the E-3 does about as well as the 40D or maybe slightly better for sharpness.
- when a common third party Adobe RAW converter software is used, the Olympus JPEG's seem even a bit worse than in-camera conversion!
[added] - some reviews have indicated that raw conversions done with Olympus software (Studio, Master) offer better resolution than those done with Adobe's software. This a been reported for previous Olympus models also.

One possible interpretation is that Adobe's RAW conversion is not (yet) as well tuned for E-3 raw files as they are for Canon's 40D raw files. (Good raw conversion seems model specific, not just brand and raw file format specific.)

[edited] Maybe those seeking the ''truth'' about inherent IQ should study RAW conversions done by the camera maker's own software? Or by seeking the best raw converter for each camera, rather than trusting Adobe to handle all cameras equally well.
[a href=\"index.php?act=findpost&pid=176430\"][{POST_SNAPBACK}][/a]

BJL,
You've brought up these sorts of issues before. I remember once when I commented that dpreviews tests of a previous model of the 4/3rds system showed images being marginally less sharp than even a P&S camera of similar pixel count, which was included in the comparison, you pointed out the fact that the images being compared were all out-of-camera jpegs and that RAW images would produce a better result; a valid point, but you never provided any link to such a comparison based on RAW images, so the matter remained unresolved.

In this latest comparison of the E-3, dpreview has produced out-of-camera jpegs which are actually perhaps marginally sharper than the 40D, confirming that one can't necessarily judge a camera on the basis of its default jpeg output, but this time they've also compared RAW output and found the E-3 RAW images to be marginally less sharp.

Now, I admit the first thing that springs to mind is that perhaps ACR is not optimised for use with the E-3. Perhaps Olympus Studio 2 would do better conversions. The thought also occurred to the dpreview team. If you read the review more carefully, you'll find that they also used Olympus's own software to do the conversions (no doubt because they also were surprised at the result in ACR), but found that it did not change matters. They have concluded that the slightly less sharp results from RAW conversions are due to a stronger than average AA filter on the E-3.

Once again, none of these differences in image quality are of any great significance. They are no great impediment to the taking of great, artistic photos if your talent lies in that direction. If all the other features and advantages of the 4/3rds format suit your purpose and shooting style, it would be silly to choose a 40D instead just because it has marginally more highlight headroom, marginally sharper RAW images and significantly less noise at maximum ISO   .
Title: The truth about the E-3 Image Quality
Post by: BJL on February 21, 2008, 08:53:56 pm
Quote
... so the matter remained unresolved.

Once again, none of these differences in image quality are of any great significance.
[a href=\"index.php?act=findpost&pid=176526\"][{POST_SNAPBACK}][/a]
The second point is the important one, so I am happy to leave the hair-splitting resolution comparison unresolved, So if other reviews came to different conclusions than DPReview about E-3 sharpness with different choices of raw conversion, the main import to me is that the differences are probably "within experimental error", especaily of the experimentis printe viewing.
So I will stay with my comforting, simplistic dogma that

In general (and excepting Foveon's X3 type sensors), DSLR sensor resolution is reasonably well measured by pixel counting: the "lines per picture height" of resolution are close enough to proportional to the "pixels per frame height".



Hence my far greater interest in ergonomic, lens selection and features of use to me in comparing cameras of similar pixel count.
Title: The truth about the E-3 Image Quality
Post by: Ray on February 21, 2008, 09:04:53 pm
Quote
Just thru the process of reading the new E-3 test, and then some field testing, and then more remarks about the Pentax K-20,  I finally understood....  Pixel peeping is a secondary hobby, not to be confused or allowed to intrude on the primary interest.  For the overwhelming majority of "important" imaging requirements, we came into that realm when Oly produced the E-1 and everyone else adopted the 6 Mp. Sony DSLR sensor.  There have been gains in higher ISO performance, but almost nothing in any other field of IQ.
     At this point, it matters almost not at all which modern DSLR you prefer.  If you don't intend to print big, there is still no urgent need to look askew at a Nikon D-70, or anything in that generation.  Post processing nullifies ALL distinction, and defines the true intent of the photographer.  It doesn't matter whether your writing slants to the right or left.  The important thing is to engage in the telling of your story.
[a href=\"index.php?act=findpost&pid=176521\"][{POST_SNAPBACK}][/a]

That is certainly a counterpoint to the pixel-peeping pnenomenon, but basically this argument can be summarised in one sentence. 'If the finer points of image quality don't matter, then it doesn't matter which DSLR you buy.'

If you are going to impose restrictions on your purposes and the use to which your images are put, then those restrictions will define the tools you need.

Those who never bother making prints larger than postcard size should be quite satisfied with any P&S camera whatever the pixel count. Even a cellphone camera might suffice.

Those who never shoot in low light without a flash, will not be particularly excited by low noise performance at high ISO, although one might wonder why anyone would impose such a restriction upon themselves.

Those who have no intention of printing larger than A4 might well see no reason to upgrade from a 5mp E-1 or the 3mp D30.

But I certainly don't fall into that category, nor do many people who make a living (or aspire to make a living) out of photography. I certainly don't make a living from photography sales, but I am an enthusiast, I do have a wide format printer and fundamental image quality is important to me, but not so important that I would encumber myself with a lot of heavy and very expensive equipment for use with MFDBs.
Title: The truth about the E-3 Image Quality
Post by: Ray on February 21, 2008, 09:21:45 pm
Quote
The second point is the important one, so I am happy to leave the hair-splitting resolution comparison unresolved, So if other reviews came to different conclusions than DPReview about E-3 sharpness with different choices of raw conversion, the main import to me is that the differences are probably "within experimental error", especaily of the experimentis printe viewing.
[a href=\"index.php?act=findpost&pid=176531\"][{POST_SNAPBACK}][/a]

Which other reviews are you referring to, BJL? Can you provide a link? Perhaps the methodology in such reviews is flawed.

The thrust of my argument in this thread is not that the E-3 may not be a worthy competitor to the 40D, D300 etc, but rather I'm questioning how it is possible for some people to apparently get it so wrong with their dogmatic claims of definite superior image quality compared with the 40D and even 5D, especially when such people claim they are shooting RAW.

To put it bluntly, are such people just kidding themselves or is it perhaps the lens they are using on the E-3 is so much better by a degree greater than what the smaller sensor requires for equal picture resolution?
Title: The truth about the E-3 Image Quality
Post by: Anthony R on February 21, 2008, 09:29:44 pm
Ray. Get a new hobby. Why do you care? (other than having nothing else to do)
Title: The truth about the E-3 Image Quality
Post by: Ray on February 21, 2008, 10:10:11 pm
Quote
I'm not sure they can. My point with going off on the "tangent" was that the incredible proliferation in equipment and technique that people use points to a very real possibility that you'll never get very many photographers to agree on any standardized measure of image quality, "camera" or otherwise. You seem to be suggesting that it's unfortunate that people might reach a different conclusion than you when they look at the same data, and I'm saying it's not surprising, or even notable.

Misirlou,
You are still off at a tangent. This is not about differences of opinion amongst people who are looking at the same data, that is the same images. I don't think there is any difference of interpretation of the dpreview data that I've seen mentioned in this thread. All the differences are marginal with the exception of noise at high ISO.

What would constitute a difference of interpretation would be a statement along the lines, "I think the E-3 RAW images shown in the Dpreview test are actually sharper than the 40D RAW images".

One has to make a distinction here between differences of interpretation of what's actually being viewed; differences of opinion as to the significance of the observed differences, in practice and in prints, and differences of opinion as to the cause of such observed differences.

Quote
If there are people satisfied with Holgas, you can bet that there are people out there who will be satisfied with the high ISO performance of a 14MP point and shoot. [Some people actually bought those disc cameras, remember those?] I've read comments from testers stating that they prefer the noisy sensors in their cameras because their particular noise is "more like film grain" than other sensors. How can you argue with that? How could you quantify it? Can you prove somehow that they're wrong?

Once again, this issue I've raised is not about preferences or taste. There is sometimes just no accounting for taste. However, I believe if you show any group of people two photos, one of which is noisier than the other, then no-one is likely to consider the noisier image as being cleaner, if he know what noise is.

Likewise, if you show a group two photos, one of which is sharper and/or more detailed than the other, then no-one is likely to interpret the slightly blurrier image as being sharper, if he knows what sharp means.

Whether or not someone prefers the blurrier image is a matter of taste. That's another matter entirely.

Quote
I would like something like DarkPenguin's approach: Let the testing sites publish some unmanipulated RAW shots of repeatable scenes from new cameras, at assorted sizes (including at least one full rsolution capture), and leave it at that.

I'd agree with that approach. However, I can see a few snags here. Some folks may not have the latest version of ACR or any other converter that can handle the RAW images. RAW images are large and many images are required for a full assessment. There could be bandwidth problems both for the review site and some internet users who are still on dial-up. Some folks only shoot jpeg and that's all they are interested in. Some tests to compare dynamic range and noise at low ISO, for example, may require more technical expertise than most viewers possess.
Title: The truth about the E-3 Image Quality
Post by: SecondFocus on February 21, 2008, 11:44:09 pm
Quote
Ray. Get a new hobby. Why do you care? (other than having nothing else to do)
[a href=\"index.php?act=findpost&pid=176539\"][{POST_SNAPBACK}][/a]

Anthony;

I just looked through your website; beautiful work!
Title: The truth about the E-3 Image Quality
Post by: Ray on February 22, 2008, 12:02:22 am
Quote
Ray. Get a new hobby. Why do you care? (other than having nothing else to do)
[a href=\"index.php?act=findpost&pid=176539\"][{POST_SNAPBACK}][/a]

Anthony,
I have plenty of other things to do. More things than my brief moment on this planet will allow. But I find it a sensible approach to get hobbies which interest one.

I am now going out into the yard to prune some tree that are interfering with my TV antenna. Pruning trees, however, is not a hobby.
Title: The truth about the E-3 Image Quality
Post by: schrodingerscat on February 22, 2008, 12:46:42 am
Quote
It always amazes me how quick people are to champion the superior attributes of a new camera they have just bought. They seem to become like salseman for their new equipment providing free advertising for the manufacturer. To the more rational amongst us, their claims often seem to defy credibility.

I'll just mention a few recent claims which I viewed with great skepticism.

1. The image quality from the 40D is virtually as good as the 5D, perhaps better in some respects.

2. The 40D image quality is a definite improvement over the 20D and 30D.

3. The E-3 definitely produces sharper images than the 40D.

4. The E-3 produces better and sharper images than the 5D. (And I know because I'm a working photographer who publishes in magazines and I've used both cameras, but I'm too busy to carry out tests.)

5. The Nikon D3 has about a 2 stop noise improvement at high ISO, compared with anything else on the market.

6. The Phase P21 DB definitely produces a more sold looking image with a greater sense of 3-dimensionality than the 1Ds3 can produce. You either see it or you don't.

Often when one questions such claims, the people making the claim are unable to provide the evidence for whatever reason, usually because they are too busy perhaps, but sometimes I think because they want to let the myth ride, whilst it can.

I now see that Dpreview have just completed their review of the Olympus E-3. There are no surprises for me, expect perhaps one. The E-3 actually produces image which are less sharp than the 40D when shooting RAW. This is apparently because the E-3 uses a slightly stronger than average AA filter which limits the potential to capture sharp detail. The lens used on the E-3 during the tests is considered by Dpreview to be one of the sharpest optics they have ever used, the Olympus 50/F2, so I don't think that claims that the lens used on the E-3 might be substandard will hold.

The other image quality characteristics are no surprise. The E-3 is definitely noisier than the 40D at ISO 1600 and above. Highlight headroom is slightly less so the camera is more prone to clipping of highlights. Jpeg output is slightly sharper and punchier (more contrast and saturation) which is no doubt the reason why some people got the impression the E-3 produced even better results than the 5D.

Of course, needless to say, there are more features to a camera than fundamental image quality. That's not in question. Here I'm only addressing fundamental image quality, not range of lenses available, speed of focussing or wieght and waterproofing advantages etc etc.
[a href=\"index.php?act=findpost&pid=176311\"][{POST_SNAPBACK}][/a]


Chevy...Ford...MOPAR

There is no truth, just ask Mr Heisenberg.
Title: The truth about the E-3 Image Quality
Post by: ErikKaffehr on February 22, 2008, 01:31:53 am
Hi,

I think that some of the issue is that the Olympus DSLRs have pretty strong AA (Anti Aliasing) filter. That is not necessarily a bad thing, but means that they need more sharpening.

Now, I think that DPReview tries to keep sharpening the same between different cameras which would put the Oly at disadvantage.

Regarding JPEGs the situation is that amateur cameras almost always oversharpen (by default settings) while professional cameras sharpen much less. The reason is that JPEGs in amateur cameras are intended for viewing/printing but in a professional workflow some more image processing would be made.

The best way to compare cameras would probably be to compare RAW-conversions followed by optimal "capture sharpening". Deciding what is optimal is not easy, tough.

Sharpening would increase noise. Lightroom can use edge masks, so sharpening is limited to "edges" or parts having significant detail.

A good example is given in "Real Word Image Sharpening with Adobe Photoshop CS2" by the late Bruse Fraser. He compares a 6 MPixel Kodak DCS 460 with a Canon 300 D. The Kodak has no AA-filter while the Canon has. In Bruce's example the Kodak needs radius "0.6" and "amount" 210 while the Canon needs radius "0.6" and "amount" 500.

Best regards
Erik

Quote
BJL,
You've brought up these sorts of issues before. I remember once when I commented that dpreviews tests of a previous model of the 4/3rds system showed images being marginally less sharp than even a P&S camera of similar pixel count, which was included in the comparison, you pointed out the fact that the images being compared were all out-of-camera jpegs and that RAW images would produce a better result; a valid point, but you never provided any link to such a comparison based on RAW images, so the matter remained unresolved.

In this latest comparison of the E-3, dpreview has produced out-of-camera jpegs which are actually perhaps marginally sharper than the 40D, confirming that one can't necessarily judge a camera on the basis of its default jpeg output, but this time they've also compared RAW output and found the E-3 RAW images to be marginally less sharp.

Now, I admit the first thing that springs to mind is that perhaps ACR is not optimised for use with the E-3. Perhaps Olympus Studio 2 would do better conversions. The thought also occurred to the dpreview team. If you read the review more carefully, you'll find that they also used Olympus's own software to do the conversions (no doubt because they also were surprised at the result in ACR), but found that it did not change matters. They have concluded that the slightly less sharp results from RAW conversions are due to a stronger than average AA filter on the E-3.

Once again, none of these differences in image quality are of any great significance. They are no great impediment to the taking of great, artistic photos if your talent lies in that direction. If all the other features and advantages of the 4/3rds format suit your purpose and shooting style, it would be silly to choose a 40D instead just because it has marginally more highlight headroom, marginally sharper RAW images and significantly less noise at maximum ISO   .
[a href=\"index.php?act=findpost&pid=176526\"][{POST_SNAPBACK}][/a]
Title: The truth about the E-3 Image Quality
Post by: Ray on February 22, 2008, 06:08:39 am
Quote
A good example is given in "Real Word Image Sharpening with Adobe Photoshop CS2" by the late Bruse Fraser. He compares a 6 MPixel Kodak DCS 460 with a Canon 300 D. The Kodak has no AA-filter while the Canon has. In Bruce's example the Kodak needs radius "0.6" and "amount" 210 while the Canon needs radius "0.6" and "amount" 500.
[a href=\"index.php?act=findpost&pid=176586\"][{POST_SNAPBACK}][/a]
 

That's a fair point. I think dpreview used an amount of 80% USM, radius 1 pixel and threshold zero in PS on all RAW images. In ACR, sharpening was set at zero and other adjustments at default.

I imagine increasing the amount of sharpening would be necessary to get the E-3 image looking as sharp as the 40D image, but as you say, sharpening also tends to increase noise and produce halos.
Title: The truth about the E-3 Image Quality
Post by: Ray on February 22, 2008, 11:45:07 am
I've just been reading some of the responses to the dpreview E-3 test on the dpreview forum, and I think I can now understand why Michael has disengaged himself from all attempts to provide objective and detailed reviews of equipment.

No matter how much trouble you take to be as objective as you possibly can be, there always seems to be a barrage of criticism from disaffected owners of the equipment you've demonstrated might not be quite up to to scratch.

They seem to squirm and wriggle and try with all their might to escape from the "facts" like a kangaroo caught by its tail.

I don't post on dpreview, but I wish Simon Joinson well. Keep your cool, mate.
Title: The truth about the E-3 Image Quality
Post by: BJL on February 25, 2008, 05:17:37 pm
Quote
Which other reviews are you referring to, BJL? Can you provide a link? Perhaps the methodology in such reviews is flawed.
[a href=\"index.php?act=findpost&pid=176536\"][{POST_SNAPBACK}][/a]
Since I have already said that I see little value in such resolution comparisons, while you apparently care a lot, I suggest that you search around: DPreview is far from the only or best DSLR testing site on the internet.

I am struck by your pre-emptive speculation that these other sites are using flawed methodolgy while your often highly critical approach to some data and testing methods seems to have overlooked the various potential flaws of the DPReview methodology discussed in the DPReview forums.

If you truly care about methodology and seeking to measure such small differences in resolution, here is a nit-picking point. DPReview did the E-3 resolution testing with the 50mm f/2 lens at f/6.3, an aperture shown in testing elsewhere to be not the sharpest available. (That lens is good enough that the resolution keeps improving down to about f/2.8, and indeed most 4/3 lenses are sharper at f/4 than f/6.3). So the resolution measurements are held back somewhat by lens limitations; specifically diffraction due to using a smaller than optimal aperture diameter.

The same is likely to be true of the other cameras tested too, so all around, diffraction effects have probably not been sufficiently controlled in the DPReview testing.
Title: The truth about the E-3 Image Quality
Post by: BJL on February 25, 2008, 05:33:06 pm
Quote
... there always seems to be a barrage of criticism from disaffected owners of the equipment you've demonstrated might not be quite up to to scratch.
[a href=\"index.php?act=findpost&pid=176660\"][{POST_SNAPBACK}][/a]
(Emphasis added)
Ray, this dismissal of criticisms of reviews and testing methodology by people who have read the reviews is ironical given that in another post in this thread, your first speculation about reviews that you have not read is that they might use flawed methodology. You (like I!) certainly have a record of vigorously disputing other people's evidence, arguments and methodology.
Title: The truth about the E-3 Image Quality
Post by: Ray on February 25, 2008, 10:51:15 pm
Quote
Since I have already said that I see little value in such resolution comparisons, while you apparently care a lot, I suggest that you search around: DPreview is far from the only or best DSLR testing site on the internet.

BJL,
It's standard practice in academic circles to always quote your sources. If you don't care about an issue, why raise the matter in the first instance or engage in debate?

Quote
I am struck by your pre-emptive speculation that these other sites are using flawed methodolgy while your often highly critical approach to some data and testing methods seems to have overlooked the various potential flaws of the DPReview methodology discussed in the DPReview forums.

Which other sites are you referring to? I always start from the premise that methodology possibly could be flawed, especially when I attempt tests myself. It is true, however, that sites such as Dpreview with a reputation for a meticulous and consistent approach, tend to generate less skepticism regarding sound methodology than do unknown, unspecified or rumoured sources.

Quote
If you truly care about methodology and seeking to measure such small differences in resolution, here is a nit-picking point. DPReview did the E-3 resolution testing with the 50mm f/2 lens at f/6.3, an aperture shown in testing elsewhere to be not the sharpest available. (That lens is good enough that the resolution keeps improving down to about f/2.8, and indeed most 4/3 lenses are sharper at f/4 than f/6.3). So the resolution measurements are held back somewhat by lens limitations; specifically diffraction due to using a smaller than optimal aperture diameter.

That's a good point. At face value, it does seem that dpreview might have blundered here. When sharpness (or resolution) is an issue, why not use both lenses at their sharpest aperture. In this situation, perhaps it would have been fairer to use the Olympus 50/2 at f2.8 and the Canon 50/1.4 at f4. PhotoZone tests show that the Canon 50/1.4 peaks in sharpness at F4, at least in the centre at MTF 50.

Checking the review again, I find the following statement from Dpreview.

Quote
Lenses used

For direct comparisons we always use sharp prime lenses stopped down, typically to F9 for 35 mm lenses and F6.3 for Four Thirds lenses. Here we have used the Minolta 50mm F1.4, Pentax 50 mm F1.4, Olympus 50 mm F2.0 Macro, and Canon EF 50 mm F1.4.

Perhaps the reason for using F9 with the 40D and F6.3 with the E-3 is for the sake of consistent results across the frame, from edge to edge. One can only speculate on the degree to which the Olympus 50/2 might be sharper at f2.8 than it is at f6.3. But it's clear from the Photozone tests that the Canon 50/1.4 is definitely sharper at F4 than it would be at f9, (assuming that centre sharpness at F9 would be marginally less, or no better than at F8. They don't show the results for F9).

Nevertheless, if I had been carrying out these tests myself, suspecting that the Olympus 50/2 might be sharpest at f2.8, I would have taken some more shots of the test target at wider apertures to see if that made a difference to the RAW image comparisons. Perhaps Simon Joinson actually did that, just as he also used Olympus Studio 2. I notice that Dpreview are now branching out into lens reviews.
Title: The truth about the E-3 Image Quality
Post by: Deep on February 27, 2008, 03:33:41 am
I download all the dpreview sample photos taken by cameras that interest me.  They are the most objective standard out there and I am very grateful to Phil, Simon and the team for what they do.

BUT:  there did seem to be a strange thing happening with the E3 samples.  The RAW one looked disturbingly soft (as everyone seems to have picked up).  I have Olympus cameras and am aware that if I use Lightroom or Photoshop, the files seem to need more sharpening than other files.  So I did the obvious, which was to gently move the sharpen and detail sliders in Lightroom a very small amount and, hey presto!, all that hidden detail sprung up.  And we are not talking about excessive sharpening either.  Now I compare the detail with any other 10-12 MP camera there and only the 5D has recorded more information (and I do try playing with settings on samples from other cameras too).  If you look for the finest details in the scene, the E3 has plucked them out, every bit as well as the RAW L10 file  that Simon heaped so much praise on.

As for the excessive AA filter - actually I think it is the opposite.  Once the file has been sorted, look at the writing on the tin man and you will see jaggies.

But yeah, it's all pretty irrelevant, 99% of the time.  I have an exhibition in town at the moment and buyers don't know, or care, where the images came from.  Those sold so far came from my best DSLR, two digital point and shoot cameras and film!

Really, we are utterly spoilt these days.

Don.
Title: The truth about the E-3 Image Quality
Post by: Ray on February 27, 2008, 09:10:45 am
Quote
BUT:  there did seem to be a strange thing happening with the E3 samples.  The RAW one looked disturbingly soft (as everyone seems to have picked up).  I have Olympus cameras and am aware that if I use Lightroom or Photoshop, the files seem to need more sharpening than other files.  So I did the obvious, which was to gently move the sharpen and detail sliders in Lightroom a very small amount and, hey presto!, all that hidden detail sprung up.  And we are not talking about excessive sharpening either.  Now I compare the detail with any other 10-12 MP camera there and only the 5D has recorded more information (and I do try playing with settings on samples from other cameras too).  If you look for the finest details in the scene, the E3 has plucked them out, every bit as well as the RAW L10 file  that Simon heaped so much praise on.

As for the excessive AA filter - actually I think it is the opposite.  Once the file has been sorted, look at the writing on the tin man and you will see jaggies.
[a href=\"index.php?act=findpost&pid=177652\"][{POST_SNAPBACK}][/a]

I don't see any significant difference between the E-3 RAW image on dpreview and the other RAW images with which it's compared. It's all very trivial.

My point really is, how come some people who claim they always shoot RAW are able to assert that the E-3 produces not only better and sharper images than the 40D but even sharper than the 5D?

It's tests like dpreview's that bring us down to reality, and that's a good thing.

As regards the strength of the AA filter on the E-3, I'm having trouble with your logic. If you think it's not stronger but actually weaker, why does the E-3 RAW need slightly more sharpening than the 40D image in order to bring out the detail? How do both images compare after they've both been given the same amount of additional sharpening? Are you claiming that the E-3 image then looks right but the 40D image looks oversharpened?
Title: The truth about the E-3 Image Quality
Post by: BJL on February 27, 2008, 06:11:21 pm
Quote
I don't see any significant difference between the E-3 RAW image on dpreview and the other RAW images with which it's compared. It's all very trivial.

My point really is, how come some people who claim they always shoot RAW are able to assert that the E-3 produces not only better and sharper images than the 40D but even sharper than the 5D?
[a href=\"index.php?act=findpost&pid=177685\"][{POST_SNAPBACK}][/a]
if that is your only point, then perhaps I have been wasting your time and mine. We seem to agree that the differences are "trivial" [your word] or "within the margin of experimental error" [my suggestion]. I will stick to counting pixels to estimate image resolution potential of a sensor.

(Which is why I do not feel the need to seek out citations to support a passing comment about other reviews that I read in passing!)


P. S. The measured total dynamic range at low to medium ISO speeds, 100-400, (ignoring the rather meaningless highlight-to-midtone and midtone-to-shadow splits) also seem about equal, or within experimental error, for the 40D, E-3, A700 and D300. And yet much blood is being spent debating those too!
Title: The truth about the E-3 Image Quality
Post by: Quentin on February 27, 2008, 06:47:42 pm
Quote
My point really is, how come some people who claim they always shoot RAW are able to assert that the E-3 produces not only better and sharper images than the 40D but even sharper than the 5D?

It's tests like dpreview's that bring us down to reality, and that's a good thing.


[a href=\"index.php?act=findpost&pid=177685\"][{POST_SNAPBACK}][/a]

Another review, another opinion.

For example, the E-3 is reviewed this month in a UK magazine called Digital Photographer,  a semi-pro magazine, which rates the E-3 image quality 10/10, saying of the quality of results "A synergy of body and lenses that produce a resolving power and an edge-of-frame sharpness currently unrivaled".  

The overall score is 96% (which is I think the highest mark they have ever given to a DSLR - the 5D got 90%), and the concluding paragraph opines the E-3 is "...one of the most rounded cameras ever tested by Digital Photographer." I could go on; the rest of the fairly detailed review is in a similar vein.  It is the very definition of a rave review.

Are they right about the E-3 or is the dpreview more accurate?  Is the Digital Photographer review the "Truth" about the E-3? Its clearly not your truth, Ray, but it is the view of a reasonably respected journal - but while the Digital Photographer review waxes lyrical about the E-3, it is, when all is said and done, just another opinion.  

You seem pretty wedded, Ray, to your Canon cameras and many of your posts seem to be defending them against others, including the fairly relentless if ultimately unconvincing attempt to show that the 5D noise at high ISO was nearly a match for the D3. The fact is all mid to top end dslrs now are excellent, you really cannot go far wrong with any of them.  

Just my opinion...of course  

Quentin
Title: The truth about the E-3 Image Quality
Post by: Ray on February 27, 2008, 07:48:43 pm
Quote
The overall score is 96% (which is I think the highest mark they have ever given to a DSLR - the 5D got 90%), and the concluding paragraph opines the E-3 is "...one of the most rounded cameras ever tested by Digital Photographer." I could go on; the rest of the fairly detailed review is in a similar vein.  It is the very definition of a rave review.

Are they right about the E-3 or is the dpreview more accurate?  Is the Digital Photographer review the "Truth" about the E-3? Its clearly not your truth, Ray, but it is the view of a reasonably respected journal - but while the Digital Photographer review waxes lyrical about the E-3, it is, when all is said and done, just another opinion. 

Quentin,
There's only one issue here for me. How accurate and how reliable are some reviews. Objective reviews are supposed to attempt to cut through mere subjective opinion. All reviews are not objective. All tests are not thoroughly and meticulously carried out and for all the reader knows, there may be occasions when one particular review is a borrowed opinion or simply a biased opinion because the site or magazine carries advertising for Olympus or because the reviewer owns an Olympus 4/3rds system or has shares in the company.

Whatever the reason for such discrepancies of opinion or test results, the cause of such discrepancies is surely of interest to all pixel peepers.

If I personally were to buy a camera because because a few sources (individuals and professional reviewers) had made claims for it that later proved to be exaggerations, I might be a little annoyed.

Quote
You seem pretty wedded, Ray, to your Canon cameras and many of your posts seem to be defending them against others, including the fairly relentless if ultimately unconvincing attempt to show that the 5D noise at high ISO was nearly a match for the D3. The fact is all mid to top end dslrs now are excellent, you really cannot go far wrong with any of them. 

I might seem to be, but I'm not. I can only make personal comparisons with equipment that I own. I rely upon reviews for most comparisons of cameras. I recently attempted to compare my 5D with a D3 under conditions which were not ideal and within a time frame which was rushed. I couldn't claim my methodology was flawless and therefore I didn't make the RAW images freely available.

I look forward with interest to the release of the Sony 24mp FF DSLR. If it's noticeably better than the upgrade to the 5D and not too expensive, I'll probably buy it. Fortunately I already have a few Minolta lenses which I was using before I switched to Canon 7 or 8 years ago. I have no specific allegiance to Canon, apart from the obvious convenience of already having a bunch of Canon lenses.
Title: The truth about the E-3 Image Quality
Post by: SecondFocus on February 27, 2008, 08:27:24 pm
Quentin you are so very correct.

I glanced through the overly ponderous review on DPReview and just could not relate to it. Does anyone actually go out and shoot photos! I am not impressed.

These things can be used as a guideline perhaps to learn about features but using gear on real shoots is where it all gets sorted out. Like a friend of mine once said, some things seem pretty good until you take it out into the jungle and your life depends on it. That wasn't about cameras of course.

I stand by all the good things I have said about the E-3 on my blog and I am certainly not alone. When I saw the title of this thread with it's subtitle pop up a while ago, it sure gave me a laugh.
Title: The truth about the E-3 Image Quality
Post by: Panopeeper on February 27, 2008, 10:02:58 pm
Quote
These things can be used as a guideline perhaps to learn about features but using gear on real shoots is where it all gets sorted out
This attitude leads to boasting with 100% center crops of the cat, "proving" how god the camera is.

You can judge a camera with experimenting, i.e. shooting with it, but that is no basis to any comparison; that is plain nonsense.

For a few days ago I read this on DPReview:

the D3 is 2 stops better then the D300 over 800 ISO....

I asked how the poster came to the 2 stops (not 1, not three); the answer was

subjective, based on experience.

Now, that's what I call BS.
Title: The truth about the E-3 Image Quality
Post by: Quentin on February 28, 2008, 07:12:00 am
Ray,

Objectivity is unattainable.  Even issues like sharpness and colour are ultimately subjective.  If someone tells me their camera is better than mine, it only makes sense if I use that camera and agree.  If I don't agree, its not better, its worse.  

Probably like you, I am pretty fed up with reviews that spout pseudo scientifc bull. Thus, Michael's reveiws are better than most because they make no attempt at false science.  So long as we bear that in mind when we debate the merits of camera gear we'll be a lot more understanding of why there can be such a wide range of different views.

Quentin
Title: The truth about the E-3 Image Quality
Post by: Deep on February 28, 2008, 01:39:22 pm
Quote
As regards the strength of the AA filter on the E-3, I'm having trouble with your logic. If you think it's not stronger but actually weaker, why does the E-3 RAW need slightly more sharpening than the 40D image in order to bring out the detail? How do both images compare after they've both been given the same amount of additional sharpening? Are you claiming that the E-3 image then looks right but the 40D image looks oversharpened?
[a href=\"index.php?act=findpost&pid=177685\"][{POST_SNAPBACK}][/a]

The beauty of the digital age is that you do not have to take my word for it.  Repeat my experiment for yourself and you'll be amazed.  And yes, I imagine the 40D would look oversharpened at the same settings.  What counts for me is the detail I can actually resolve at the optimum settings and I don't really care if those settings are different for different cameras.  I know it is quite different for my Sony, for example.

Don