Pages: [1] 2 3 4   Go Down

Author Topic: Sony A700 Dynamic Range test: 9.5 f-stops  (Read 52703 times)

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Sony A700 Dynamic Range test: 9.5 f-stops
« on: December 10, 2007, 03:42:34 pm »

DISCLAIMER: I made a mistake accounting the f-stops zones in the zone map, so the DR is less that it claims the Title.

In the same way I did a subjective test to check how much DR is usable in the Nikoon D3 (Nikon D3 Dynamic Range Test: 9 f-stops), I repeat it here for the Sony A700. My final verdict is: 9 f-stops of usable DR.
The same processing and criteria applied on both tests (see Nikon's link).

This was the high DR scene:




A bit brighter to distinguish what's going on:




And the noise tests on the 9th f-stop areas, marked as -9EV (the 9th f-stop is clearly usable):




It would be great to have all the cameras at the same scene to compare. Unluckily it's difficult to put all that expensive stuff together.
« Last Edit: December 14, 2007, 04:14:15 pm by GLuijk »
Logged

marcmccalmont

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1780
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #1 on: December 10, 2007, 04:42:18 pm »

I could send you a P1 P30 RAW file for comparison?
Marc
Logged
Marc McCalmont

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #2 on: December 10, 2007, 04:48:45 pm »

Quote
I could send you a P1 P30 RAW file for comparison?
Marc
[a href=\"index.php?act=findpost&pid=159749\"][{POST_SNAPBACK}][/a]

sure, do it trough yousendit.com at gluijk(at)hotmail.com
Make sure you shoot a high dynamic range scene (like those with the bright window and very dark room). It's important that many textured (wood, clothes, floor tiles) objects in different degrees of shadow appear on the scene. It's also recommended if you use a tripod in order not to confuse noise with hand blur (what's the term for this in English?).
« Last Edit: December 10, 2007, 04:52:08 pm by GLuijk »
Logged

Kenneth Sky

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 463
    • http://
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #3 on: December 10, 2007, 06:17:09 pm »

Thank you for at least evaluating a part of IQ on the A700. None of the on line reviews that started the previews of this camera when it was announced have completed their evaluations or stated any conclusions
Logged

marcmccalmont

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1780
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #4 on: December 11, 2007, 12:52:59 am »

Quote
sure, do it trough yousendit.com at gluijk(at)hotmail.com
Make sure you shoot a high dynamic range scene (like those with the bright window and very dark room). It's important that many textured (wood, clothes, floor tiles) objects in different degrees of shadow appear on the scene. It's also recommended if you use a tripod in order not to confuse noise with hand blur (what's the term for this in English?).
[a href=\"index.php?act=findpost&pid=159751\"][{POST_SNAPBACK}][/a]

I'll be home in a couple of days and will take a HDR image with both the P30 and a 5D, should be interesting
Marc
Logged
Marc McCalmont

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #5 on: December 11, 2007, 04:21:40 am »

Quote
I'll be home in a couple of days and will take a HDR image with both the P30 and a 5D, should be interesting

good. I bet the result of the 5D will surprise many (for being worse than expected).

douglasf13

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 547
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #6 on: December 11, 2007, 07:38:49 pm »

Quote
good. I bet the result of the 5D will surprise many (for being worse than expected).
[{POST_SNAPBACK}][/a]

These DIWA tests seem to confirm your results.  

[a href=\"http://www.diwa-labs.com/wip4/test_result.epl]http://www.diwa-labs.com/wip4/test_result.epl[/url]
Logged

laughfta

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 89
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #7 on: December 11, 2007, 08:47:57 pm »

Quote
It's also recommended if you use a tripod in order not to confuse noise with hand blur (what's the term for this in English?).
[a href=\"index.php?act=findpost&pid=159751\"][{POST_SNAPBACK}][/a]

'Camera shake' is I think the term you are looking for, Guillermo! Btw, why do you think the 5D will be worse than expected?
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #8 on: December 12, 2007, 01:25:04 am »

Quote
good. I bet the result of the 5D will surprise many (for being worse than expected).
[{POST_SNAPBACK}][/a]

Guillermo,
If the Sony A700 really does beat the 5D in terms of high ISO noise and dynamic range, and at least equals that of the D3, I'll definitely be getting one. It so happens I still have a number of Minolta lenses which are quite good, including the standard Minolta 50mm, the Sigma 24/2.8 (one of the sharpest wide angle lenses that Sigma have made) and the Tamron SP 90/2.8 (one of the best lenses that Tamron have produced).

However, what worries me here is the possibility that DR might appear to be better than it really is as a result of detail-destroying noise reduction.

Here's a link to a number of 100% jpeg crops comparing the A700 with the 40D with the 5D, at various ISOs. The 5D is better in my opinion and the A700 really no better than the 40D, in these shots at least.

However, I know this is not a test of dynamic range. It might well be the case, if one were to examine the deepest shadows in these comparison crops, the A700 might reveal more detail. I don't know. Something tells me if a camera could reveal more detail in the deepest shadows in these circumstances, it could also reveal at least equal detail in the higher tones, if it has the same pixel count, which it does.

[a href=\"http://www.cameralabs.com/reviews/Sony_Alpha_DSLR_A700/noise.shtml]http://www.cameralabs.com/reviews/Sony_Alp...700/noise.shtml[/url]
« Last Edit: December 12, 2007, 01:27:24 am by Ray »
Logged

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #9 on: December 12, 2007, 05:14:45 am »

Quote
Here's a link to a number of 100% jpeg crops comparing the A700 with the 40D with the 5D, at various ISOs. The 5D is better in my opinion and the A700 really no better than the 40D, in these shots at least.

However, I know this is not a test of dynamic range. It might well be the case, if one were to examine the deepest shadows in these comparison crops, the A700 might reveal more detail. I don't know. Something tells me if a camera could reveal more detail in the deepest shadows in these circumstances, it could also reveal at least equal detail in the higher tones, if it has the same pixel count, which it does.


Yes Ray, but as you state in the end, think that one thing is DR (i.e. maximum dynamic range in optimum conditions: lowest electronic ISO + perfect ETTR), which is limited by noise in the shadows, and a different thing is general noise behaviour at high ISOs.

The 5D could well be still the best camera at high ISOs when compared to the Sony A700 (or to any other camera), i.e. acceptably low noise in low dynamic range scenes when ISO value set is very high, and could instead have lower dynamic range at the lowest ISO than the Sony.

They are two different criteria to find out which camera is "more noisy". Unfortunately usually the super-mega-high-ISO criteria is used to determine the noise winner.  But the best camera in terms of noise depends on the application.

For example, if you shoot arquitecture and interiorism, you will use a tripod so there is no reason to rise ISO. In that case your preferred camera regarding noise would be the one with better signal to noise ratio in the shadows, as this will provide a larger dynamic range for that particular application.

I am fond of dynamic range, and I like to study the ways to make our cameras capture the highest possible dynamic range, their Aquiles heel today. I am not that interested in noise at ISO3200.
That's why I like a lot to do these kind of tests, so as study different means to expand camera's DR: Fuji Super CCD and Multiexposure blending for instance.
« Last Edit: December 12, 2007, 05:21:36 am by GLuijk »
Logged

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #10 on: December 12, 2007, 08:15:25 am »

I don't understand your Step 2 ' A bit brighter to distinguish what's going on'. If you will, seemingly arbitrarily, allow more of the highlights to get 'even more blown' so to speak, why not expose the scene lighter in the first place (and then tone it down if you wish for fitting it in your zone system).

I would have at least fixed a variable here (e.g. the highlights) and see how low I could go.

Your methodology, If I understand it correctly, will also 'punish' a camera like the Fuji's which extract their increased dynamic range not through lowering the noise floor but by providing a 'shoulder' to their highlight response.

I guess what I'm trying to say is I question your methodology for selecting a suitable exposure in the first place. It seems quite arbitrary to me, but of course I might not have understood well what you're doing.
« Last Edit: December 12, 2007, 08:16:24 am by NikosR »
Logged
Nikos

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #11 on: December 12, 2007, 08:52:51 am »

Quote
I don't understand your Step 2 ' A bit brighter to distinguish what's going on'. If you will, seemingly arbitrarily, allow more of the highlights to get 'even more blown' so to speak, why not expose the scene lighter in the first place (and then tone it down if you wish for fitting it in your zone system).

I guess what I'm trying to say is I question your methodology for selecting a suitable exposure in the first place. It seems quite arbitrary to me, but of course I might not have understood well what you're doing.

Well 'A bit brighter to distinguish what's going on' is just to show you all the things we could find in the scene (to be able to distinguish them I mean, since the original image was too dark inside due to the high contrast), but that processed image was never used in the test for measuring the levels falling into each f-stop.
Instead I just ran along the f-stops of the linear RAW after it was developed, checking later how much noise I found in each f-stop.

The exposure chosen is unimportant; with a different exposure value simply all areas of image would have fallen into different f-stops, but signal to noise ratio would have stayed the same on each f-stop, which is what we want to measure (signal to noise ratio = perception of noise that allows us or to distinguis texture detail or not).

The only tricky step is to decide whether how to apply the white balance:
- Clipping multipliers
- Non-clipping multipliers with magenta highlights
- Non-clipping multipliers with neutral highlithts

No one satisfies me entirely but some criteria had to be taken. I finally chose the last option and corrected exposure up so that the image was finally exposed to the right (histogram filled till the right side), since it's the option that agrees more with a real photographic concept and workflow.

A real calculation of DR would take into account objetive measures of signal to noise ratio for each individual RGB channel; but the result would not be so easy to be interpreted in a photographic environment.
« Last Edit: December 12, 2007, 08:59:44 am by GLuijk »
Logged

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #12 on: December 12, 2007, 12:59:42 pm »

Quote
Well 'A bit brighter to distinguish what's going on' is just to show you all the things we could find in the scene (to be able to distinguish them I mean, since the original image was too dark inside due to the high contrast), but that processed image was never used in the test for measuring the levels falling into each f-stop.

Fair enough. Three questions

1. In the original picture (both this and the D3 one) how do you establish highlight EV values. I can't see how one can do that without first measuring the original scene.

2. Repeat of a previous question: How do you account for the camera exhibiting a shoulder in its highlight response (e.g. Fujifilm) which will provide it with an expanded DR with the same noise floor vs. a camera with a linear response. In other words, how do you account for potential highlight compression?

3. Noise floor: How do you establish the noise floor without taking into account potential difference in details shown (e.g. due to  hypothetical in camera NR reduction before 'raw' output) when comparing two cameras.
Logged
Nikos

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #13 on: December 12, 2007, 02:04:25 pm »

Guillermo,
Is there anything wrong with the following approach? I stick a target on a white wall; test chart, newspaper, whatever. I then take a number of shots, halving the exposure each time.

I start off with say 1 sec at f8 and ISO 100 (or base ISO) and which is a full exposure to the right. After halving the exposure 10 times, I'm down to a 1,000th sec at f8. If I can still see some recognisable detail in the shot at a 1,000th sec (after processing in ACR) but not at 1/2,000th, then I can claim my camera has 10 stops of DR. Would this be correct?
« Last Edit: December 12, 2007, 02:06:29 pm by Ray »
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #14 on: December 12, 2007, 02:58:17 pm »

Quote
how do you account for potential highlight compression?

1. What is "potential highlight compression"? The highlights are either compressed in given raw file or are not. (Some cameras offer a selection between linear and highlight-compressed data.)

2. The highlight compression does not affect the dynamic range.
Logged
Gabor

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #15 on: December 12, 2007, 03:04:53 pm »

Quote
2. The highlight compression does not affect the dynamic range.
[a href=\"index.php?act=findpost&pid=160168\"][{POST_SNAPBACK}][/a]

Panopeeper,
Surely highlight compression increases the dynamic range of the camera.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #16 on: December 12, 2007, 03:24:31 pm »

Quote
The only tricky step is to decide whether how to apply the white balance:
- Clipping multipliers
- Non-clipping multipliers with magenta highlights
- Non-clipping multipliers with neutral highlithts

No one satisfies me entirely but some criteria had to be taken

This is a weak point of your way of measuring the DR, as it is a weak point of all customary measurements: the result depends on the light source.

IMO the pixels need to be viewed on their own, without demosaicing and without white balancing, so that the result be independent of the light source.

For example I see the shot of a Stouffer wedge, which is a standard tool for measuring the DR. The wedge is originally color neutral, but it looks greenish without white balancing on that given shot, due to the composition of the light and to the spectral responses of the filters.

When looking at it closer, it turns out, that the greens are clipping at least 1/2 stop before the blues, and even more ahead of the reds.

At the other end, looking at the steps, the blue and red pixels are making the result noisy on steps, where the green pixels are "clean".

Consequently, the DR would be different probably by more than one stop in other lighting, or with a magenta filter.
Logged
Gabor

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #17 on: December 12, 2007, 03:26:40 pm »

Quote
Surely highlight compression increases the dynamic range of the camera.

Do you know of any specific example for that? I know of specific examles, which do not increase the dynamic range, and never heard of any others.
Logged
Gabor

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #18 on: December 12, 2007, 03:43:43 pm »

Quote
1. In the original picture (both this and the D3 one) how do you establish highlight EV values. I can't see how one can do that without first measuring the original scene.

Actually I don't care about the highlights, I am not going to measure the DR of the scene but of the camera, and for that I don't need to  look into the highlights but into the shadows.


Quote
2. Repeat of a previous question: How do you account for the camera exhibiting a shoulder in its highlight response (e.g. Fujifilm) which will provide it with an expanded DR with the same noise floor vs. a camera with a linear response. In other words, how do you account for potential highlight compression?

I don't too much understand what you mean bu 'shoulder' here. I have studied in deep Super CCD RAW files; I have found out they are made of two separate RAW files that are 3.6EV apart. After correcting down the exposure of the S sensor image, they are blent and the final image can be measured in the same way as any other developed file. Super CCD IMO is a simple and fantastic technology to achieve a dynamic range that makes any other camera look ridiculous; but it "cheats", it is actually using multiexposure.

I don't think the Super CCD performs any highlight compression in its RAW files, they are both linear. Maybe Fuji's JPEGs are another story, but I am not interested in JPEGs.


Quote
3. Noise floor: How do you establish the noise floor without taking into account potential difference in details shown (e.g. due to  hypothetical in camera NR reduction before 'raw' output) when comparing two cameras.

If a camera performs NR in its RAW files, obviously I will not be able to see how the RAW was before that NR. I just develop them making sure I don't apply any additional NR, and compare texture detail.

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Sony A700 Dynamic Range test: 9.5 f-stops
« Reply #19 on: December 12, 2007, 03:46:30 pm »

Quote
This is a weak point of your way of measuring the DR, as it is a weak point of all customary measurements: the result depends on the light source.

IMO the pixels need to be viewed on their own, without demosaicing and without white balancing, so that the result be independent of the light source.

For example I see the shot of a Stouffer wedge, which is a standard tool for measuring the DR. The wedge is originally color neutral, but it looks greenish without white balancing on that given shot, due to the composition of the light and to the spectral responses of the filters.

When looking at it closer, it turns out, that the greens are clipping at least 1/2 stop before the blues, and even more ahead of the reds.

At the other end, looking at the steps, the blue and red pixels are making the result noisy on steps, where the green pixels are "clean".

Consequently, the DR would be different probably by more than one stop in other lighting, or with a magenta filter.
[a href=\"index.php?act=findpost&pid=160175\"][{POST_SNAPBACK}][/a]

I have to agree with you in all you said, although at the same time I think this kind of subjective tests are useful as they add something a more objective DR measure lacks: practical results for the photographer.

It's too bad that we cannot have the same scene, with the same exposure in the highlights, for all those camera models. That would be really great to make more objective comparisions.

BTW I am planning to write a program to do exactly what you claim: it would be fed with a pile of RAW shots over a uniform card at different exposures. It would then open them without demosaicing (native 12-bit or 14-bit or wathever range) and would calculate for each linear f-stop and each RGB channel the Signal to noise ratio.
But I have a problem here: I have checked that even shooting with my 300mm, over a uniformly lighted white surface, and setting focus to the closest distance so that the image gets smoothly blurred (just to make any deviation over the mean level be due just to noise), I get some micro-vignetting, so I have problems to calculate what SHOULD be the true signal value with the entire image MEAN.

What would you do to properly calculate the signal value in the SNR equation?
I have thought of 2 options:
1. Restrict the calculation to a small patch centered in the image (let's say 100x100 pixels).
2. For every pixel (x,y) location, calculate the mean value of the signal in a surrounding circle of certain radius, and consider that the true signal value for that particular pixel.


What do you think? Panopeeper and anyone else of course.
« Last Edit: December 12, 2007, 04:04:57 pm by GLuijk »
Logged
Pages: [1] 2 3 4   Go Up