Sony A900 noise, dynamic range and noise reduction

[ejmartin]

wow! what does make D3X's noise so attractive? I thought digital noise was just a gaussian distribution on any camera sensor, with no spatial structure and which amplitude solely depends on the number of converted photons and the ISO gain setting.

There is also pattern noise, see here for an example from a 20D:

http://theory.uchicago.edu/~ejm/pix/20d/te...e/#patternnoise

these sorts of noises have a visual impact that exceeds their contribution to noise measures like the standard deviation, because even though they may be small in magnitude for any given pixel, they are coherent across many pixels and therefore get picked up by visual processing that is looking for patterns.

The D3x seems to have such noises well under control.  Also the read noise std dev of the D3x at base ISO is quite low compared to other DSLRs at base ISO.

[ziocan]

I am working on it. It is not a simple, straightforward issue; I am demonstrating it through a whole bunch of observations, all heavy duty pixel peeping.

Practically, nothing that has to do with photography.
 

[Iliah]

Practically, nothing that has to do with photography.
 

Of course, if you shoot jpegs.

[ErikKaffehr]

Guillermo,

I can send you the other images if you are interested, I have similar images from 100 to 1600 ISO with different noise reductions.

Best regards
Erik

Gabor, following the use of FFT to find traces of noise reduction made by Emil Martinec here, I calculated the FFT distribution of noise found in the cup of post #19:



There is no evidence, from the FFT perspective, that any noise reduction has been applied to the RAW data in the 'OFF' setting. So would it be logical that NR is applied to low ISOs from 100 to 800 in the 'OFF' setting?

On the other side for any other setting a clear spatial pattern arises in the FFT (in fact is the same pattern for 'LOW', 'MEDIUM' and 'HIGH', just with different amplitudes) that makes think of some noise reduction algorithm based not only in adjacent pixels but also pixels at greater distances.

Perhaps Emil can give his interpretation.

BR

[Iliah]

To see if there is noise reduction, clipping, or other shadow treatment; or simple in-camera flare one needs to perform resolution tests. Shots of a newspaper from a proper distance (so that the text is easily readable if the newspaper is exposed according to the meter), taken with a good prime lens with as few elements as possible (think Tessar) stopped down to f/5.6 - f/8 and viewfinder covered, -2 to -6 EV, at various ISO settings, from a steady tripod, - that is a useful photographic test, especially when done under incandescent, fluorescent, flash, and daylight.

[Guillermo Luijk]

I can send you the other images if you are interested, I have similar images from 100 to 1600 ISO with different noise reductions.

ISO100 with NR set to OFF and HIGH would be the most interesting. However let's wait for Gabor's results first.

 

[douglasf13]

ISO100 with NR set to OFF and HIGH would be the most interesting. However let's wait for Gabor's results first.

I believe that Gabor is saying that NR is always on at low ISO, and it doesn't change, regardless of the high ISO NR setting. Granted, he's got a lot of convincing to do, since he doesn't really have proper test shots like was mentioned above.

[ErikKaffehr]

Hi,

I'd suggest that Gabor is having almost all test shots he needs. By the way,  I can see the patterns Gabor found in my 100 ISO shots using the  DxO converter. Gabor has invested a lot of work on this issue an I have contributed some.

Best regards
Erik

I believe that Gabor is saying that NR is always on at low ISO, and it doesn't change, regardless of the high ISO NR setting. Granted, he's got a lot of convincing to do, since he doesn't really have proper test shots like was mentioned above.

[douglasf13]

I'm not saying that there hasn't been a lot of work done. I appreciate the effort.  The question is whether the work is being done correctly, which I believe is what Iliah implied above.

[ErikKaffehr]

Hi,

I guess that we can see different aspects depending the tests/evaluations we have made. We can see some artifacts in raw-images from the A900, that's pretty certain. Now there are some questions:

- Are these artifacts relevant?
- If so under which conditions?
- What can we do about them?

It is unfortunate if Sony gets some bad reputation because of these discussions. The artifacts we see are not normally visible under low ISO conditions. It is conventional wisdom that the A900 performs worse on high ISO noise than the competition, possibly exaggerated by unadvantageous processing in ACR/LightRoom.

Regarding the noise reduction at low ISO I do have 100-1600 ISO shots like the ones Guillermo analyzed. So I plan to go ahead and check FFTs on all those images, but I don't have image analysis software on my primary workstation (Apple Imac using Tiger), so I need to revert to another platform, or find good software for the Mac. The need is essentially to be able to read unprocessed images in TIFF or similar format and perform FFT on them.

I also have a comment on Michaels Reichmann's findings on the Canon 5DII, A900 and Nikon D3x. What he says that thet are virtually indistinguishable up to 800 ISO but above that the Nikon and the Canon have a one step advantage. MR also very clearly stated that he was not discussing DR (Dynamic Range), because that was beyond his means to evaluate. Michael's findings are essentially consistent with DxO-mark. In my view the simple truth is probably that at ISO 100-200 all the cameras are good enough. There may be a theoretical advantage to the D3x under lab conditions but they may or may not be relevant under real life conditions.  Bernard has shown some pretty impressive images by his Nikon D3x which may demonstrate the need for DR at low ISO, but for my picture taking DR at ISO 200 seldom was a limiting factor.

Best regards
Erik

I'm not saying that there hasn't been a lot of work done. I appreciate the effort.  The question is whether the work is being done correctly, which I believe is what Iliah implied above.

[ErikKaffehr]

Hi,

I guess that we can see different aspects depending the tests/evaluations we have made. We can see some artifacts in raw-images from the A900, that's pretty certain. Now there are some questions:

- Are these artifacts relevant?
- If so under which conditions?
- What can we do about them?

It is unfortunate if Sony gets some bad reputation because of these discussions. The artifacts we see are not normally visible under low ISO conditions. It is conventional wisdom that the A900 performs worse on high ISO noise than the competition, possibly exaggerated by unadvantageous processing in ACR/LightRoom.

Regarding the noise reduction at low ISO I do have 100-1600 ISO shots like the ones Guillermo analyzed. So I plan to go ahead and check FFTs on all those images, but I don't have image analysis software on my primary workstation (Apple Imac using Tiger), so I need to revert to another platform, or find good software for the Mac. The need is essentially to be able to read unprocessed images in TIFF or similar format and perform FFT on them. I can write that kind of software, by the way, but it's better to use something that is well tested and proven.

I also have a comment on Michaels Reichmann's findings on the Canon 5DII, A900 and Nikon D3x. What he says that thet are virtually indistinguishable up to 800 ISO but above that the Nikon and the Canon have a one step advantage. MR also very clearly stated that he was not discussing DR (Dynamic Range), because that was beyond his means to evaluate. Michael's findings are essentially consistent with DxO-mark. In my view the simple truth is probably that at ISO 100-200 all the cameras are good enough. There may be a theoretical advantage to the D3x under lab conditions but they may or may not be relevant under real life conditions.  Bernard has shown some pretty impressive images by his Nikon D3x which may demonstrate the need for DR at low ISO, but for my picture taking DR at ISO 200 seldom was a limiting factor.

Best regards
Erik

[Ray]

I get a sense that discussions like this drift too far from practical, real world photos, which are surely the main purpose of photography.

Dynamic range has always been an issue with photography. The camera with a single shot has never been able to capture the dynamic range that the eye sees, because the camera has a fixed aperture at the time of exposure, whereas the eye's pupil contracts and dilates almost instantly as its gaze changes from shadows to highlights.

Here's a shot taken years ago with my Canon D60. Exposure was very much to the right, requiring a -1.1EV adjustment in ACR as well as a 95 recovery of highlights. I suspect that Gabor, using Rawlanyze could point out that the following shot is in fact overexposed. Yet the shadows are awful, as you can see from the 100% crop of the bottom right corner.

This scene was simply beyond the DR capability of the Canon D60. My D700 would have probably have managed it with aplomb.

[attachment=12188:full_image.jpg]  [attachment=12189:Bottom_r...t_corner.jpg]

[Iliah]

Hi,

I'd suggest that Gabor is having almost all test shots he needs. By the way,  I can see the patterns Gabor found in my 100 ISO shots using the  DxO converter. Gabor has invested a lot of work on this issue an I have contributed some.

Best regards
Erik

No wonder, DxO applies noise reduction during the conversion.

[ErikKaffehr]

Ray,

I see two reasons for testing, one is curiousity the other the need to explore the limitations of a system in order to learn living with it.

Erik

I get a sense that discussions like this drift too far from practical, real world photos, which are surely the main purpose of photography.

Dynamic range has always been an issue with photography. The camera with a single shot has never been able to capture the dynamic range that the eye sees, because the camera has a fixed aperture at the time of exposure, whereas the eye's pupil contracts and dilates almost instantly as its gaze changes from shadows to highlights.

Here's a shot taken years ago with my Canon D60. Exposure was very much to the right, requiring a -1.1EV adjustment in ACR as well as a 95 recovery of highlights. I suspect that Gabor, using Rawlanyze could point out that the following shot is in fact overexposed. Yet the shadows are awful, as you can see from the 100% crop of the bottom right corner.

This scene was simply beyond the DR capability of the Canon D60. My D700 would have probably have managed it with aplomb.

[attachment=12188:full_image.jpg]  [attachment=12189:Bottom_r...t_corner.jpg]

[Iliah]

I plan to go ahead and check FFTs on all those images, but I don't have image analysis software on my primary workstation (Apple Imac using Tiger)

ImageJ works on Mac, you may need Java upgrade. For Tiger you may need to use 32-bit version, worth a try IMHO. http://rsb.info.nih.gov/ij/docs/install/osx.html

To extract raw channels from a raw file you can use command-line utility, 4channels, pre-compiled for Mac in LibRaw http://www.libraw.org/data/LibRaw-0.7.0-MacOSX.zip If you will have problems running it on Tiger please let us know.

[Ray]

Ray,

I see two reasons for testing, one is curiousity the other the need to explore the limitations of a system in order to learn living with it.

Erik

Curiosity is fine. But how are you going to translate the data that Gabor produces to real world scenes in a way that is practically useful?

We're into hair-splitting here, aren't we?

[Iliah]

Curiosity is fine. But how are you going to translate the data that Gabor produces to real world scenes in a way that is practically useful?

We're into hair-splitting here, aren't we?

No, we are not into hair-splitting. Translation of raw data into the images of real world scenes is done with the help of those who develop raw converters. The more we know about raw data the better are the raw converters we code and the better are the converters we use.

[Ray]

No, we are not into hair-splitting. Translation of raw data into the images of real world scenes is done with the help of those who develop raw converters. The more we know about raw data the better are the raw converters we code and the better are the converters we use.

Well that's a different matter. Is this what the thread's really about? Prising information about the structure of the RAW file in order to produce a better RAW converter? I'm all in favour of that, but I find it strange that any company such as Sony would not cooperate with developers of RAW converters who wish to enhance the images from Sony cameras.

[douglasf13]

Well that's a different matter. Is this what the thread's really about? Prising information about the structure of the RAW file in order to produce a better RAW converter? I'm all in favour of that, but I find it strange that any company such as Sony would not cooperate with developers of RAW converters who wish to enhance the images from Sony cameras.

From what I understand, none of the camera companies provide disclosure on their raw "recipe," and that's part of the problem for raw converter developers.

[Iliah]

From what I understand, none of the camera companies provide disclosure on their raw "recipe," and that's part of the problem for raw converter developers.

Actually, "recipe" is a good term. Structure of raw is not very hard to decode. It is parametadata that is missing.