Luminous Landscape Forum
Equipment & Techniques => Cameras, Lenses and Shooting gear => Topic started by: Panopeeper on September 28, 2009, 08:00:37 pm
-
The following is intended as explanation for the evaluation of digital cameras' noise and DR characteristics based on raw images. It is an alternative way to DXO's related evaluations in some aspects, but the principles are the same.
I am performing the measurements on single raw channels. The upside of this method, compared to the measurement on demosaiced images, is, that this does not depend on the color of objects and on the actual illumination; the downside is, that the resulting values are not directly applicable to any actual setting; they are to be seen relatively to each other, i.e. in comparison of cameras and ISO settings.
Some preliminary remarks:
- It is a wide-spread misconception, that different raw channels have different noise characteristics. This is not so; normally all channels behave the very same way regarding noise and dynamic range; the exceptions are, when the camera's hardware is faulty, or when the firmware cooks the raw data too much, like the Sony A850 and A900.
A demonstration of this fact is in the chapter Noise and raw channels of The Source of Noise (http://www.cryptobola.com/PhotoBola/SourceOfNoise.htm), close to the beginning.
- One can often read about "headroom". In conjunction with raw data, the "headroom" is a myth; there is no such thing. The headroom is an entity of the metering, not of the raw image data itself.
- The concepts dynamic range and noise can not be separated. The dynamic range is practically limited by the noise in the deep shadows; it could be limited by the bit depth as well, but that is not the case with any modern DSLR or MFDB I know of.
- The noise is usually measured as the standard deviation of the pixel values on a smooth, uniform and uniformly illuminated areas. This is an objective measurement; however, the actual appearance of the noise may make it worse than the standard deviation indicates; think of the 5D2's pattern noise ("banding") and the blotches of the Sony A900,
- DXO presents two sets of measurements for the DR/noise complex:
- the signal-noise ratio measured on an 18% intensity patch. This is IMO totally useless, because 18% intensity corresponds to 2.5 EV from clipping, i.e. in the middle of the third stop of the dynamic range. The noise of all modern DSLRs is in the range of 1-2% with the best ISO and 4-6% with ISO 1600; this is so low, that it does not pay to talk about it at all,
- the dynamic range at SNR=1. This is just the opposite; SNR=1 means 100% noise (this does not mean, that everything is noise, rather, it means that the standard deviation is as high as the average pixel intensity). This noise is very high, I don't see any point in taking it as the threshold for the dynamic range measurement.
Let's see some samples for different noise levels; the level of noise in percentage of the average pixel intensity is highlighted with the pixel intensity as well; the latter is expressed in EV from clipping, for example -9.06 means 9.06 EV from clipping, i.e. at the start of the tenth stop of the dynamic range, the noise is 53.1%.
2.5% with ISO 100 (http://www.panopeeper.com/Noise/DR_Noise_Level2.5_ISO00100_Green_Canon5DMkII.GIF)
5% with ISO 100 (http://www.panopeeper.com/Noise/DR_Noise_Level5_ISO00100_Red_Canon5DMkII.GIF)
10% with ISO 100 (http://www.panopeeper.com/Noise/DR_Noise_Level10_ISO00100_Red_Canon5DMkII.GIF)
20% with ISO 100 (http://www.panopeeper.com/Noise/DR_Noise_Level20_ISO00100_Red_Canon5DMkII.GIF)
20% with ISO 1600 (http://www.panopeeper.com/Noise/DR_Noise_Level20_ISO01600_Red_Canon5DMkII.GIF)
25% with ISO 1600 (http://www.panopeeper.com/Noise/DR_Noise_Level25_ISO01600_Red_Canon5DMkII.GIF)
50% with ISO 1600 (http://www.panopeeper.com/Noise/DR_Noise_Level50_ISO01600_Blue_Canon5DMkII.GIF)
50% with ISO 1600 (http://www.panopeeper.com/Noise/DR_Noise_Level50_ISO01600_Red_Canon5DMkII.GIF)
100% with ISO 1600 (SNR=1) (http://www.panopeeper.com/Noise/DR_Noise_Level100_ISO01600_Blue_Canon5DMkII.GIF)
Now, slowly I come to the point: how do I measure the dynamic range?
I create or receive raw images of some suitable scenery (template, color checker sheet) with suitable exposure (i.e. sufficiently underexposed) and measure the noise on patches with different intensities. The template can be downloaded from here (http://www.panopeeper.com/Download/DR_Noise_Template_3600x2700.tif) and printed. Our fellow poster, Marc MacCalmont did that and sent me the raw files with all full-stop ISOs of the 5DMkII; I did the same with my Canon 40D. The ISO 1600 raw files can be downloaded from here, 5DMkII (http://www.panopeeper.com/...ownload/Canon5DMkII_NoiseDRTemplate_ISO01600.CR2) and from here, 40D (http://www.panopeeper.com/.../Download/Canon40D_NoiseDRTemplate_ISO01600.CR2)
I used only the ISO 100, 400 and 1600 shots and added for good measure the 3200 shot with the 5D2. ISO 3200 of the 5D2 is not better than ISO 1600, but the 3200 shot is 1 EV lower exposed than the 1600 shot (which is matching the 40D shot); this shows, that even the 1 EV lower exposed ISO 1600 of the 5D2 is far superior to the 40D.
I converted the raw images with DPP, all settings neutral, except RGB brightness +50, Raw brightness +2.00 for the 40D shots and +1.7 to +1.9 for the 5D2 shots to achieve almost identical luminosity on the brightest spot. NR, sharpening, lens correction were turned off.
As the captured intensities are pretty identical , the results are directly comparable.
I selected three patches from each image: the brightest one for matching, and the leftmost and rightmost patches from the bottommost row; the last is the darkest patch in the shot. I included the 100% crops from the 5D2 shots and the 66.66% crops (resizing: bicubic sharper) as well as the 100% crops from the 40D shots; these are almost identical in size.
This layered TIFF file (http://www.panopeeper.com/Download/Canon5DMkII_vs_40D_NoiseDR.tif) contains thirty-some layers (20 MB); the names of the layer groups and layers are self-explanatory. The number triplets in brackets indicate the average pixel intensity in the raw channels, (red, green, blue) from clipping, in EV. For example (-10.9,-10.3,-10.7) in the 5D2 ISO 1600 crop indicates, that the red channel in the grey area is almost 11 stops under clipping, i.e. it is at the end of the 12th stop of the DR.
TL means "top left", "BL" is "bottom left", and, of course, "BR" stays for "bottom right".
The test with the 5D2 could have gone even deeper (i.e. with even lower exposure), but the darkest patches of the 40D shots are pretty much useless already with this exposure.
The results of measurements can be displayed as graphs, the dynamic range depending on the ISO and the acceptable noise level; for example 5DMkII (http://www.panopeeper.com/Noise/Canon5DMkII_Noise.GIF).
A final note: there is no need to have anything in the sample raw images in highlights, not even in the mid range. This seems to be a cause of confusion, it occured several times that readers were astonished to see, that I need only extremely underexposed shots. Well,
1. who cares, if the noise in the top EV is 0.1% or 1%?
2. the presence of highlights in an image is not necessary to measure the shadows.