Pages: [1] 2   Go Down

Author Topic: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5  (Read 37577 times)

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« on: November 30, 2010, 03:29:42 pm »

I did my own noise measurements over uniform patches to calculate the Signal to Noise Ratio (SNR) for several cameras: Canon 5D, 5D2 and 7D, and Pentax K5.

I used the lowest real ISO (ISO100 in Canon and ISO80 for the Pentax), and ISO1600, which is the highest useful ISO value for any RAW shooter on any camera (pushing ISO further means no improvement in SNR).

These plots were obtained:



Since those cameras have different pixelcounts, a fair comparision requires to normalise all SNR values for the same output resolution. I used the Canon 5D as a reference, so the other cameras (with higher pixelcount than the 5D), obtain improved SNR plots:



Looking at this noise response, the following conclusions are clear:

* At ISO1600 the K5 performs like any other modern APS-C in the whole range
* At ISO80, the SNR curve of the K5 is excellent, yielding a very low presence of noise for low exposure values

This means the K5 at ISO80 is a very high dynamic range camera, capable of recording detail in the deep shadows where other cameras fail. Another advantage of this behaviour is the ability to 'recover' detail when the capture was erroneously underexposed.

Calculating the Dynamic Range over the last set of plots, using a 12dB SNR threshold criteria, the following comparable DR figures are obtained:



At ISO80, the K5 has almost 2 stops more DR than the Canon 5D Mark II FF camera. When new FF sensors appear with such a performance, we can start to think about forgetting the need to shoot several times in HDR scenes.

Using FFT in order to try to detect noise reduction applied to RAW data, doesn't provide any evidence of clandestine process:

ISO1600:



Complete article (Spanish): http://www.guillermoluijk.com/tutorial/noisedr/


Regards

« Last Edit: November 30, 2010, 08:08:24 pm by Guillermo Luijk »
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #1 on: November 30, 2010, 06:36:25 pm »


Calculating the Dynamic Range over the last set of plots, using a 12dB SNR threshold criteria, the following comparable DR figures are obtained:

When new FF sensors appear with such a performance, we can start to think about forgetting the need to shoot several times in HDR scenes.

Guillermo,

A very well done and informative analysis. From what I read, the K5 and Nikon D7000 are a new generation of cropped frame sensor cameras whose electronics can keep up with the DR of the sensor. Read noise is not increased at low ISO, and the downstream electronics are low noise. Indeed, with these cameras there is no need to use an ISO over the base: one can simply increase exposure in the raw converter and not worry about ETTR. Of course, one should still give as much exposure as f/stop and shutter speed considerations permit. Your noise floor of 12dB is more realistic for an estimate of photographic DR than the DXO floor of 0 dB. Your data seem consistent with the DXO data. One could determine the DR at a SNR of 12 dB from the full SNR data as Emil Martinec suggests, but the log scales are difficult to interpolate.

The SNR and DR graphs by DXO of the Nikon D3x show that it also has electronics equal to the sensor, and I think we do not need to wait (if we can afford that camera). The increased sensor size will give a better signal to noise ratio in the midtones and highlights. Doubling the sensor size would increase SNR by a factor of 1.4 (square root of 2). The medium format sensors double the area of the full frame 35 mm cameras, but the inflated claims for DR of these cameras has no scientific basis.

Regards,

Bill

Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #2 on: November 30, 2010, 06:54:03 pm »

Thanks for these results.

They are totally aligned with DxO and might, hopefully, open the door of a new era where LL finally accepts the reality: some manufacturers have really listened to us and have delivered these high DR cameras we were requesting... 2 years ago.

All we need to do is accept facts and pick our options. :)

Cheers,
Bernard

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #3 on: November 30, 2010, 07:34:41 pm »

Indeed, with these cameras there is no need to use an ISO over the base: one can simply increase exposure in the raw converter and not worry about ETTR. Of course, one should still give as much exposure as f/stop and shutter speed considerations permit.

Looking at the SNR plots that is correct Bill. Unlike in Canon cameras, which specially benefit from pushing ISO to obtain a good exposure, in the K5 going from the ISO80 curve (nearly ISO100) to the ISO1600 curve means nearly no SNR improvement, so it becomes worthless caring about pushing ISO to face underexposure (in fact, doing so could blow highlight information without any benefit).

A 6-stops underexposed Pentax K5 DNG file was posted in the Spanish forums; the RAW histogram shows this:



The embedded JPEG is obviously almost black (left). Processing this file provides a decent image (right):


Shadows noise in 100% crops is clearly visible, but still an outstanding result for such underexposure in a APS-C camera:




Your noise floor of 12dB is more realistic for an estimate of photographic DR than the DXO floor of 0 dB. Your data seem consistent with the DXO data. One could determine the DR at a SNR of 12 dB from the full SNR data as Emil Martinec suggests, but the log scales are difficult to interpolate.

Looking at DxO's K5 SNR plots, 12dB are reached at aprox. -3,3 in the log10 scale along the X axis used by these guys. Doing log2(10^(-3,3)) yields -11,0, i.e. the DR with a 12dB criteria would be 11,0 stops according to DxO.

My plots after normalising provide 11,2 stops, but we must compare the DxO figure with the DR calculated on a per-pixel basis, i.e. before normalising to the Canon 5D. And this was 10,8 stops. So both measurements match quite well.


Regards
« Last Edit: November 30, 2010, 08:10:50 pm by Guillermo Luijk »
Logged

uaiomex

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1211
    • http://www.eduardocervantes.com
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #4 on: November 30, 2010, 07:47:27 pm »

So...... should..... we... expect 13 stops DR for next generation FF?   :) ;) :D ;D
Eduardo
Logged

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #5 on: November 30, 2010, 07:53:58 pm »

So...... should..... we... expect 13 stops DR for next generation FF?   :) ;) :D ;D

Using the 12dB criteria, and DR values normalised to Canon 5D (i.e. the same criteria I used above for the 11,2EV figure on the Pentax), I have estimated that a FF camera with a sensor with the same technology and density as the K5 would be a 38Mpx camera with 12 stops of effective DR.

Just to put this figure into the real world, this scene's DR was exactly 12 stops:



Regards
« Last Edit: November 30, 2010, 08:00:46 pm by Guillermo Luijk »
Logged

clkirksey

  • Newbie
  • *
  • Offline Offline
  • Posts: 19
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #6 on: November 30, 2010, 10:41:39 pm »

Interesting. But you apparently did not question your results very carefully. Can you explain why for SNR does not fall at least 3dB per stop for the Pentax as one would expect based on nothing but photon noise?? This is a theoritical value and should show up if there hasn't been any manipulation of the data. Notice that the other bodies tend this way.
Logged
A bird in the viewfinder...

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #7 on: December 01, 2010, 12:51:24 am »

Thanks for these results.

They are totally aligned with DxO and might, hopefully, open the door of a new era where LL finally accepts the reality: some manufacturers have really listened to us and have delivered these high DR cameras we were requesting... 2 years ago.

All we need to do is accept facts and pick our options. :)


I used the full DXO SNR data to calculate the DR of the Nikon D3x, Nikon D7000, and the Phase 1 P65+ according to the method outlined by Emil Martinec.

I downloaded the graphs and interpolated the log scale according to this method. See the section logarithmic interpolation. I measured the graph distances using the ruler in Photoshop.

The DR for a noise floor at a SNR of 12 dB was used for the calculation, and the measurements were entered into an Excel spreadsheet. The results are per pixel (screen in DXO terminology) and the normalization for the megapixel count as per the DXO method is also shown. As a check, the DRs at a noise floor of 0 dB as reported by DXO are also shown. The results are in good agreement. The calculated results are negative as per Emil's method. At a higher noise floor, the P65+ would do better since it collects more photons with its large sensor. At a floor of 12 dB, it is handicapped by a high read noise. What floor would the LL crowd you mention recommend, or will they simply ignore the data?

An example graph is also shown.

Regards,

Bill

« Last Edit: December 01, 2010, 01:03:43 am by bjanes »
Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #8 on: December 01, 2010, 01:06:58 am »

Good point!

I have not the slightest idea...

Erik


Interesting. But you apparently did not question your results very carefully. Can you explain why for SNR does not fall at least 3dB per stop for the Pentax as one would expect based on nothing but photon noise?? This is a theoritical value and should show up if there hasn't been any manipulation of the data. Notice that the other bodies tend this way.
Logged
Erik Kaffehr
 

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #9 on: December 01, 2010, 04:43:19 am »

Interesting. But you apparently did not question your results very carefully. Can you explain why for SNR does not fall at least 3dB per stop for the Pentax as one would expect based on nothing but photon noise?? This is a theoritical value and should show up if there hasn't been any manipulation of the data. Notice that the other bodies tend this way.

Yes that is true, for exposure values close to saturation the samples seem to generate a slope even below the expected theoretical 3dB/EV. Surely it is not because any manipulation since it happens to me on all cameras I tested, so it must be something about the calculation of stdDev of noise.
In low exposure areas (the really interesting ones for DR calculation) the curves seem much closer to the expected theoretical model.

I asked Emil Martinec about possible explanations for this effect but reached no conclusion; will email him again.



Regards

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #10 on: December 01, 2010, 10:04:57 am »

Yes that is true, for exposure values close to saturation the samples seem to generate a slope even below the expected theoretical 3dB/EV. Surely it is not because any manipulation since it happens to me on all cameras I tested, so it must be something about the calculation of stdDev of noise.
In low exposure areas (the really interesting ones for DR calculation) the curves seem much closer to the expected theoretical model.

I asked Emil Martinec about possible explanations for this effect but reached no conclusion; will email him again.


I too await Emil's explanation, but have my own idea.

One reason is that when you measure the noise in a patch, the result is the total noise, and for relatively short exposures of a second or less, the main components are read noise, shot noise and PRNU (pixel response nonuniformity). Read noise is constant for a given ISO and is negligible for relatively high exposure values. However, shot noise varies as the square root of exposure, while PRNU rises in direct proportion to the exposure, and therefore is most prominent as one approaches saturation of the sensor.

See Figure 7 of Emil's treatise on noise.

PRNU is not random, but is a type of pattern noise, and can be removed by subtracting two frames shot with the same exposure. One usually adds an offset to one exposure to prevent negative numbers and subtracts the two images. The standard deviation of the result is for two images, so one divides by the square root of 2 to obtain the SD of one image (See Roger Clark). The results are different for the red, blue and green channels of the Bayer array, so one usually separates the channels and results are most often reported for the green channel.

Regards,

Bill

Logged

madmanchan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2115
    • Web
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #11 on: December 01, 2010, 11:12:45 am »

When you are close to saturation, you will measure less variance because noise will lead to clipping.

e.g., consider image values in a nominal range of [0,1], where 1 represents the maximum representable value. Suppose the actual scene radiance is 0.99, and the noise fluctuation is 0.05. 0.99 + 0.05 is 1.04, but this will be clipped to 1.0. In other words, the recorded signal near 1 is no longer linear (due to the non-linearities introduced by clipping).
Logged
Eric Chan

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #12 on: December 01, 2010, 12:44:19 pm »

When you are close to saturation, you will measure less variance because noise will lead to clipping.

e.g., consider image values in a nominal range of [0,1], where 1 represents the maximum representable value. Suppose the actual scene radiance is 0.99, and the noise fluctuation is 0.05. 0.99 + 0.05 is 1.04, but this will be clipped to 1.0. In other words, the recorded signal near 1 is no longer linear (due to the non-linearities introduced by clipping).

Eric,

You raise a valid point, but the effect we are talking about occurs below the point where the sensor starts to clip. As you point out, as soon as the right portion of the histogram begins to clip, the standard deviation will fall due to clipping of the noise.

This is shown on my Nikon D3 using Iris and looking at the green 1 channel. The threshold for the display is set to maximize the pattern of noise. A pattern arises at clipping on this camera, presumably because of the parallel readouts. Here we have two peaks, but sometimes 4 peaks are evident.

Regards,

Bill

Logged

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #13 on: December 02, 2010, 01:10:00 am »

I don't know about Guillermo's data, but DxO data for both the K-5 and D7000 show nice linearity with a slope of about 3dB/EV over a wide range.  These cameras have such a linear SNR curve that they are approaching the ideal of the ISO-less raw capture, where ISO can be relegated to metadata, as white balance is currently. 
Logged
emil

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #14 on: December 02, 2010, 09:40:34 am »

When you are close to saturation, you will measure less variance because noise will lead to clipping.

e.g., consider image values in a nominal range of [0,1], where 1 represents the maximum representable value. Suppose the actual scene radiance is 0.99, and the noise fluctuation is 0.05. 0.99 + 0.05 is 1.04, but this will be clipped to 1.0. In other words, the recorded signal near 1 is no longer linear (due to the non-linearities introduced by clipping).

SNR at base ISO on these cameras is on the order of 100 or so, clipping the histogram will only be significant in the last few percent of the range, which is a tiny fraction of a stop of EV.  Any visible flattening of the curve below 3dB/EV is likely caused by column gain variations, that lead to PRNU as Bill was mentioning.
Logged
emil

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #15 on: December 03, 2010, 03:47:36 pm »

Any visible flattening of the curve below 3dB/EV is likely caused by column gain variations, that lead to PRNU as Bill was mentioning.

Thanks Emil. Do you agree that in the areas where DR is calculated (typ. for a 12dB threshold criteria that means 8-11 stops from saturation), PRNU noise is negligible compared to photon+read noise? that would mean my DR calculations could be considered accurate.

Regards

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #16 on: December 03, 2010, 04:26:10 pm »

Thanks Emil. Do you agree that in the areas where DR is calculated (typ. for a 12dB threshold criteria that means 8-11 stops from saturation), PRNU noise is negligible compared to photon+read noise? that would mean my DR calculations could be considered accurate.

Regards


Yes, typically PRNU is only relevant in the top stop or so of the range; it will not affect the lower end of the range where you are trying to decide the threshold.  The DR is the interval along the horizontal axis over which the height of the graph is above some threshold, not the height of the graph at the right edge which is far above the threshold.
Logged
emil

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
capture with no need for analog gain adjustment now possible?
« Reply #17 on: December 03, 2010, 05:25:09 pm »

... both the K-5 and D7000 ... have such a linear SNR curve that they are approaching the ideal of the ISO-less raw capture, where ISO can be relegated to metadata ...
The idea that SLR A/D conversion has reached the level of recording all the useful signal range that photosites provide (simultaneously handling full well signals and giving quantization noise sufficiently below the noise floor of the analog signal from the photosites) is exciting: photographers can be freed of a lot of technical worries in setting up shots.

I have lost track of who really makes or is alleged to make the sensors for various Nikon and Pentax SLRs, so is it known if these two cameras are using the Sony Exmor approach of on-chip, column-parallel A/D conversion? That used to be 12-bit at best, but Sony seems to be improving rapidly. (And I tend to believe that the core technological capability differences between major CMOS sensor makers is rarely more than a year or two, so all mainstream "system cameras" could be there by the time I need to buy a new camera!)
Logged

falconeye

  • Newbie
  • *
  • Offline Offline
  • Posts: 2
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #18 on: December 03, 2010, 08:18:07 pm »

My plots after normalising provide 11,2 stops, but we must compare the DxO figure with the DR calculated on a per-pixel basis, i.e. before normalising to the Canon 5D. And this was 10,8 stops. So both measurements match quite well.

This is my first post at LL, so hello everybody :)

Guillermo's work is awesome and I like it a lot.

The above may not be 100% correct though. The Full SNR curves at DxO are pixel level curves (aka "screen") and have not been normalized to 8MP. So if I'm not mistaken, the DxO figure of 11.0 stops would compare to Guillermo's figure of 11.2 stops which still is a nice match.

The flatter slope above 30dB is normally caused by fixed pattern noise which is very hard to get rid off. I don't know if DxO uses differencing. Also, I don't know if DxO's ISO definition implies clipping at white mean or mean + x*std.dev. for some x. However, the difference is only about 1 8Bit-RGB-step per x.
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Re: Noise and DR comparision: Canon 5D vs 5D2 vs 7D vs Pentax K5
« Reply #19 on: December 03, 2010, 10:47:32 pm »

The bottom line here, for those who might get a bit confused by graphs, is that the significantly higher DR shown in the DXO graphs for the Pentax K-5 (and by inference the Nikon D7000), seems to be confirmed.

These results are not just theoretical, mathematical constructs which may be irrelevant in practical terms as a result of photonic shot noise, which we can do little to combat (a bit like the effects of diffraction in lenses), but real performance improvements delivering real increases in DR which can be observed on monitor and on print.

I have confirmed myself that in compressed jpeg format even, the superior DR of the D7000, compared with the Canon 50D, is clearly evident. Refer to my thread on this issue at http://www.luminous-landscape.com/forum/index.php?topic=48209.msg405802#msg405802
Logged
Pages: [1] 2   Go Up