Technical article on noise in digital imaging

[ejmartin]

... for those with a technical bent:

http://theory.uchicago.edu/~ejm/pix/20d/te...oise/index.html

Put on your propeller beanies  

Seriously, though, fans of this site's disquisition on ETTR might be interested in the analysis on page 3 of the article, among other things.

[Tim Gray]

Actually the article seems to be reasonably accessible - not a whole lot of nasty math.  

After a quick read (particularly of the ETTR section) it seems the author is positing that the benefits of ETTR accrue due to the fact that more photons are captured as the exposure is migrated to the right, improving SN as opposed to to the quantization benefit of subsequently re-mapping the midtones of an ETTR image down into the shadows during post processing (assuming the move to the right is accomplished by holding the shutter open longer - or aperture wider - rather than boosting the ISO) - did I get that right?

Having said that, if I remember correctly, it seems to me that Jonathan W.'s sample of a "broken" image reflected damage to tonal gradations rather than noise issues.

And maybe it's just the impact of Canon's marketing, but I think I can see better shadow quality in some circumstances with the 14 bit over the previous 12 bit models.  But I might be fooling myself.

[ejmartin]

Actually the article seems to be reasonably accessible - not a whole lot of nasty math. 

After a quick read (particularly of the ETTR section) it seems the author is positing that the benefits of ETTR accrue due to the fact that more photons are captured as the exposure is migrated to the right, improving SN as opposed to to the quantization benefit of subsequently re-mapping the midtones of an ETTR image down into the shadows during post processing (assuming the move to the right is accomplished by holding the shutter open longer - or aperture wider - rather than boosting the ISO) - did I get that right?

Yes that is an accurate summary.  Actually, there is a slight benefit to ETTR by ISO boost, but it is much milder than doing it by decrease in Tv or Av and almost non-existent at high ISO.

Having said that, if I remember correctly, it seems to me that Jonathan W.'s sample of a "broken" image reflected damage to tonal gradations rather than noise issues.

I'd be interested in the link if you have it...

And maybe it's just the impact of Canon's marketing, but I think I can see better shadow quality in some circumstances with the 14 bit over the previous 12 bit models.  But I might be fooling myself.
[a href=\"index.php?act=findpost&pid=197482\"][{POST_SNAPBACK}][/a]

The higher shadow quality in newer Canons (which I agree is real) has mostly to do with improvements in controlling patterned read noise and a slight lowering of overall read noise.  It has nothing to do with the passage from 12-bit to 14-bit tonal depth.

[Guillermo Luijk]

Actually, there is a slight benefit to ETTR by ISO boost, but it is much milder than doing it by decrease in Tv or Av and almost non-existent at high ISO.

Emil, I agree that improvement when reaching high ISO values is very slight, but just doing ETTR using ISO200 instead of ISO100 can be much more than a slight improvement:


Canon 350D shots setting constant aperture/shutter:

[bjanes]

... for those with a technical bent:

http://theory.uchicago.edu/~ejm/pix/20d/te...oise/index.html

Put on your propeller beanies   

Seriously, though, fans of this site's disquisition on ETTR might be interested in the analysis on page 3 of the article, among other things.
[a href=\"index.php?act=findpost&pid=197353\"][{POST_SNAPBACK}][/a]

This is the best treatise on the subject that I have seen online and anyone with an interest in these technical issues should study it.

The analysis of ETTR is informative and confirms what I have felt for some time: the number of raw levels in the brightest f/stop of a 12 or 14 bit digital capture is not that important (because of noise), but the increased signal to noise ratio with ETTR is significant. While ETTR is important for optimal image quality, it should not be overdone since clipping does result in data loss. Since the S:N varies as the square root of the exposure, an increase of one full f/stop in exposure only improves the S:N by a factor of 1.414.

[Guillermo Luijk]

Emil I freely took your plots figures to make a DR comparision between 5D, 40D and D3. For simplicity I used the criteria SNR>2EV which was easy to estimate over your plots and yields DR values arounf 8-9 f-stops which seem logical in photographic usability:



5D plot crosses 40D's, which means while 5D is better at high ISOs even if it's an older camera, for static scenes where there is no problem to ETTR at the lowest electronic ISO 40D is around 1/3 f-stops better than 5D. For this kind of scenes D3 is 2/3 f-stops better than 5D.

Considering ISO values are correctly normalised on the three cameras, at the same ISO, D3 has about 1/2 f-stop additional DR over 40D which makes it a less noisy camera for action shooting. For ISO below 800 D3 is also better than 5D, but at ISO800 and ISO1600 5D and D3 behave the same.

Considering again static scenes, D3 is also better than 40D but for less than 1/3 f-stop.

______________________________


I have a question however: your SNR measures refer to measures of signal vs noise over individual pixels, so the effect that a larger amount of pixels (more resolution) could have in the final noise perception was not entered in the equations, right?

For instance, according to your measures, two cameras with the same SNR per pixel but having camera A twice as many pixels as camera B, would result in the same plot; but in practice camera A will have less visible noise when rescaled to the same final size in pixels due to noise averaging.

Is that right?

[bjanes]

... for those with a technical bent:

http://theory.uchicago.edu/~ejm/pix/20d/te...oise/index.html

Put on your propeller beanies 
[{POST_SNAPBACK}][/a]

Considering the scope and quality of Emil's post, I am quite surprised that there have not been more comments. Perhaps readers need additional time to digest the article. To get discussion started, here are a few observations.

"one might think that read noise is a fixed cost per pixel in recording an image, again assuming that the same quality of circuit elements are employed."

This does appear to be true, but should one measure the cost in terms of noise expressed in data numbers (DN, raw pixel values) or electrons? In [a href=\"http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary/#read_noise]Clark Fig 3[/url] of a post, Roger Clark notes that read noise expressed in electrons is similar in sensors of the same generation but of differing pixel sizes. Read noise varies with ISO, but read noise quoted in manufacturer's spec sheets is typically for unity gain, which complicates the analysis since unity gain is correlated with pixel size.

One can increase signal to noise at the expense of resolution by pixel binning as Emil discusses. The results of binning differ when binning is done in hardware prior to digitization or in software by downsizing. Consider hardware binning in which 4 pixels in a 2 by 2 array are combined and read out together, as described here. The four pixels are combined into a superpixel, which is read out with the same read noise (in electrons) as for a single small pixel. This implies that when other factors are held constant, read noise is independent of pixel size. If one performed binning in Photoshop after the fact, the resulting superpixel would have incurred 4 read noise contributions, not a single one as with hardware binning.

If the read noise is similar for large and small pixels, the large pixel camera still has an advantage when read noise is measured in DNs rather than electrons. This is because of camera gain as Roger explains in this example: Clark Table 3. See the text just following table 3.

"The realization from Table 3 is even though the read noise is similar in terms of electrons, the effect on the image is huge because of the gain factor. Thus shadow detail on a small sensor is severely compromised as the gain factor drops."

Seriously, though, fans of this site's disquisition on ETTR might be interested in the analysis on page 3 of the article, among other things.
[a href=\"index.php?act=findpost&pid=197353\"][{POST_SNAPBACK}][/a]

Any comments by the resident "experts" who have expounded on the "levels theory", where half the levels are in the brightest f/stop? The improved signal to noise ratio with ETTR is a definite benefit, but how does it scale to human perception? Considering only shot noise, the S:N increases by a factor of sqrt(2) with doubling of the exposure. Looking at Emil's S:N graphic demonstration, I perceive only slight image improvement with a doubling of S:N. Is this a log function or what?

Bill

[ejmartin]

Emil, I agree that improvement when reaching high ISO values is very slight, but just doing ETTR using ISO200 instead of ISO100 can be much more than a slight improvement:
Canon 350D shots setting constant aperture/shutter:

[a href=\"index.php?act=findpost&pid=197523\"][{POST_SNAPBACK}][/a]

Well it is a substantial improvement in this example -- but if the ISO 1600 image was properly exposed, the ISO 100 shot here is 4 stops underexposed, a stop or more than that in the red and blue channel if these were shot under indoor lighting, then you are way down into the read noise.  My comment about mild improvement had in mind ISO 200 properly exposed vs ISO 100 one stop underexposed.  But you are right the biggest improvement will come at ISO 100 vs 200 since the read noise in Canons is almost constant between the two.  Going all the way from 100 to 1600 on some cameras is about a factor of 4 or two stops improvement in shadow noise comparing in-camera to post-processing gain-up.

[Guillermo Luijk]

My comment about mild improvement had in mind ISO 200 properly exposed vs ISO 100 one stop underexposed.  But you are right the biggest improvement will come at ISO 100 vs 200 since the read noise in Canons is almost constant between the two.

That's perfect, thank you Emil.

BTW I sent you a message, just to find out if you have some reference url with links to all your articles. I think they are incredibly valuable.

BR

[ejmartin]

I have a question however: your SNR measures refer to measures of signal vs noise over individual pixels, so the effect that a larger amount of pixels (more resolution) could have in the final noise perception was not entered in the equations, right?

Correct.  Data are for individual pixels; comments on how to scale the data to account for varying pixel size are in the section on pixel size on page 3.

For instance, according to your measures, two cameras with the same SNR per pixel but having camera A twice as many pixels as camera B, would result in the same plot; but in practice camera A will have less visible noise when rescaled to the same final size in pixels due to noise averaging.

Is that right?
[a href=\"index.php?act=findpost&pid=197796\"][{POST_SNAPBACK}][/a]

Two comparisons that I think are valuable are noise scaled to the image size, and noise scaled to fixed spatial scale -- that is, referred to a scale in lines/picture height, or to a scale in lines/mm.  Noise scales linearly rather than as an area (as a result of combining as sum of squares rather than a simple sum), and so noise scales with the pixel spacing for instance if referring to fixed scale in lines/mm, or inversely with the number of pixels per picture height if referring to the frame size.

[Guillermo Luijk]

or with the number of pixels per picture height if referring to the frame size.

I think this is the more valuable performance indicator for the photographer.

[Ray]

Considering the scope and quality of Emil's post, I am quite surprised that there have not been more comments. Perhaps readers need additional time to digest the article. To get discussion started, here are a few observations.
[a href=\"index.php?act=findpost&pid=198063\"][{POST_SNAPBACK}][/a]

I think it's an excellent exposition of the subject. Unfortunately, the time I first attempted to download the article, the connection was very,very, very slow. The mouseovers didn't work at all and it just took too long to move from one heading to the next. It's now a bit faster and the mouseover image is working.

What I find particularly interesting are the plots of S/N ratio at various ISO's for the 5D, D3 and 40D.

It's interesting because I happen to own both a 5D and 40D and always got the impression that the 5D provided noticeably better image quality at high ISO. These plots confirm that impression.

Whilst I'm holding off buying a Nikon D3, I did take the trouble to do a few test shots in a store in Bangkok (comparing it with my 5D), but all my test shots were at ISO 3200 and above. I never thought of comparing the D3's noise performance at lower ISO's, and I didn't really have the time if I had thought of it.

I'm surprised that according to these plots the Nikon D3 from ISO 200 to ISO 1600 has the same read noise as the 5D. But I'm puzzled how Guillermo's plots comparing dynamic range can show the D3 being better than the 5D at ISO 100 to the same degree that the 5D is better than the Nikon D3 at ISO 800. That seems really surprising.

[Guillermo Luijk]

I'm surprised that according to these plots the Nikon D3 from ISO 200 to ISO 1600 has the same read noise as the 5D. But I'm puzzled how Guillermo's plots comparing dynamic range can show the D3 being better than the 5D at ISO 100 to the same degree that the 5D is better than the Nikon D3 at ISO 800. That seems really surprising.

Hi Ray, I just plotted DR according to the chosen criteria SNR>2EV on the SNR plots referred close to Emil's Fig. 12. According to the article these plots combine read and shot noise (perhaps that's the reason).

Values are aproximated, just visually inspected:





However 5D is never better than D3 using this SNR criteria for DR.

[ejmartin]

I think it's an excellent exposition of the subject. Unfortunately, the time I first attempted to download the article, the connection was very,very, very slow. The mouseovers didn't work at all and it just took too long to move from one heading to the next. It's now a bit faster and the mouseover image is working.

What I find particularly interesting are the plots of S/N ratio at various ISO's for the 5D, D3 and 40D.

It's interesting because I happen to own both a 5D and 40D and always got the impression that the 5D provided noticeably better image quality at high ISO. These plots confirm that impression.

Whilst I'm holding off buying a Nikon D3, I did take the trouble to do a few test shots in a store in Bangkok (comparing it with my 5D), but all my test shots were at ISO 3200 and above. I never thought of comparing the D3's noise performance at lower ISO's, and I didn't really have the time if I had thought of it.

I'm surprised that according to these plots the Nikon D3 from ISO 200 to ISO 1600 has the same read noise as the 5D. But I'm puzzled how Guillermo's plots comparing dynamic range can show the D3 being better than the 5D at ISO 100 to the same degree that the 5D is better than the Nikon D3 at ISO 800. That seems really surprising.
[a href=\"index.php?act=findpost&pid=198191\"][{POST_SNAPBACK}][/a]

John Sheehy pointed out that the read noise figures for the 5D were out of line with those from 5D's he had measured.  While I trust the source of the data I used, the body may have not been a typical copy.  Some data from Peter Ruevski (whose 1D3 data I used) are more in line with the figures John quoted, so I may update the 5D plots to reflect this.  Current data seem to be about 15-20% low for 5D read noise at high ISO.

This is one shortcoming of current testing by the people who do this sort of thing -- there is considerable sample variation, and so it is difficult to put reliable error bars on the data.


Here is the plot of the data from the other 5D:

[BJL]

... for those with a technical bent:

http://theory.uchicago.edu/~ejm/pix/20d/te...oise/index.html[a href=\"index.php?act=findpost&pid=197353\"][{POST_SNAPBACK}][/a]

Emil,
    as one of those with a technical bent (also a one-time Cargese participant!), thanks very much for your excellent essay. To answer Bill Janes' question, I have not commenting so far because I am happy reading, thinking and learning.

One comment though: I am reassured to see further evidence that 12-bit A/D converters depth are not in practice a limit to current DSLR image quality. For one thing it adds to the promise of the Sony/Nikon approach of massively parallel on-sensor A/D conversion (by removing concerns about its being 12-bit only). That approach could help to eliminate the frame rate disadvantage that  digital SLR's still often have compared to film SLRs, and finally offer cutting edge high resolution and high frame rates together.

[ejmartin]

"one might think that read noise is a fixed cost per pixel in recording an image, again assuming that the same quality of circuit elements are employed."

This does appear to be true, but should one measure the cost in terms of noise expressed in data numbers (DN, raw pixel values) or electrons?

For the noise coming from sensor readout (what I called noise "upstream" of the ISO amplifier) electrons makes sense to me -- the noises are directly related to counting electrons; for subsequent contributions to noise (what I called "downstream" noise) the electron count has long since been converted into a voltage, the noises are voltage fluctuations, and I don't see any great benefit to continuing to think of the noise in terms of electrons.  I chose ADU for a couple of reasons; first, converting the data to electrons compounds measurement errors -- those of the read noise measurement, which is directly in ADU, and the gain measurement, are included when one refers the noise to electrons.  Second, it seems more intuitive to me to set the noise in ADU, since it grows with ISO due to the amplification of the sensor readout noise, and this accords with the increased visual appearance of noise at higher ISO (rather, decreased S/N).

In Clark Fig 3 of a post, Roger Clark notes that read noise expressed in electrons is similar in sensors of the same generation but of differing pixel sizes. Read noise varies with ISO, but read noise quoted in manufacturer's spec sheets is typically for unity gain, which complicates the analysis since unity gain is correlated with pixel size.

If the read noise is similar for large and small pixels, the large pixel camera still has an advantage when read noise is measured in DNs rather than electrons. This is because of camera gain as Roger explains in this example: Clark Table 3. See the text just following table 3.

"The realization from Table 3 is even though the read noise is similar in terms of electrons, the effect on the image is huge because of the gain factor. Thus shadow detail on a small sensor is severely compromised as the gain factor drops."


Bill
[a href=\"index.php?act=findpost&pid=198063\"][{POST_SNAPBACK}][/a]

Well, this ties in with the previous comments -- it depends which noise contributions one wants to focus on as staying fixed.   By referring back to electrons, downstream noise contributions to read noise are falsely downplayed in the DSLR relative to the digicam (since they are more fixed in ADU; only upstream read noise and photon noise are properly referred to electrons).

There is no doubt that small pixels gather fewer photons, have lower gain/pixel, and so have lower SNR/ph; but they do NOT, as I tried to emphasize, have a substantially lower SNR/mm.  One should be clear in these sorts of comparisons whether one is talking about small vs large format sensors, and the different pixel sizes they entail -- or about merits/deficiencies of higher pixel densities on a given format; that makes a huge difference in the conclusions to be drawn.

[bjanes]

For the noise coming from sensor readout (what I called noise "upstream" of the ISO amplifier) electrons makes sense to me -- the noises are directly related to counting electrons; for subsequent contributions to noise (what I called "downstream" noise) the electron count has long since been converted into a voltage, the noises are voltage fluctuations, and I don't see any great benefit to continuing to think of the noise in terms of electrons.  I chose ADU for a couple of reasons; first, converting the data to electrons compounds measurement errors -- those of the read noise measurement, which is directly in ADU, and the gain measurement, are included when one refers the noise to electrons.  Second, it seems more intuitive to me to set the noise in ADU, since it grows with ISO due to the amplification of the sensor readout noise, and this accords with the increased visual appearance of noise at higher ISO (rather, decreased S/N).

Well, this ties in with the previous comments -- it depends which noise contributions one wants to focus on as staying fixed.   By referring back to electrons, downstream noise contributions to read noise are falsely downplayed in the DSLR relative to the digicam (since they are more fixed in ADU; only upstream read noise and photon noise are properly referred to electrons).

[a href=\"index.php?act=findpost&pid=198359\"][{POST_SNAPBACK}][/a]

I agree that expressing noise in ADUs rather than electrons is most logical, since ADUs are what one measures and what are in the raw file. However, since electrons and ADUs are related by a linear scaling factor (the gain, expressed as electrons/ADU), one may use either in calculations. The point of my post was that shot noise and read noise scale differently with respect to pixel size.

The following chart represents a noise model for three different pixel sizes for sensors with otherwise identical characteristics. Calculations are shown for the maximum signal and for the signal 8 stops down from saturation. With the 8 um pixel,  shot noise predominates at all signal levels, whereas with the 2 um pixel read noise becomes important in the shadows. The chart demonstrates the effect of gain on the read noise for the different pixel sizes. Although the read noise is the same when expressed in electrons (5 e-) for all pixel sizes, the read noise expressed in ADUs (DN in the chart), is much higher for the smaller pixels.

[vandevanterSH]

I found this interesting:

"On the other hand, if shooting raw, it makes little sense to use the extended ISO's since they are simply mathematical manipulations of the raw data post-capture, and their main effect is to throw away one or more stops of highlight headroom as the doubling, quadrupling etc of the raw values pushes more and more of them beyond the maximum recordable value of 4095 for 12-bit, or 16383 for 14-bit data. Setting the highest analog ISO amplification keeps the headroom, and one can always do as much additional software amplification as is needed afterward during raw conversion."

Does this also apply to the "low end"?  For the D3 and D300, ISO 200 is the lowest "real" ISO but there is an L1 ISO 100..is something "thrown away" with this setting?

Steve

[bjanes]

I found this interesting:

Does this also apply to the "low end"?  For the D3 and D300, ISO 200 is the lowest "real" ISO but there is an L1 ISO 100..is something "thrown away" with this setting?

Steve
[{POST_SNAPBACK}][/a]

For additional analysis, you might want to look at Roger Clark's article on testing the Canon 1D MII in the conclusions section, where he discusses ISO 50 on the Canon cameras:

[a href=\"http://www.clarkvision.com/imagedetail/evaluation-1d2/index.html]Clark Conclusions[/url]

In the case of low ISO, additional exposure is given and the camera amplifier gain is decreased. The S:N is increased, but the dynamic range is decreased and there is danger of blowing the highlights.

Active D lighting is essentially the reverse. Exposure is decreased, giving more headroom for the highlights, and the amplifier gain is increased.

Bill

[vandevanterSH]

For additional analysis, you might want to look at Roger Clark's article on testing the Canon 1D MII in the conclusions section, where he discusses ISO 50 on the Canon cameras:

Clark Conclusions

In the case of low ISO, additional exposure is given and the camera amplifier gain is decreased. The S:N is increased, but the dynamic range is decreased and there is danger of blowing the highlights.

Active D lighting is essentially the reverse. Exposure is decreased, giving more headroom for the highlights, and the amplifier gain is increased.

Bill
[a href=\"index.php?act=findpost&pid=198550\"][{POST_SNAPBACK}][/a]

For the Nikon,  I am assuming that ISO 200 is the lowest AD amp gain and that the L1 (100) ISO is an in camera software manipulation.  If so, does the L1 "ISO" provide any benefit for a RAW image that can't be done better on a computer.  Also, is there a "downside" of L1 as there seems to be with the H1 and H2 "ISO" on the high end.

This question came to mind after reading that Michael used "ISO" 100 to expose his "Maelstrom" photo.

Steve