noise (high ISO vs. long exposure)

[macgyver]

So...would it be better to shoot at 1600 and push a stop?

[John Sheehy]

I wasn't aware of that being the case with the 30D's "between stops" ISO settings. The 1-series bodies seem to use varying voltage amplification for all ISO settings in the "native range", as opposed to firmware tricks on the RAW data after-the-fact. What tool are you using to check/verify this?
[a href=\"index.php?act=findpost&pid=91275\"][{POST_SNAPBACK}][/a]

The histograms of the RAW data show clear manipulation.  Gaps and spikes spaced just right to show that the data was integer-scaled.

IRIS is the easiest tool for examining RAW data.  It loads all RAW data literally, with no manipulation.

Even thought the 1-series and 5D use analog amplification for the in-between ISOs, they can also be quite inefficient noise-wise, because the cameras are optimized for the higher ISOs, so a 160 that is 100 amplified by 1.6 has more shadow noise than 200.

[John Sheehy]

So...would it be better to shoot at 1600 and push a stop?
[a href=\"index.php?act=findpost&pid=91306\"][{POST_SNAPBACK}][/a]

If you shoot RAW and want the extra highlight headroom; many ISO 3200 scenes are ones with lights in them, at night, so that can come in handy.

I often shoot low light in Tv-mode, so I set the camera to 1600 and as high an EC as I can safely use; that covers a wide range.  If light is ample, I can get a very good ISO 800 or even 400 exposure, and if the light is weak, I just have whatever I get.

[macgyver]

If you shoot RAW and want the extra highlight headroom; many ISO 3200 scenes are ones with lights in them, at night, so that can come in handy.

I often shoot low light in Tv-mode, so I set the camera to 1600 and as high an EC as I can safely use; that covers a wide range.  If light is ample, I can get a very good ISO 800 or even 400 exposure, and if if light is weak, I just have whatever I get.
[a href=\"index.php?act=findpost&pid=91381\"][{POST_SNAPBACK}][/a]

Meh.  When I'm shooting 3200 its usually because I'm shooting college sports for the university paper, meaning I'm shooting JPG anyway.  I guess I'll just remember this for other more vital projects, thanks for the info.

[Chris_T]

The in-between ISOs on the 30D are made the same way, even if they are within what you call the "native range".  ISO 160 is 200 pulled one stop, then the digitized data divided by 1.26; ISO 250 is 200 pushed one stop and multiplied by 1.26.  The ISO 250 RAW histogram has a gap about every 5th value, ISO 160 has a doubly-populated value about every 4th value.  ISO 250 is missing 1/3 stop of highlights that it had from before its digitization was multiplied.
The lowest ISO requires amplification, too.  Just not as much as the higher ISOs.  There is some constant, K, where ISO 100 has 1K gain, ISO 1600 16K gain, etc.

Even when signals are not amplified, they are still buffered, which is basically an amplification that just happens to be unity.
The native base ISO of the sensor may be somewhere between 0.7k to 1.5K, but not used by the camera as such.
[a href=\"index.php?act=findpost&pid=91271\"][{POST_SNAPBACK}][/a]

These are much better answers. Thanks.

In summary, "native" and "native range" ISO settings are controlled by hw, while "artificial" ISO settings are altered by sw. But unfortunately, the manufacturers won't tell you which is which.

[John Sheehy]

In general there is not much point using an ISO much higher than that at unity gain of the sensor, which maxes out at 1600 for the Canon 5D and is probably around 1000-1200 for your camera. At unity gain one electron corresponds to one 12 bit data number in the raw file, and it does not make sense to digitize anything less. In other words, ISOs above 1600 achieve little benefit.
[a href=\"index.php?act=findpost&pid=91302\"][{POST_SNAPBACK}][/a]

We've been through this before -  there is nothing special about unity, and nothing in Roger's work proves that there is.  He is using a mathematically-derived 3200, which tells us nothing about what a gain-based one might be.

The cameras aren't even close to counting electrons.  The noise makes the count very uncertain.  The RAW ADU unit is pretty meaningless, too.  Would lowering the bit depth to 11 bits make one more stop of gain (3200) suddenly feasible?  Would 13 bits make it suddenly only feasible to do 800?

The only trade-off concern of any value is the noise, and it is very clear that in the recent Canon DSLRs, graphing various noise components at various ISOs shows improvement possible past ISO 1600.  Graphing 3200 does not show what 3200 would be if it were analog; it is only 1600 gain repeated, as measured in electrons.

There is no real unity in any real physical sense, even if it did mean something.  There is simply a trade-off curve; additional amplification means less noise as measured in electrons, though the benefit gets smaller and smaller with each increase.  So-called "unity" is only a featureless point in the curve.

[bjanes]

We've been through this before -  there is nothing special about unity, and nothing in Roger's work proves that there is.  He is using a mathematically-derived 3200, which tells us nothing about what a gain-based one might be.

The cameras aren't even close to counting electrons.  The noise makes the count very uncertain.  The RAW ADU unit is pretty meaningless, too.  Would lowering the bit depth to 11 bits make one more stop of gain (3200) suddenly feasible?  Would 13 bits make it suddenly only feasible to do 800?

The only trade-off concern of any value is the noise, and it is very clear that in the recent Canon DSLRs, graphing various noise components at various ISOs shows improvement possible past ISO 1600.  Graphing 3200 does not show what 3200 would be if it were analog; it is only 1600 gain repeated, as measured in electrons.

There is no real unity in any real physical sense, even if it did mean something.  There is simply a trade-off curve; additional amplification means less noise as measured in electrons, though the benefit gets smaller and smaller with each increase.  So-called "unity" is only a featureless point in the curve.
[{POST_SNAPBACK}][/a]

Yes, we have gone over this before and I'm not certain that there is much to add. In Table 1a in the link, Roger does give data for ISO 3200 but it is not clear to me if it they are measured experimentally or derived mathematically.

[a href=\"http://www.clarkvision.com/imagedetail/evaluation-1d2/index.html%5bURL=http://www.clarkvision.com/imagedetail/evaluation-1d2/index.html]Clark Table 1a[/url]


Unity gain would change with bit depth, but whatever the bit depth, it does not make sense to quantize the signal any finer than to the nearest electron, especially when the read noise at ISO 1600 for the 1D M2 is 3.9 electrons and the photon noise down 8 stops from maximum is around 5 electrons (both expressed as SD). According to Roger's table, read noise actually goes up at ISO 3200 and dynamic range is lower.

If you have other data, I would be interested in seeing them.

Bill

[John Sheehy]

According to Roger's table, read noise actually goes up at ISO 3200 and dynamic range is lower.[a href=\"index.php?act=findpost&pid=91759\"][{POST_SNAPBACK}][/a]

There is no ISO 3200 on any Canon.  All Canon ISO 3200s are an illusion.  They are ISO 1600, under-exposed by a stop, and then the RAW numbers doubled (or in the case of the 10D, the numbers are left alone as ISO 1600 numbers, which are 800 doubled, and it is assumed that a converter will recognize the need for a stop more software EC).

Doubling causes a stop of DR to be clipped away.

If you have other data, I would be interested in seeing them.

This is read noise on my 20D, total, and broken down further into horizontal and vertical line noise (banding):



The yellow line is divided by 10, so that all the data fits into the same area.  The 3200 is artificial, and has the same noise in electrons, or the normalized ISO 100 ADUs that I use here (which are directly proportional to electrons).

The trend shows room for small improvement for total and vertical line noise, and room for tremendous improvement for horizontal line noise, which is by far the most detrimental noise in Canon low-light shadows.  1-dimensional line noise is extremely problematic, as it does not disappear with small prints, or downsampling as much as 2D noise does.  I have severely binned down shots taken at ISO 3.2 million, and the only noise left is banding noise, in bold black and white bands for a greyscale image.

[bjanes]

This is read noise on my 20D, total, and broken down further into horizontal and vertical line noise (banding):

The yellow line is divided by 10, so that all the data fits into the same area.  The 3200 is artificial, and has the same noise in electrons, or the normalized ISO 100 ADUs that I use here (which are directly proportional to electrons).

The trend shows room for small improvement for total and vertical line noise, and room for tremendous improvement for horizontal line noise, which is by far the most detrimental noise in Canon low-light shadows.  1-dimensional line noise is extremely problematic, as it does not disappear with small prints, or downsampling as much as 2D noise does.  I have severely binned down shots taken at ISO 3.2 million, and the only noise left is banding noise, in bold black and white bands for a greyscale image.
[a href=\"index.php?act=findpost&pid=91850\"][{POST_SNAPBACK}][/a]

John,

The data are interesting, but I have difficulty interpreting them because I am not familiar with your terminology. What is the equation for normalized ISO 100 ADU and what are the units? Also, what is the equation for line noise, what are the units, and how is it measured?

Bill

[John Sheehy]

John,

The data are interesting, but I have difficulty interpreting them because I am not familiar with your terminology. What is the equation for normalized ISO 100 ADU and what are the units? Also, what is the equation for line noise, what are the units, and how is it measured?
[a href=\"index.php?act=findpost&pid=91899\"][{POST_SNAPBACK}][/a]

The vertical axis is standard deviation at ISO 100 in ADUs.  It is only literal for the ISO 100 line noises.  The ISO 100 total noise (yellow) is divided by ten, so it fits in the same chart, because the other two would just become unreadable flat lines if they were all at the same scale.

For ISO 200, all the values are divided by 2; for ISO 400, they're all divided by 4, etc.  That's to make the values in the chart meaningful relative to absolute signal.  This is the same thing as measuring in electrons (which noone really does; people just convert to electrons with an assumed knowledge of how ADUs translate to electrons), the unit is just arbitrary.  The unit is irrelevant to the point of the chart; the point of the chart is that noise at the higher ISOs doesn't look like it's going to level off immediately above 1600, with analog gain; especially horizontal line noise, THE most visible and annoying noise of Canon low-light shadows (even though it is only about 1/10 as strong as the total noise).  Roger's analysis looks at noise monolithically, as a statistic, with no concern for the fact that all noise is not equally visible.

The methodology is very simple; I take the standard deviation of the RAW blackframe bitmaps, and the bitmaps with all the pixels in each line averaged, both horizontal and vertical.

[Ray]

John,
Those are convincing graphs. Do we have an issue about in-camera processing of ETTR ISO 3200 shots versus 1 stop underexposed ISO 1600 shots processed in commonly used converters such as ACR, to which one might apply a +1 stop EC?

In other words, is the boost or doubling (in ISO 3200 mode) to the RAW ISO 1600 data, equal to, better than or worse than, the job that some, or all RAW converters might do with a 1 stop underexposed ISO 1600 shot?

[bjanes]

The unit is irrelevant to the point of the chart; the point of the chart is that noise at the higher ISOs doesn't look like it's going to level off immediately above 1600, with analog gain; especially horizontal line noise, THE most visible and annoying noise of Canon low-light shadows (even though it is only about 1/10 as strong as the total noise).  Roger's analysis looks at noise monolithically, as a statistic, with no concern for the fact that all noise is not equally visible.
[a href=\"index.php?act=findpost&pid=91912\"][{POST_SNAPBACK}][/a]

The unit of noise is important if one wishes to compare your data with other published data. Also, the definition of noise and how it is measured is important. Roger subtracts two frames taken under identical conditions and the difference represents random variations in the signal, the usual definition of noise. If one takes the standard deviation of an area of a homogeneous object in the image as a measure of noise, these deviations could represent random noise or systematic variations arising from nonuniform illumination or non-homogeneity of the sensor response or processing. Of course, for a black-frame, there is no illumination. Furthermore, once one begins to weight noise according to its visibility, complications imposed by the weighting method are introduced.

The methodology is very simple; I take the standard deviation of the RAW blackframe bitmaps, and the bitmaps with all the pixels in each line averaged, both horizontal and vertical.
[a href=\"index.php?act=findpost&pid=91912\"][{POST_SNAPBACK}][/a]

Is the standard deviation derived from variations within the lines or between the lines? Between line variations could be non-random and equalized by a smart algorithm. Finally, you concentrate on read noise, but most people are concerned with total noise. According to Roger's analysis, photon noise predominates except in the deepest shadows.

[Jonathan Wienke]

In other words, is the boost or doubling (in ISO 3200 mode) to the RAW ISO 1600 data, equal to, better than or worse than, the job that some, or all RAW converters might do with a 1 stop underexposed ISO 1600 shot?

If you're shooting RAW, ACR, C1, etc. are probably a bit more sophisticated than the one in-camera, but the difference you see is minor. I'd say "equal to" with perhaps a very slight edge to external processing. But if you shoot JPEG, it's better to let the camera push the exposure a stop in 12-bit mode internally than to push it in 8-bit mode externally.

[John Sheehy]

John,
Those are convincing graphs. Do we have an issue about in-camera processing of ETTR ISO 3200 shots versus 1 stop underexposed ISO 1600 shots processed in commonly used converters such as ACR, to which one might apply a +1 stop EC?

In other words, is the boost or doubling (in ISO 3200 mode) to the RAW ISO 1600 data, equal to, better than or worse than, the job that some, or all RAW converters might do with a 1 stop underexposed ISO 1600 shot?
[a href=\"index.php?act=findpost&pid=91924\"][{POST_SNAPBACK}][/a]

The main reasons that *I* don't use 3200 on my 20D are that 1600 has a stop more DR, and that most of the time I'm shooting in low light, the camera is set to Tv-pri AE, so with a positive EC, I stand a chance of getting an exposure index less than 1600 if the light improves, but I am not sacrificing much of anything by using 1600 instead of 3200 if I don't get a lot of light.

However, a couple of people have reported that ACR's line noise (banding reduction) works better with 3200.  I haven't tested that, but it seems logical that this is the case, since ACR most likely works in a fixed bit depth (possibly just 12 bits), so the 3200 RAW data is expanded one more bit.  In my more casual comparisons, I really haven't noticed much of a difference between the two except for the greater highlight headroom of the 1600.  There may not have been much visible banding in those comparisons.