Pages: [1]   Go Down

Author Topic: IF, WHEN and HOW to use higher ISOs  (Read 14321 times)

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
IF, WHEN and HOW to use higher ISOs
« on: March 31, 2008, 08:18:12 pm »

The high ISO capabilities of cameras seem to fascinate many photographers. Unfortunately, there are many misconceptions about this subject. I undertake an attempt to make the subject understandable with detailed demonstrations.

The demonstrations are based on images created by a Nikon D3, but everything applies to many other cameras. The analysis was done on unadultered raw data, by Rawnalyze.

The highest "true" ISO setting for a few other cameras:

- Nikon D300: 1600
- Nikon D40X: 1600
- Canon 40D: 1600
- Canon G9: 800
- Sony A700: 2500 (?)
- Olympus E3: 3200

Six images are used in the demo: one ISO 6400, one 12800 and one 25600, all in 12bit and 14bit. The setting was the same, the results are comparable.

There was an important difference between the two sets, more important than the bit depth. The 12-bit set was created with different exposures (1 EV lower with ISO 12800 than ISO 6400, and two EV lower for ISO 25600); thus, the histograms of these shots look identical, at least at first sight.

The 14-bit set was created with fixed exposure, measured with ISO 6400, thus the 12800 and 25600 images are much brighter than the ISO 6400 image.

(The images used in the demonstration are courtessy Albert C. Lee, a fellow poster on the DPReview forum and the owner of a D3, unlike me.)

For those interested: before reading on, pls download following TIFF, as I will refer to the 25 or so layers in it. File size: 6 MB.

Screen captures in layered TIFF

The speciality of "very high" ISOs

The main difference between ISOs is, of course the gain: how the electric change is converted in a numerical value; another difference is, how reliable the result is, i.e. how "clean" the resulting data is (noise).

And then there is a little known difference: if the gain with an ISO is really different from the gain of other ISOs. The answer is in short: not always.

Specifically relating the D3: the highest ISO, which yields a different gain from the lower settings is ISO 6400. The higher settings are in reality equal to ISO 6400, but the resulting pixel values are numerically manipulated. Actually, the same is true relating to the 1/3 EV settings as well, but I have not analyzed these with the D3.

Before anyone gets upset either with this fact, or with me stating it: there is nothing new on this, most cameras are acting so. My Canon 40D goes up to ISO 3200, but ISO 1600 is the last true ISO, and all 1/3 EV ISOs too are numerical derivatives.

I will refer to those ISO settings as "fake ISO".


I am pretty sure some (or many) of the readers would immediately ask for proof, so I rather demonstrate the situation now.

The layers are in three groups; now I am referring to the group Histograms. There are two screen captures for each shot, both show the *raw* histogram of the shot. The "coarse" histogram shows the entire range, 0-4095 or 0-16383. The histogram consists of 512 columns; accordingly, each column represents an 8 or 32 wide range of pixel values. However, there is one column for each pixel value in the "fine" histogram; the range of currently displayed pixel values is shown as well.

We are dealing now exclusively with the fine histograms.


Let's see first ISO 6400, 14 bit. Pls magnify the image in PS, so that you can mark an area and count the number of columns.

There are "gaps" (black columns) in the histogram; their meaning is, that there are no pixels with the respective pixel values. When this occurs in the middle of a large "bulk", like here, then this fact has a special meaning: that pixel value is "spare".

Take a look at the magnified red histogram. Mark a repetitive range of gaps and count them. I see 17 gaps in a 100 wide range, and this remains so in higher ranges as well, with some small disturbances. Thus, we can say that the proportion of effective pixel levels is about 83%. This translates to about 13600 effective levels from the 16384 (don't worry, they are more than enough). Actually, these numbers changes slightly with the firmware version, but this scheme applies to all lower ISOs as well.

The blue channel is a bit different, the proportion of the effective pixels is about 92%.

Forget about the green channel, it seems to be "stirred up".


Now, let's go to the ISO 12800 image. The proportion of effective red levels is about 43% and of the blue ones is about 46%.

The ISO 25600 image yields about 21% for the effective red pixels and 23% for the blues.

What does thie above mean? It's simple: the pixel values gained from ISO 6400 get multiplied by two and that becomes ISO 12800, multiplied by four it "yields" ISO 25600.

This trick can not be observed on the 12-bit images so clear, because the number of levels are enough to populate the range better, even after the multiplication by two. However, the top edges of the histograms are telling, and the histogram of ISO 25600 shows the gaps clearly.

This is the reason I call these "fake ISO".

High ISO and noise

It is a very common misconception, that high ISO causes noise. The truth is, that not the ISO is the cause of the noise but the low exposure; although this often goes hand in hand with high ISO, this differentiation is very (but very) important.

Let's see now the noise in the demo-images.

Notes:

1. the color of the crops is off, for


a. Rawnalyze, the program I am using for the analysis does not carry out demosaicing and color conversion from the camera's color space in sRGB,

b. no WB has been carried out yet, for I wanted to keep the raw channel related values intact even in the RGB statistics,

Furthermore, the colors from the 12-bit and 14-bit images are way different due to different light source,

2. the relevant difference between the 12-bit and 14-bit set is *the exposure*; the bit depth does not play any role here, except in the appearance of the fine histogram.

So, let's compare the noise.


First, compare the exposures on the coarse historams of the 12-bit shots: they are practically identical (of course, as the shutter was halved with doubling the ISO). Based solemnly on the histogram, the images ought to look quite the same.

Disable the Histograms layer group and open the Noise group, comparing the layers "ISO 6400 12bit noise" with "ISO 12800 12bit noise" and "ISO 25600 12bit noise". The higher noisiness of the higher ISO images is not only clearly visible, but it can be quantified on the RGB statistics: the third number in each of the three number groups is the average RGB pixel valu; these remain relative consistent between the images. However, the fourth number, the standard deviation increases sharply with the ISO.

(These statistics values relate to the small selection, marked by an orangy recangle.)

The reason for the noise can be seen in the "ISO 6400 12bit to16" etc. layers: the strangely colored pixels are those with values under 16.

I chose 16 as pixel value limit for "noise" with 12bit depth, and 64 with 14bits. This choice is not completely arbitrary: these are the "borders" to the 9th EV from the maximum exposure downwards.

Anyway, the number of pixels in the "noise zone" increased drastically with the higher ISOs. I.e. the noise is the product of low pixel values with high variations (here quantified by the standard deviation).

Now, check out the coarse histograms of the 14bit images. (I feel compelled to repeat: this is not a 12bit vs 14bit issue!) The histograms reflects the fact, that the aperture and shutter were the same in all three shots. And the noise? Virtually equal in all three shots, with close the same proportion of "low" pixels - despite the different ISOs.

So, do higher ISOs cause the noise or the low exposure?

Dynamic range

I hope to have made it understandable by now, that ISO 12800 and 25600 on the D3 do not contribute to the *raw* image data at all. However, is that all to know?

Using the fake ISOs is not only not advantageous, it is doublessly disadvantageous, because the "numerically improved" pixel values take up space in the numerical range. This sounds a bit abstract, what does it mean?

First the demonstration: pls turn on the Clipping layer group and off the others. There are three captures there, showing the same segment of the image.

One capture is with ISO 6400, 14bit. The clipping indication is turned on (the checkbox "Raw clipping"), but there is no clipping here, apart from a few stray pixels (and those are clipped at the dark end). The DoF was not particularly large, but some details are visible in the middle of the crop as well.

Now, look at the layer "ISO 25600 14bit highlights gone": some of those details are not perceivable any more. Why? Look at the next layer, showing the clipping. Magenta indicates the lack of green, and the red shows, that both the green and the bluwe pixels clipped there.

So, using ISO 25600 not only did not contribute to the image quality, but it damaged that: it caused clipping of some areas. 12800 decreases the dynamic range by one stop and 25600 by two stops, compared to 6400.

Conclusion

Using the fake ISOs should be limited to cases, when the exposure can not be increased by any means, AND either JPEG image is created in-camera, or the brightness of the image on the LCD is crucial (why ever).
« Last Edit: March 31, 2008, 08:38:08 pm by Panopeeper »
Logged
Gabor

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
IF, WHEN and HOW to use higher ISOs
« Reply #1 on: March 31, 2008, 09:11:39 pm »

I only read now the short conclusion, and is what I expected. And coming from you I am pretty sure is all true.
I will take the whole article tomorrow.

Regards.

PS: just to add more info about the topic, I did an analysis on the Canon 5D ISO50 and ISO3200, the conclusions were different on each:

Canon 5D ISO50
ISO50 is a shot taken at ISO100 but with exposure values accordingly to an ISO50 light measure.
So shooting ISO50 on the 5D is _exactly_ the same as shooting ISO100 with +1EC set on camera.

Usage:
- For RAW users is stupid and creates confusion since camera's display will show less exposure than real in RAW file, and what you achieve with it can be obtained shooting at ISO100
- For JPEG users can help to obtain longer exposure times (silk effect,...)

Note: RAW developers and image viewers will _display_ ISO50 1 f-stop darker than ISO100 with same aperture/shutter; while both have identical RAW histograms.


Canon 5D ISO3200
ISO3200 is a shot taken at ISO1600 but with exposure values accordingly to an ISO3200 light measure. After that, the camera _modifies_ the RAW data by doubling all levels (i.e. applies +1EV).
So shooting ISO3200 on the 5D is _exactly_ the same as shooting ISO1600 with -1EC set on camera _and_ applying +1EV exposure compensation in the RAW developer.

Usage:
- For RAW users should be simply forbidden, since it blows 1 f-stops in the highlights with no benefit at all.
- For JPEG users can help to obtain a reasonable degree of exposure under very low light conditions


The whole article (Spanish) with image samples and explanation: ISO50 & ISO3200 ON CANON 5D
« Last Edit: March 31, 2008, 09:18:02 pm by GLuijk »
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
IF, WHEN and HOW to use higher ISOs
« Reply #2 on: April 01, 2008, 12:46:12 am »

Quote
ICanon 5D ISO50
ISO50 is a shot taken at ISO100 but with exposure values accordingly to an ISO50 light measure.
So shooting ISO50 on the 5D is _exactly_ the same as shooting ISO100 with +1EC set on camera.
This surprizes me. I do not have any ISO 50 image from the 5D. Do you mind uploading a well-exposed one?
Logged
Gabor

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
IF, WHEN and HOW to use higher ISOs
« Reply #3 on: April 01, 2008, 04:06:40 am »

Quote
This surprizes me. I do not have any ISO 50 image from the 5D. Do you mind uploading a well-exposed one?

Sure Gabor, I was offered them here. Links to original RAW files in the end of post #56.

I calculated -D with DCRAW, which means just extract RAW data (no scaling, no demosaicing,...) and at zoom 1:1 both histograms were clones.
I was most surprised at ISO3200, never thought the camera could modify the RAW data, hence blowing 1 f-stop in the highlights. A total disappointment taking into account that many RAW shooters insist ISO3200 is useful to them.
« Last Edit: April 01, 2008, 08:20:55 pm by GLuijk »
Logged

David Sutton

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1345
    • David Sutton Photography
IF, WHEN and HOW to use higher ISOs
« Reply #4 on: April 01, 2008, 04:29:05 am »

Hi Panopeeper. Thank you for the article. So the jist is: turn off the 1/3 ev? Good. I'm happy to take you advice and I've done this on my 40D. Cheers, David
Logged

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
IF, WHEN and HOW to use higher ISOs
« Reply #5 on: April 12, 2008, 12:12:06 pm »

Quote
Hi Panopeeper. Thank you for the article. So the jist is: turn off the 1/3 ev? Good. I'm happy to take you advice and I've done this on my 40D. Cheers, David
[a href=\"index.php?act=findpost&pid=186038\"][{POST_SNAPBACK}][/a]

Only for xxD series Canon cameras.  For 1 series and 5D, the intermediate ISO's are done through analog amplification, and there is a slight benefit in noise to doing it in-camera (though much less than for the main ISO's 100-200-400 etc) which might compensate somewhat for the loss of headroom compared to underexposing and boosting exposure in PP.  For xxD series Canons, the intermediate ISO's are implemented through software multiplication after quantization of the raw data, it is done as well or better in the raw converter (assuming the latter is competently programmed) without any loss of highlight headroom as there is when it is done in-camera.

For Nikons, all ISO's are implemented uniformly and the issue does not arise.
« Last Edit: April 12, 2008, 12:42:30 pm by ejmartin »
Logged
emil

bernie west

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 323
    • Wild Photo Australia
IF, WHEN and HOW to use higher ISOs
« Reply #6 on: April 12, 2008, 05:12:55 pm »

Quote
For 1 series and 5D, the intermediate ISO's are done through analog amplification[a href=\"index.php?act=findpost&pid=188972\"][{POST_SNAPBACK}][/a]

That's interesting.  I was always under the impression that for the 5D they were software fudges.  I've never read anything, but other people often quote white papers as their source.  How did you determine this for the 5D?
Logged

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
IF, WHEN and HOW to use higher ISOs
« Reply #7 on: April 12, 2008, 05:49:07 pm »

Quote
That's interesting.  I was always under the impression that for the 5D they were software fudges.  I've never read anything, but other people often quote white papers as their source.  How did you determine this for the 5D?
[{POST_SNAPBACK}][/a]

From this thread at OPF which discusses combing artifacts in raw histograms for both the 5D and 30D:

[a href=\"http://www.openphotographyforums.com/forums/showthread.php?t=546]http://www.openphotographyforums.com/forum...hread.php?t=546[/url]


Basically, if one looks a the raw histograms, if the intermediate ISO's were software fudges one would see gaps in the histogram for all color channels, with the percentage of gaps corresponding to the amplification from the lower ISO it is derived from by a software push (if derived from higher ISO by pulling, there will be a pattern of spikes on an otherwise smooth envelope).  For a 1/3 push there will be roughly 20% gaps, for a 2/3 push about 37%.  There are no such gaps or spikes at intermediate ISO's on the 5D, and so the conclusion is that the amplification is analog.  

Furthermore, the read noise data is a nice fit to a model that supposes analog amplification for the intermediate ISO's.  I posted the results in this thread at the end:

http://www.openphotographyforums.com/forum...p?t=4771&page=2

which I hope to write up in more detail some time soon.
« Last Edit: April 13, 2008, 12:04:07 am by ejmartin »
Logged
emil

bernie west

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 323
    • Wild Photo Australia
IF, WHEN and HOW to use higher ISOs
« Reply #8 on: April 12, 2008, 07:28:38 pm »

That's good to know.  From memory, I don't think you can turn off the 1/3 stop isos on a 5D, and that would explain why.  However, a few years ago I do remember seeing an analysis by someone on some forum (sorry, forgot all details) to do with noise levels at each iso.  And basically the graph roughly went from relative lows at the full stop isos to higher levels at the intermediate isos.  The implication was that as the signal was pushed on the discrete data after the fact, then noise would increase more so than when pushed on the analogue data.
Logged

bernie west

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 323
    • Wild Photo Australia
IF, WHEN and HOW to use higher ISOs
« Reply #9 on: April 12, 2008, 07:41:14 pm »

I've found the post I was referring to above.  It was at http://photo.net/bboard/q-and-a-fetch-msg?msg_id=00KTHK.  There is a graph a little over half way down and an explaination a few posts further on.  See what you make of it.
Logged

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
IF, WHEN and HOW to use higher ISOs
« Reply #10 on: April 17, 2008, 01:21:03 am »

Thank you for your analysis. You proved something that was well known amongst the community.

Let me just point that Nikon, at least, seem fairly honest in what they document although in an implicit sort of way. They only refer to the 'real' ISOs as ISOs (including their 1/3 stops) and refer to the others as Hi and Lo respectively.
Logged
Nikos

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
IF, WHEN and HOW to use higher ISOs
« Reply #11 on: April 17, 2008, 10:30:12 am »

Quote
Conclusion

Using the fake ISOs should be limited to cases, when the exposure can not be increased by any means, AND either JPEG image is created in-camera, or the brightness of the image on the LCD is crucial (why ever).
[a href=\"index.php?act=findpost&pid=185920\"][{POST_SNAPBACK}][/a]


Gabor,
That's a good point. It's sometimes useful to be able to see the review of the shot clearly, especially if you are under-exposing to a greater degree than one stop at ISO 1600. However, there is an unnecessary danger when using ISO 3200 with the intention of getting a full ETTR. A slight over-exposure might result in a ruined shot, whereas the same exposure used at ISO 1600 instead, would result in, say, a 1/2 stop under-exposure instead of the full stop underexposure which ISO 3200 represents. In such circumstances, the choice of ISO 1600 would result in an image with no blown highlights, but also, significantly, no shadow noise disadvantage or any other disadvantage compared with the same exposure at ISO 3200.

The only purpose for these false ISOs I can think of, for RAW shooters, is the convenience of being able to see a correctly exposed review on the camera's LCD screen, instead of an underexposed review with impenetrable shadows.
Logged

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
IF, WHEN and HOW to use higher ISOs
« Reply #12 on: April 17, 2008, 03:55:43 pm »

Ray,

I suppose your choice of ISO values in your post above  is meant to be indicative as these will vary between cameras.
Logged
Nikos

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
IF, WHEN and HOW to use higher ISOs
« Reply #13 on: April 17, 2008, 09:41:14 pm »

Quote
Ray,

I suppose your choice of ISO values in your post above  is meant to be indicative as these will vary between cameras.
[a href=\"index.php?act=findpost&pid=190210\"][{POST_SNAPBACK}][/a]


Yes. I should have mentioned I was referring specifically to the 5D. With a camera such as the D3, the same principle would apply at ISOs 12,800 and 25,000, although I might prefer to underexpose 1 stop at ISO 12,800 rather than 2 stops at ISO 6,400 for the sake of a clearer review image on the camera's LCD.

Of course, the D3 has the advantage of auto ISO. At high ISO's this could be a trap. For example, you choose the shutter speed and aperture that you consider is appropriate, but the camera chooses, say, ISO 25,600 which might result in some street lamps being unnecessarily blown. In such circumstances, it would be better to under-expose 2 stops at ISO 6400. Doing so would retain the same shadow detail and same degree of noise as the shot at ISO 25,600, but allow for more tonality in the street lamps or any other excessively bright areas of the image.
Logged

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
IF, WHEN and HOW to use higher ISOs
« Reply #14 on: April 18, 2008, 12:23:12 am »

Quote
Yes. I should have mentioned I was referring specifically to the 5D. With a camera such as the D3, the same principle would apply at ISOs 12,800 and 25,000, although I might prefer to underexpose 1 stop at ISO 12,800 rather than 2 stops at ISO 6,400 for the sake of a clearer review image on the camera's LCD.

Of course, the D3 has the advantage of auto ISO. At high ISO's this could be a trap. For example, you choose the shutter speed and aperture that you consider is appropriate, but the camera chooses, say, ISO 25,600 which might result in some street lamps being unnecessarily blown. In such circumstances, it would be better to under-expose 2 stops at ISO 6400. Doing so would retain the same shadow detail and same degree of noise as the shot at ISO 25,600, but allow for more tonality in the street lamps or any other excessively bright areas of the image.
[a href=\"index.php?act=findpost&pid=190297\"][{POST_SNAPBACK}][/a]

Ray,

Auto ISO in Nikon cameras is implemented such that you can choose the highest ISO it will go up to. So no problem in that sense.
Logged
Nikos

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
IF, WHEN and HOW to use higher ISOs
« Reply #15 on: April 18, 2008, 05:18:20 am »

Quote
Ray,

Auto ISO in Nikon cameras is implemented such that you can choose the highest ISO it will go up to. So no problem in that sense.
[a href=\"index.php?act=findpost&pid=190318\"][{POST_SNAPBACK}][/a]

Sure, but isn't that another issue. What happens if you forget to exclude those false ISO's or are not aware that there is any advantage to excluding them?
Logged

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
IF, WHEN and HOW to use higher ISOs
« Reply #16 on: April 18, 2008, 06:00:17 am »

Quote
Sure, but isn't that another issue. What happens if you forget to exclude those false ISO's or are not aware that there is any advantage to excluding them?
[a href=\"index.php?act=findpost&pid=190342\"][{POST_SNAPBACK}][/a]

Ray,

You're a master of cyclic trains of thought  
Logged
Nikos

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
IF, WHEN and HOW to use higher ISOs
« Reply #17 on: April 18, 2008, 10:20:53 am »

Quote
Ray,

You're a master of cyclic trains of thought  
[a href=\"index.php?act=findpost&pid=190348\"][{POST_SNAPBACK}][/a]

Thanks! I always knew, deep down, I was master of something. I just wasn't sure what it was, though.  
Logged
Pages: [1]   Go Up