Hi ISO is no easy stuff

[ErikKaffehr]

Hi,

Note: I have significantly rewritten the original posting.

Phase One now has the IQ 250, and one of the main sales arguments is high ISO. I guess that Phase One is doing OK about it.

What needs to be kept in mind is that light comes in Photons, and we are pretty good in capturing those Photons. Photons come in limited quantities. Let us assume that a pixel can detect 64000 photons and has base ISO of 100. In this case 64000 will correspond to saturation. Exposure will normally be chosen three stops (or so) below saturation so midtones (like 18% reflectivity) will receive 64000 / (2*3) = 8000 photons.

No, let us assume that nominal ISO is 100 and we expose at 6400, that is to reduce exposure 64 times. So, our mid tones will only get 8000 / 64 = 125 photons. The natural variation of photons will be the square root of 125 = 11.18. So most of the pixels will vary between 114 and 136 but quite a few will vary more. So, even with a perfect sensor, we will get a lot of noise, simply because of nature of light.

If we consider dark areas they will perhaps only see around 30 photons per pixel (125 / 4 = 31.25), with a natural variation of 5.5, say vary between 25 and 35 photons, quite a variation. The readout noise of the sensor and the thermal noise will be added to this variation.

Two things can be done to improve ISO:

- Increasing QE (Quantum Efficiency) which can be achieved by microlenses and more permissive CGA-s. The latter can have negative effect on colour rendition.

- Reducing read noise, that is something the Sony chips excel at

But whatever is done, high ISO will be low photo count territory and there will be a lot of noise due to photon statistics.

Best regards
Erik

[Bart_van_der_Wolf]

Hi,

Phase One now has the IQ 250, and one of the main sales arguments is high ISO. I guess that Phase One is doing OK about it.

What needs to be kept in mind is that light comes in Photons, and we are pretty good in capturing those Photons. Photons come in limited quantities. Let us assume that a pixel can detect 64000 photons and has base ISO of 100. At 6400 ISO it can collect 1000 photons, as we have 1/64-th the exposure.

Not exactly. The sensel can still collect 64000 photons, but we underexpose it so only 1000 are recorded. From there on, the ADC programming will determine whether the data is amplified to the maximum bit depth of the Raw file, e.g. 64x to produce 16-bit Raw Analog to Digital Units (ADUs), or 16x to produce 14-bit ADUs. There can also be some normalization of Color Balance, where not all channels are amplified the same amount, which could be used for later lossless or (lossy) compression.

Something like 2-3 EV needs to be reserved for highlights so we are down to perhaps 100-200 photons for mid tones and around say 15-20 for reasonably dark areas.

There is no need for highlight protection, if the ADU doesn't multiply to the maximum ADU size, and produces ISO-less Raw data. If the ADU does multiply, then we can still expose-to-the-right (ETTR) to preserve highlights, which may mean we have to underexpose the sensor even a bit more than the ISO setting would indicate.

The ADU conversion can also be programmed differently for different ISO settings.

But whatever is done, high ISO will be low photo count territory and there will be a lot of noise due to photon statistics.

Yes, that's the general cost of underexposing. Then there is additional noise that can muddy the water even more, things like Read-noise, Dark-current, pixel response non-uniformity (PRNU), and systematic Pattern-noise. At the low end, some manufacturers reserve some room in the Raw data range to more accurately record some of the noise floor by adding an offset to all ADUs, other manufacturers just clip some of that data. Some manufacturers even apply lossy noise reduction before writing the Raw data. It takes a bit more knowledge about the Raw data specifics to allow 'optimal' Rawconversion.

Cheers,
Bart

P.S. I edited my initial 6400 and 1600 multiplications to read 64x and 16x multiplications.

[jerome_m]

Not exactly. The sensel can still collect 64000 photons, but we underexpose it so only 1000 are recorded. From there on, the ADC programming will determine whether the data is amplified to the maximum bit depth of the Raw file, e.g. 6400x to produce 16-bit Raw Analog to Digital Units (ADUs), or 1600x to produce 14-bit ADUs.

I think that what Erik means is that, if we use such low exposure that we can collect at most 1000 photons in the pixels representing the highlights of our picture taken in darkness, we can only have 1000 different values amongst the pixels of our picture. Each pixel will have between 1, 2, 3... or up to 1000 photons. We may stretch the ADC so that the value "1000 photons" is in bin #64000, but we don't need 64000 bins, because most of them will be empty.

[Bart_van_der_Wolf]

I think that what Erik means is that, if we use such low exposure that we can collect at most 1000 photons in the pixels representing the highlights of our picture taken in darkness, we can only have 1000 different values amongst the pixels of our picture.

Hi Jerome,

It's possible that he intended it that way. I attempted to verify that he does, and add a warning for those who interpret it differently. The ISO setting alone doesn't have to change much, but it might (depending on ADC gain amplification). Underexposing will change everything (due to Poisson distribution statistics), regardless of the sensor.

Each pixel will have between 1, 2, 3... or up to 1000 photons. We may stretch the ADC so that the value "1000 photons" is in bin #64000, but we don't need 64000 bins, because most of them will be empty.

Indeed, although various noise sources will fill in some of the empty histogram spaces. We also need to assume that the ISO gain is 1.0 at some ISO setting. With an analog gain amplification it may well be close to 4.0 for 14-bit quantization at ISO 100 , i.e. 1 ADU difference per 4 photons, so 0.0625 at ISO 6400 or 16 ADU difference per photon. This will be relevant for the optimization of High ISO performance, which I expect (but I could be proven wrong) to be around ISO 1600, and underexpose with a postprocessing exposure push for lower light situations.

IMHO, there are just too many assumptions we need to consider at this moment because we do not yet know how the Raw quantization will actually work out for the new Sony sensor. The only conclusion we can draw is that underexposing will leave us with fewer photons, and no ISO setting can improve that.

Cheers,
Bart

[jerome_m]

I think that we should not forget the realities of photons counting. Whether we are underexposing or not, when it is dark and we push iso on the camera, we don't actually get many photons on the sensor. We might indeed get a maximum of 1000 per pixel for the highlights of the pictures and considerably less for the darker parts. So the camera will count values between 0 and 1000 photons per pixel. That makes 1001 possible values for the actual signal.

Now, we may have a detector with a full well capacity of over 60000 photons and we may digitise this with a 16 bits ADC which gives us 65536 discrete bins, but if the signal is between 0 and 1000 maximum, that capacity is wasted. It is wasted whatever the amplifier setting between the pixel and the ADC.

And actually, if we can only get 60000 photons at most when the pixel is full and can count up to 65536, we don't really need an amplifier at all... (I know that it actually helps to differentiate signal from other noise sources, I am just making the point that the ADC has more capacity than we can use with that full well capacity).

[ErikKaffehr]

Hi,

Amplifier is needed as the ADC probably cannot handle voltages corresponding to single electron charges. It also seems that external ADC:s are quite noisy, that is the reason Canon does not loose DR when increasing ISO to say 800. Amplifying makes better use of signal.

Best regards
Erik

I think that we should not forget the realities of photons counting. Whether we are underexposing or not, when it is dark and we push iso on the camera, we don't actually get many photons on the sensor. We might indeed get a maximum of 1000 per pixel for the highlights of the pictures and considerably less for the darker parts. So the camera will count values between 0 and 1000 photons per pixel. That makes 1001 possible values for the actual signal.

Now, we may have a detector with a full well capacity of over 60000 photons and we may digitise this with a 16 bits ADC which gives us 65536 discrete bins, but if the signal is between 0 and 1000 maximum, that capacity is wasted. It is wasted whatever the amplifier setting between the pixel and the ADC.

And actually, if we can only get 60000 photons at most when the pixel is full and can count up to 65536, we don't really need an amplifier at all... (I know that it actually helps to differentiate signal from other noise sources, I am just making the point that the ADC has more capacity than we can use with that full well capacity).

[bjanes]

Indeed, although various noise sources will fill in some of the empty histogram spaces. We also need to assume that the ISO gain is 1.0 at some ISO setting. With an analog gain amplification it may well be close to 4.0 for 14-bit quantization at ISO 100 , i.e. 1 ADU difference per 4 photons, so 0.0625 at ISO 6400 or 16 ADU difference per photon. This will be relevant for the optimization of High ISO performance, which I expect (but I could be proven wrong) to be around ISO 1600, and underexpose with a postprocessing exposure push for lower light situations.

IMHO, there are just too many assumptions we need to consider at this moment because we do not yet know how the Raw quantization will actually work out for the new Sony sensor. The only conclusion we can draw is that underexposing will leave us with fewer photons, and no ISO setting

Bart,

Your post has caused me to rethink some of my assumptions regarding unity gain. Roger Clark has stated "Since 1 electron (1 converted photon) is the smallest quantum that makes sense to digitize, there is little point in increasing ISO above the Unity Gain ISO". However, he does state that increasing ISO above unity gain does lead to some minor improvements quantization, since more of the full scale of the ADC can be utilized.

I previously reported my findings on unity gain for the Nikon D800e. The ISO 100 gain is about 3.24 e- /14 bit DN, and the gain is decreased by a factor of 2 for each doubling of ISO. It is unity at an ISO of approximately 320 as shown graphically below.



Bill Claff has published an excellent interactive graph showing the DR of various sensors with respect to ISO.



The Nikon D800e (CMOS) and D200 (CCD) are nearly ISO less in that increasing the ISO has little effect on DR. The D800e does show improvement up to unity gain and then minor improvements thereafter, perhaps due to better quantization or other factors. I should correct my statement that unity gain has little to do with optimal ISO in relation to DR with the D800e. It does have a small effect. The Canon CMOS sensors and the D4 CMOS show gains up to about ISO 1600 and taper off thereafter.

With the D800e, it makes little sense to use an ISO much over 800 other than obtaining a brighter image on the LCD preview and the downside of increasing the ISO beyond this point is decreased highlight headroom. With the D4 and Canons, increasing ISO beyond 1600 gives little benefit in DR. With any of these cameras, the use ISO 6400 gives little benefit and one would be better off leaving the ISO setting at 1600 and compensating exposure in the raw converter. With the D800e, one gains little more than 0.5 EV by increasing the ISO over baseline.

Bill

[Vladimirovich]

Bill Claff

btw is everything OK w/ him ? it seems that he was not active anywhere for > 1 year...

[BJL]

The ISO setting alone doesn't have to change much, but it might (depending on ADC gain amplification). Underexposing will change everything (due to Poisson distribution statistics), regardless of the sensor.

This sort of confusion would be avoided if we stopped using "ISO" to mean so many related but different things.  Erik is clearly talking about reducing exposure level and giving the sensor less light, and thus when he says "high ISO", he means high Exposure Index, one of the numerous quantities defined in ISO standards.  Exposure Index, is, by the way, the main intended significance of the "ISO" setting on digital camera, through effects like adjusting the shutter speed and aperture combination chosen in auto-exposure modes and doing default raw-to-JPEG conversions on the assumption that exposure levels are lower when a higher ISO exposure index setting is used.

But some people insist on assuming that measures like the saturation-based base-ISO sensitivity, a measure of highlight headroom in the sensor or raw files, are or should be what the ISO dial on a camera specifies.  By the way, the traditional ISO measure of film speed is distinctly different from either of these two: it is a measure of how high the exposure index can go (how little exposure the film can be given) while still meeting some standard for image quality in the shadows. The closest counterparts for digital photography are the rarely-used noise based SNR10 and SNR40 standards, which are the exposure index levels at which mid-tone SNR levels are 10:1 and 40:1 respectively.


Anyway, the bottom line is that Erik's figure of about 125 photons at mid-tones when exposing at an exposure index of 6400 limits the SNR to about 11:1 no matter how low the sensor's internal noise levels are. This is just better than the SNR10 standard, traditionally described as a barely acceptable mid-tone SNR level (while 40:1 in the mid-tones is traditionally described as "excellent").

So to restate Erik's conclusion in something closer to the language of the ISO standards: for a sensor with full well capacity of 40,000 and saturation-based "base-ISO" sensitivity of 100, an exposure index of 6400 gives a mid-tone SNR of at best about 11:1, so the more generous noise-based SNR10 sensitivity is at best just slightly more than 6400.  Perhaps Phase One has chosen its maximum exposure index setting of 6400 based on this being near the SNR10 limit.

Aside: for the more cautious SNR40 mid-tone noise standard, a mid-tone signal of at least 1600 is needed, and in Erik's case of exposure at EI 100 giving about a 8000 mid-tone signal, this would give a SNR40 sensitivity of only about 500! That EI level is about where people seeking excellent IQ and very fine tonal gradations still need to be operating, and it is determined almost entirely by photon shot noise and photons counts, not by the sensor's noise floor, so the new Sony sensor will not change that situation much.

[BJL]

Roger Clark has stated "Since 1 electron (1 converted photon) is the smallest quantum that makes sense to digitize, there is little point in increasing ISO above the Unity Gain ISO". However, he does state that increasing ISO above unity gain does lead to some minor improvements quantization, since more of the full scale of the ADC can be utilized.

I can see the possibility of benefit in going a little beyond unity gain. From a mathematical perspective, perfect "rounding" of signal to an integer level on an ADU scale would introduce an error of at most 1/2 unit in either direction (for example, a voltage that "should be" 23.5 units instead rounds to either 23 or 24), and that is a maximum error, to the typical (RMS) error is a bit less than 1/2. So I can see a possible case of counting at up to about two to four ADUs per electron before ADC quantization error dominates over the error in the incoming signal (i.e. the photo-electron count.)

P. S. However this is only relevant to very dark parts of the image; relevant to engineering measure of DR and to technical uses like astrophotography, but probably irrelevant to photography because all it can do is raise a pathetically low SNR is a very dark part of the image to a slightly higher but still miserable local SNR, that will still look horribly noisy if the levels are raised enough for it to display as anything other than black.

[ErikKaffehr]

Hi Bill,

You explain what I intended to say very well. The message I try to send that at high ISOs photon statistics are a real issue, that sensor makers can do little about.

Best regards
Erik

This sort of confusion would be avoided if we stopped using "ISO" to mean so many related but different things.  Erik is clearly talking about reducing exposure level and giving the sensor less light, and thus when he says "high ISO", he means high Exposure Index, one of the numerous quantities defined in ISO standards.  Exposure Index, is, by the way, the main intended significance of the "ISO" setting on digital camera, through effects like adjusting the shutter speed and aperture combination chosen in auto-exposure modes and doing default raw-to-JPEG conversions on the assumption that exposure levels are lower when a higher ISO exposure index setting is used.

But some people insist on assuming that measures like the saturation-based base-ISO sensitivity, a measure of highlight headroom in the sensor or raw files, are or should be what the ISO dial on a camera specifies.  By the way, the traditional ISO measure of film speed is distinctly different from either of these two: it is a measure of how high the exposure index can go (how little exposure the film can be given) while still meeting some standard for image quality in the shadows. The closest counterparts for digital photography are the rarely-used noise based SNR10 and SNR40 standards, which are the exposure index levels at which mid-tone SNR levels are 10:1 and 40:1 respectively.


Anyway, the bottom line is that Erik's figure of about 125 photons at mid-tones when exposing at an exposure index of 6400 limits the SNR to about 11:1 no matter how low the sensor's internal noise levels are. This is just better than the SNR10 standard, traditionally described as a barely acceptable mid-tone SNR level (while 40:1 in the mid-tones is traditionally described as "excellent").

So to restate Erik's conclusion in something closer to the language of the ISO standards: for a sensor with full well capacity of 40,000 and saturation-based "base-ISO" sensitivity of 100, an exposure index of 6400 gives a mid-tone SNR of at best about 11:1, so the more generous noise-based SNR10 sensitivity is at best just slightly more than 6400.  Perhaps Phase One has chosen its maximum exposure index setting of 6400 based on this being near the SNR10 limit.

Aside: for the more cautious SNR40 mid-tone noise standard, a mid-tone signal of at least 1600 is needed, and in Erik's case of exposure at EI 100 giving about a 8000 mid-tone signal, this would give a SNR40 sensitivity of only about 500! That EI level is about where people seeking excellent IQ and very fine tonal gradations still need to be operating, and it is determined almost entirely by photon shot noise and photons counts, not by the sensor's noise floor, so the new Sony sensor will not change that situation much.

[Bart_van_der_Wolf]

Bart,

Your post has caused me to rethink some of my assumptions regarding unity gain. Roger Clark has stated "Since 1 electron (1 converted photon) is the smallest quantum that makes sense to digitize, there is little point in increasing ISO above the Unity Gain ISO". However, he does state that increasing ISO above unity gain does lead to some minor improvements quantization, since more of the full scale of the ADC can be utilized.

I previously reported my findings on unity gain for the Nikon D800e. The ISO 100 gain is about 3.24 e- /14 bit DN, and the gain is decreased by a factor of 2 for each doubling of ISO.

Hi Bill,

I find the method that Jim Kasson presented in that thread very informative. It confirms what I already established for my 1Ds3, and what a friend's D800 showed for his camera. A marginally higher ISO gain setting than unity gain seems to be the point beyond which no improvement can be expected. The D800 tapers off very slowly beyond that optimum, so it could be pushed further, but then one would lose the benefit of additional headroom for specular highlights.

The Canon CMOS sensors and the D4 CMOS show gains up to about ISO 1600 and taper off thereafter.

Using Jim's SNR based method confirmed (see attachment) my earlier finding for the sweetspot of my 1Ds3 at ISO 400 gain. Underexposure (-EV exposure compensation for camera metered exposures) plus push in post beyond that point give the best result. The D800 indeed seems to show a (very flat) peak at 800, or maybe 1600 with very little losses.

That's why I expect the Sony MF CMOS to also have an approx. ISO 1600 optimum gain, being a newer generation and with a somewhat larger sensel than the D800.

QED.

Cheers,
Bart

[bjanes]

btw is everything OK w/ him ? it seems that he was not active anywhere for > 1 year...

I hope he is OK and I just sent him an e-mail. However, there is not much new to report in the Nikon and Canon dSLR market over the last couple of years.

Bill

[bjanes]

Anyway, the bottom line is that Erik's figure of about 125 photons at mid-tones when exposing at an exposure index of 6400 limits the SNR to about 11:1 no matter how low the sensor's internal noise levels are. This is just better than the SNR10 standard, traditionally described as a barely acceptable mid-tone SNR level (while 40:1 in the mid-tones is traditionally described as "excellent").

So to restate Erik's conclusion in something closer to the language of the ISO standards: for a sensor with full well capacity of 40,000 and saturation-based "base-ISO" sensitivity of 100, an exposure index of 6400 gives a mid-tone SNR of at best about 11:1, so the more generous noise-based SNR10 sensitivity is at best just slightly more than 6400.  Perhaps Phase One has chosen its maximum exposure index setting of 6400 based on this being near the SNR10 limit.

Aside: for the more cautious SNR40 mid-tone noise standard, a mid-tone signal of at least 1600 is needed, and in Erik's case of exposure at EI 100 giving about a 8000 mid-tone signal, this would give a SNR40 sensitivity of only about 500! That EI level is about where people seeking excellent IQ and very fine tonal gradations still need to be operating, and it is determined almost entirely by photon shot noise and photons counts, not by the sensor's noise floor, so the new Sony sensor will not change that situation much.

Yes, I think that Eric's basic analysis is right on. With the newer sensors, one could increase the quantum efficiency of the sensor. However, QE is about 50% with the latest sensors and doubling this would increase the ISO at which unity gain is achieved. However electron density (e-/um^2) would still be a limiting factor. Not much is made of anti-blooming incorporated into our sensors, but this could be critical when full well is approached.

Doubling the sensor size while keeping other factors constant would give an additional stop of DR. Your SNR analysis could be updated if one accepts decreased resolution via pixel binning as a trade off for more DR.

Physical limits are being approached, and one should not expect magical results from this new CMOS crop frame medium format sensor.

Bill

[bjanes]

Hi Bill,

I find the method that Jim Kasson presented in that thread very informative. It confirms what I already established for my 1Ds3, and what a friend's D800 showed for his camera. A marginally higher ISO gain setting than unity gain seems to be the point beyond which no improvement can be expected. The D800 tapers off very slowly beyond that optimum, so it could be pushed further, but then one would lose the benefit of additional headroom for specular highlights.

Using Jim's SNR based method confirmed (see attachment) my earlier finding for the sweetspot of my 1Ds3 at ISO 400 gain. Underexposure (-EV exposure compensation for camera metered exposures) plus push in post beyond that point give the best result. The D800 indeed seems to show a (very flat) peak at 800, or maybe 1600 with very little losses.

That's why I expect the Sony MF CMOS to also have an approx. ISO 1600 optimum gain, being a newer generation and with a somewhat larger sensel than the D800.

QED.

Cheers,
Bart

Bart,

Yes, Jim's approach is very elegant, as one would expect from a brilliant electrical engineer. The more laborious approach that I used makes fewer assumptions but does fall down when read noise becomes significant, since read noise was not subtracted out.

Bill

[Fine_Art]

If you are at the point of squeezing a stone, there are the 2 traditional solutions:
Bigger lens
Bigger pixels
and one less obvious one:
Multishot frame addition.
The Sony A55 has done that well for years. Unfortunately it is only to jpg output.

There is nothing stopping people from doing their own multi-shot raw addition especially when you have the potential side benefits of super-resolution.

[jerome_m]

There is nothing stopping people from doing their own multi-shot raw addition

Subjects which move may cause undesired results.

[bjanes]

Using Jim's SNR based method confirmed (see attachment) my earlier finding for the sweetspot of my 1Ds3 at ISO 400 gain. Underexposure (-EV exposure compensation for camera metered exposures) plus push in post beyond that point give the best result. The D800 indeed seems to show a (very flat) peak at 800, or maybe 1600 with very little losses.

That's why I expect the Sony MF CMOS to also have an approx. ISO 1600 optimum gain, being a newer generation and with a somewhat larger sensel than the D800.

Bart,

I assume that unity gain is at ISO 400 for your camera. Bill Claff's tool indicates that an additional half stop of photographic range can be obtained by increasing the ISO from 400 to 1600 as shown below. Thus, unity gain does not seem to be the final arbiter in determining when raising the ISO ceases to improve DR. Bill's photographic DR has a noise floor well above 0 dB, so the difference in engineering DRs might be even higher. As I understand the situation, increasing ISO above unity gain will not improve quantization, but if the read noise continues to decrease beyond unity gain, there will be an improvement in DR as the read noise decreases. Read noise for the Canon 1Dm3 is shown in Figure 15a of Emil's treatise. The asymptote of Emil's graph agrees with Bill Claff's tool for this camera.

What do you think?

Regards,

Bill

[Jim Kasson]

There is nothing stopping people from doing their own multi-shot raw addition especially when you have the potential side benefits of super-resolution.

If by super-resolution you mean pixel-level shifting and compensation when demosaicing (a la Hasselblad), that's not a do it yourself operation for most people.

On the other hand, multi-shot averaging, even after demosaicing, can yield useful improvement, just as you said. If you want more than a factor of 3 or 4 improvement in the SNR, the number of required exposures can be daunting. As Jerome pointed out subject motion may result in unpleasant -- or possibly pleasant -- surprises.

Jim

[bclaff]

Thanks for your concern. Everything is fine.
 
I have been consumed for the past 14 months on a project at work; spending most waking hours on it.
 
I do take breaks to go to a nearby National Wildlife Refuge and have taken some vacation time including an 18 day trip to Switzerland.
But, as you have noticed; my online presence is essentially zero.
 
I expect things to improve after the initial delivery of our new product at a trade show in the first week of February.
 
Regards,
Bill