“Expose to the Right” & relation to ISO Invariance

[dwswager]

Is having to normalize the ETTR capture in a raw converter post processing and some new revelation to you? Is that what you mean by post processing? As Bernie stated, not that it's necessary if you follow the examples and text provided, this has nothing to do with JPEGs or post processing unless you feel that ETTR captures should have zero processing parameters applied to them in the raw processor. That would seem a rather silly approach.

Would you mind having that discussion with the Nikon executives?  They are so 'out of camera' centric in their approach, functionally that should be built into the camera is overlooked.

[bjanes]

As far as the term ISO-less is concerned, this is my understanding :-

Sensor Read noise can be introduced prior to the ISO amplifier or after the amplifier in the A/D converter circuitry. The noise introduced before the amp is amplified when the gain is increased as a result of an increase in ISO setting but the A/D noise is not.

With Canon sensors, it appears that at base ISO, A/D converter noise is a significant contributor to the total noise in the final digital signal. As ISO is increased, the noise introduced before the amplifier is increasingly amplified until at some point, it is the dominating contributor and the contribution from the A/D is insignificant. Once you reach this point, the sensor becomes virtually ISO-less as it makes little difference whether you increase the “exposure” digitally or in the amplifier. Before this point however, increasing gain digitally increases both sources of noise and hence is less effective than increasing ISO gain.

On the other hand, the latest Sony/Nikon sensors seem to have very little A/D noise (presumably because of the use of integrated column A/D’s.) So for these sensors, the noise introduced before the amplifier is always the dominant noise for any ISO. These cameras are therefore virtually ISO-less for the entire ISO range as it makes little difference whether the ‘exposure” is adjusted digitally or in the column amplifiers.

ETTR is really a separate issue and is more about making use of the full dynamic range of the camera. It may involve more exposure than is necessary for a given scene which in turn allows the exposure (and also the  noise) to be wound down in pp.

These principles can be demonstrated with a few calculations. The main sources of noise in a digital capture with scenes of relatively normal luminance are shot noise and read noise. As Roger Clark demonstrates in his post on determining noise and signal to noise ratios (SNR) of a digital sensor, the characteristics of the sensor can be modeled reasonably well by shot noise and read noise. At relatively normal exposures, dark current is not significant. Shot noise is the square root of the number of photons captured, and read noise for a given ISO is constant. One can determine the total noise by adding the shot noise and read noise in quadrature (one adds the squares of the noise sources and takes the square root of the sum).

The table below uses Roger’s data for the Canon 1D Mark II to calculate SNRs for various exposures with this camera. At base ISO the read noise is relatively high at 16.1 e- (electrons) and it drops to 4.04 e- at ISO 800. The full well of the sensor is 53,000 e-.  If one exposes to the right at base ISO, the brightest f/stop is represented by 53,000 e- and this number is halved for each progressively darker f/stop as shown in the left most portion of the table. The total noise is calculated by adding the shot noise and read noise in quadrature, and the signal to noise ratio (SNR) is the quotient of the number of electrons (signal) and the total noise.

Above base ISO, each doubling of the ISO halves the number of electrons at highlight clipping, so 6625 e- are collected by an ETTR exposure at ISO 800. The calculations for ISO 800 are shown in the middle portion of the table. The SNRs at all levels are superior for ETTR exposures at base ISO as compared to ISO 800. In a situation where shutter speed and aperture considerations restrain exposure to 6625 electrons, one can use ISO 800 rather than base ISO to reduce the read noise. The SNRs for this exposure are shown for ISO 800 and ISO 100. In the brighter f/stops of the exposure where shot noise is predominant, the SNR is only slightly worse at ISO 100, but in the shadows where read noise predominates, the SNR is better at ISO 800.

I don’t have data for Andrew’s 5D Mark II, but if one can extrapolate from data for the 1D Mark II, it is apparent that he would obtain the best SNRs by fully exposing at base ISO.  With reduced exposure at 1/60 sec and f/5.6, the SNRs in the highlights are not much different, but the SNRs in the deep shadows are markedly better. The mid-tones (18%) would be represented by 1192 e-, somewhere between steps 2 and 3, and the SNR is only marginally better at ISO 800. With an ISO-less sensor, the read noise would be constant and there would be little difference between ISO 100 and ISO 800 for a given exposure.

The minimal SNR that gives acceptable image quality is somewhat a personal decision. Emil Martinec has a graphic demonstrating relatively low SNRs (Figure 19 in his post). A SNR of 8 in the shadows is acceptable to me, and a SNR of 4 is marginal.

Bill

[Jack Hogan]

Good work, Bill.

To relate this to ISO invariant cameras, there would be no difference in read noise between ISO 800 and 100 for them.

Jack

[jrsforums]

These principles can be demonstrated with a few calculations. The main sources of noise in a digital capture with scenes of relatively normal luminance are shot noise and read noise. As Roger Clark demonstrates in his post on determining noise and signal to noise ratios (SNR) of a digital sensor, the characteristics of the sensor can be modeled reasonably well by shot noise and read noise. At relatively normal exposures, dark current is not significant. Shot noise is the square root of the number of photons captured, and read noise for a given ISO is constant. One can determine the total noise by adding the shot noise and read noise in quadrature (one adds the squares of the noise sources and takes the square root of the sum).

The table below uses Roger’s data for the Canon 1D Mark II to calculate SNRs for various exposures with this camera. At base ISO the read noise is relatively high at 16.1 e- (electrons) and it drops to 4.04 e- at ISO 800. The full well of the sensor is 53,000 e-.  If one exposes to the right at base ISO, the brightest f/stop is represented by 53,000 e- and this number is halved for each progressively darker f/stop as shown in the left most portion of the table. The total noise is calculated by adding the shot noise and read noise in quadrature, and the signal to noise ratio (SNR) is the quotient of the number of electrons (signal) and the total noise.

Above base ISO, each doubling of the ISO halves the number of electrons at highlight clipping, so 6625 e- are collected by an ETTR exposure at ISO 800. The calculations for ISO 800 are shown in the middle portion of the table. The SNRs at all levels are superior for ETTR exposures at base ISO as compared to ISO 800. In a situation where shutter speed and aperture considerations restrain exposure to 6625 electrons, one can use ISO 800 rather than base ISO to reduce the read noise. The SNRs for this exposure are shown for ISO 800 and ISO 100. In the brighter f/stops of the exposure where shot noise is predominant, the SNR is only slightly worse at ISO 100, but in the shadows where read noise predominates, the SNR is better at ISO 800.

I don’t have data for Andrew’s 5D Mark II, but if one can extrapolate from data for the 1D Mark II, it is apparent that he would obtain the best SNRs by fully exposing at base ISO.  With reduced exposure at 1/60 sec and f/5.6, the SNRs in the highlights are not much different, but the SNRs in the deep shadows are markedly better. The mid-tones (18%) would be represented by 1192 e-, somewhere between steps 2 and 3, and the SNR is only marginally better at ISO 800. With an ISO-less sensor, the read noise would be constant and there would be little difference between ISO 100 and ISO 800 for a given exposure.

The minimal SNR that gives acceptable image quality is somewhat a personal decision. Emil Martinec has a graphic demonstrating relatively low SNRs (Figure 19 in his post). A SNR of 8 in the shadows is acceptable to me, and a SNR of 4 is marginal.

Bill

Thanks, Bill.....

Supports what Emil was saying in his statement on Maximizing Exposure but with data to explain it.

It is important to remember that Andrew's ISO 800 exposure is only better, as you show, because he maximized exposure (ETTR).  Had he also maximized exposure at ISO 100, by lowering aperture/shutter (left section of your chart) he would have gotten the best image of the three.  ISO 800 only gives a better image than ISO 100 because on ETTR.

THANKS AGAIN....

[Dave Ellis]

Yes thanks for that Bill, very interesting.

One other minor point I would raise with these considerations is this - how closely does the clipping point of the amp or A/D converter match the full well capacity of the sensor ? Presumably it's a bit higher which means that when ISO is raised it is possible to get a bit more exposure without clipping. But I think this is probably a minor issue.

Dave

[Jack Hogan]

One other minor point I would raise with these considerations is this - how closely does the clipping point of the amp or A/D converter match the full well capacity of the sensor ? Presumably it's a bit higher which means that when ISO is raised it is possible to get a bit more exposure without clipping. But I think this is probably a minor issue.

I am not sure I understand this.  If Exposure is fixed, it is fixed independently of ISO.  In an ISOless camera that's it.  In an ISOful camera raising ISO from base may bring out the shadows while clipping the highlights stop for stop, correct?  See this example.

Jack

[Dave Ellis]

I am not sure I understand this.  If Exposure is fixed, it is fixed independently of ISO.  In an ISOless camera that's it.  In an ISOful camera raising ISO from base may bring out the shadows while clipping the highlights stop for stop, correct?  See this example.

Jack

Jack, as is often the case, I probably didn't explain what I was trying to say very well.

In the comparisons being made in this thread, the assumption is made that if a certain exposure just causes clipping at base ISO, then if the ISO is increased by say 8 times, then the exposure will have to be reduced by 8 times to still just avoid clipping. What I'm saying is that this is not necessarily quite true. If we assume that at base ISO, the onset of clipping corresponds to the sensels reaching full well capacity, then as ISO is increased this will at some point no longer be the case and the clipping point will be determined by the overload point of the amplifier or A/D converter input. I am assuming that this will occur at a slightly higher voltage than that reached at the A/D input at base ISO with the sensel at full well capacity. So if the ISO is raised by 8 times, the exposure might only have to be reduced by say 7.8 times to just reach the clipping point.

Does that make any more sense ?

Dave

[Jim Kasson]

One other minor point I would raise with these considerations is this - how closely does the clipping point of the amp or A/D converter match the full well capacity of the sensor ? Presumably it's a bit higher which means that when ISO is raised it is possible to get a bit more exposure without clipping. But I think this is probably a minor issue.

The clipping point of the ADC at base ISO is usually set somewhat below the FWC of the sensor, because the sensor usually goes a little nonlinear before it goes really nonlinear. So the situation you raised won't normally happen.

That said, when I and many others calculate the FWC from experimenting with the camera, the number that we report is that corresponding to full scale at base ISO. The real FWC is probably higher.

Jim

[Dave Ellis]

The clipping point of the ADC at base ISO is usually set somewhat below the FWC of the sensor, because the sensor usually goes a little nonlinear before it goes really nonlinear. So the situation you raised won't normally happen.

That said, when I and many others calculate the FWC from experimenting with the camera, the number that we report is that corresponding to full scale at base ISO. The real FWC is probably higher.

Jim

Ah that makes sense, thanks Jim.

I think the reason I had it in my head that the A/D clipping point was a bit higher than the FWC was that quite some time ago I found that I could get a higher maximum digital raw value out of my old Canon600D (Rebel) at ISO200 (and above) than at ISO100. However that was older technology and of course my measurements could have been flawed.

Dave

[Guillermo Luijk]

The clipping point of the ADC at base ISO is usually set somewhat below the FWC of the sensor, because the sensor usually goes a little nonlinear before it goes really nonlinear.

Jim, when the RAW saturation level is below the end of the ADC scale (2^n-1 being n the number of bits), don't you think this could mean the FWC is below the clipping point of the ADC?.

Typical Canon 5D RAW histogram, the clipping point 3692 is below the right end of the ADC theoretical scale (4095):


Regards

[Dave Ellis]

For what it's worth, some green channel clipping values I've observed on the two cameras I've owned

Canon 600D
ISO100  : 12,000
ISO200-1600 : 13,700

Nikon D610
ISO100-800 : 15,780
ISO1600 : 16,273
ISO2000-3200 : 16,383

I suspect the values for ISO above 800 for the D610 relate to the use of "digital ISO".

Dave

[bjanes]

For what it's worth, some green channel clipping values I've observed on the two cameras I've owned

Canon 600D
ISO100  : 12,000
ISO200-1600 : 13,700

Nikon D610
ISO100-800 : 15,780
ISO1600 : 16,273
ISO2000-3200 : 16,383

I suspect the values for ISO above 800 for the D610 relate to the use of "digital ISO".

FWIW, here are clipping values for the D800e at base ISO obtained by gross overexposure as shown by Rawdigger. The red and blue clip at 16383, presumably due to clipping in the ADC. The green clips at 15768 in both channels. If this is due to well saturation, I would expect some variation due to pixel response non-uniformity. Perhaps Jim or others can comment.

Bill

[Jim Kasson]

Jim, when the RAW saturation level is below the end of the ADC scale (2^n-1 being n the number of bits), don't you think this could mean the FWC is below the clipping point of the ADC?.

Typical Canon 5D RAW histogram, the clipping point 3692 is below the right end of the ADC theoretical scale (4095):


Regards

Maybe I spoke too fast. I know nothing about Canons, mainly Leicas, Nikons, and Sonys. I do note that Nikon full scale is sometimes not as high as you'd think, but there is no sign of saturation before clipping, so I think that real sensor saturation must be higher. With the Nikons, WB prescaling is part of the problem.

Jack, want to weigh in on this?

Jim

[bernie west]

Thanks, Bill.....

Supports what Emil was saying in his statement on Maximizing Exposure but with data to explain it.

It is important to remember that Andrew's ISO 800 exposure is only better, as you show, because he maximized exposure (ETTR).  Had he also maximized exposure at ISO 100, by lowering aperture/shutter (left section of your chart) he would have gotten the best image of the three.  ISO 800 only gives a better image than ISO 100 because on ETTR.

THANKS AGAIN....

This is wrong.  Andrew's images were the SAME exposure.  By boosting ISO in his non-ISOless camera he increased signal to noise ratio.  NOT exposure.

[jrsforums]

This is wrong.  Andrew's images were the SAME exposure.  By boosting ISO in his non-ISOless camera he increased signal to noise ratio.  NOT exposure.

You have defined what ETTR does, as I said in my post.  Andrew seemed to be asking why 800 ISO had lower noise than 100 ISO.  It's not, if both images are normalized to either "standard" exposure or "maximized" exposure. It only appears to be if the 100 is a standard exposure and the 800 is maximized....as shown in Bill's chart.

[bernie west]

I don't really understand what you are trying to convey.  We already knew why Andrew's image's are the way they are.  I'm just saying that you are not correct to say one was a higher exposure than the other.  They were the same exposure and the noise (actually SNR) difference is the variable that has been isolated.

[Jack Hogan]

Maybe I spoke too fast. I know nothing about Canons, mainly Leicas, Nikons, and Sonys. I do note that Nikon full scale is sometimes not as high as you'd think, but there is no sign of saturation before clipping, so I think that real sensor saturation must be higher. With the Nikons, WB prescaling is part of the problem.

Jack, want to weigh in on this?

Jim, I am also not versed in Canon.

However the Nikon numbers are typically explained by black level subtraction.  For instance most pre-2015 Nikons clipped blacks before writing ADUs into the raw data.  At 14-bits at base ISO the black level is typically around 600 ADUs, so the maximum number possible out of an unscaled 14 bit ADC becomes 16383-600 = 15783 or so, what can be seen in the green channels.  As you know the B and R channels are pre-conditioned after black level subtraction (gotta figure out why) so they can be scaled back to (or near) the top value.  The black level sometimes changes with the ISO (I think for the D610 it lowers to around 100 ADUs a couple ISO stops up).  This should explain Bill and David's numbers.

So your thinking is correct, as far as I understand Nikon cameras: the gain/bias to the ADC at base ISO is chosen so that it saturates before full well capacity is reached.  And as long as the histogram looks like it hits a brick wall I believe that is always the case, independently of camera make, including Guillermo's Canon.

Jack

PS The top level of the ADC can be an arbitrary level, chosen so as not to show the non linear part of the curve before FWC.  I believe Canon does this in several of its cameras.  The value at which the curve becomes non-linear may change based on amplification (ISO).

[digitaldog]

This is wrong.  Andrew's images were the SAME exposure.  By boosting ISO in his non-ISOless camera he increased signal to noise ratio.  NOT exposure.

Exactly! I thought this was a simple demo of just that fact.

[Bart_van_der_Wolf]

Maybe I spoke too fast. I know nothing about Canons, mainly Leicas, Nikons, and Sonys. I do note that Nikon full scale is sometimes not as high as you'd think, but there is no sign of saturation before clipping, so I think that real sensor saturation must be higher.

Hi Jim,

That's what I've understood as well, usually (always is a big word) the real sensor saturation is higher than the ADC saturation which produces the ADU or DN we find in Raw files. This is to benefit from the virtually linear portion of the Photon Transfer Curve, which makes color math a lot easier. I seem to remember a post on DPreview, not sure if it was for Eric Fossum, that the real sensor saturation can be something like 40% higher than the ADC saturation suggests. Here is a nice compilation of DxO originated ADC/Raw file Saturation data for various cameras, expressed in e- or actually converted Photons. When we divide the ADC photon saturation by the number we can find for clipped highlights in a Raw file, we'll get the conversion gain.

It also demonstrates that Andrews EOS 5D2 has a Read noise that keeps dropping as the ISO is boosted, all the way to 1600 (I think  the ISO 3200 value is a glitch), but at the same time the saturation level reduces faster, thus lowering the total DR. As I've said before, and Bill already showed in his calculation example, nothing beats real photons if one is to maximize the S/N of a capture. However, if one approaches the read noise level by underexposure (due to constraints like avoiding camera shake or subject motion) of the brightest relevant scene data, the benefit of the reduced read noise at boosted ISO settings becomes relatively important. But this is specific for ISO variant cameras, like the Canons, and not what the OP was asking about.

With the Nikons, WB prescaling is part of the problem.

It complicates understanding what the real exposure level was, so one needs to reverse engineer that from the photon statistics for the Red and Blue color planes, compared to the Green color planes.

I believe that Nikon at least dropped the read noise clipping for the D810, so that becomes easier to accurately evaluate, and maybe that also hints at the multiplication for R and B.

Cheers,
Bart

[bjanes]

Here is a nice compilation of DxO originated ADC/Raw file Saturation data for various cameras, expressed in e- or actually converted Photons. When we divide the ADC photon saturation by the number we can find for clipped highlights in a Raw file, we'll get the conversion gain.

Bart,

Thanks to the link for the Sensogren data for the 5DMII. Using that data, I redid my calculations for that camera. The read noise at base ISO for this camera is considerably higher than for the 1D Mark II that I used in my original calculations, and the read noise at ISO 800 is only slightly higher. The improvement in SNR by going to ISO 800 with constant exposure is even more marked.

Bill