Pages: 1 ... 7 8 [9]   Go Down

Author Topic: ETTR vs ISO  (Read 71376 times)

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Optical Black RN versus Curve Fitted RN
« Reply #160 on: May 30, 2013, 05:39:37 am »

By the way, if you use dark-field exposures to calculate read noise, you get values that are quite different from these, probably because of the D800E's truncation of negative values. I understand that Canon has a built-in offset to keep this from happening.

Jim

Jim,

Those read noises were from Bill Claff's data. He took the read noise from the optical black region whose data are not truncated.

Yes I have always noticed and wondered about that myself.  Why would the optical black standard deviation values be quite a bit better than those we compute using standard curve fitting?  Perhaps they are totally unprocessed - or differently processed?  Which is more representative for our purposes?
Logged

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2370
    • The Last Word
Re: Optical Black RN versus Curve Fitted RN
« Reply #161 on: May 30, 2013, 10:38:52 am »

Why would the optical black standard deviation values be quite a bit better than those we compute using standard curve fitting?  Perhaps they are totally unprocessed - or differently processed?  Which is more representative for our purposes?

If the optical black values are simply offset, they should be the most direct way to measure read noise. The sample size is limited, though. I'm assuming that those sensels are representative; that's the whole point, but it's not guarenteed.

Jim

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: Optical Black RN versus Curve Fitted RN
« Reply #162 on: May 30, 2013, 11:20:02 am »

Yes I have always noticed and wondered about that myself.  Why would the optical black standard deviation values be quite a bit better than those we compute using standard curve fitting?  Perhaps they are totally unprocessed - or differently processed?  Which is more representative for our purposes?

If the optical black values are simply offset, they should be the most direct way to measure read noise. The sample size is limited, though. I'm assuming that those sensels are representative; that's the whole point, but it's not guaranteed.

Jack & Jim,

To address these concerns, I repeated my analysis with Rawdigger, selecting the entire optical black area for analysis.

This frame shows the entire raw image with the optical black on the right selected by the numbers.


Here is the raw histogram. 17K pixels are included for each channel, with 34K green values. The bell curve is skewed to the left. I don't know what this means.


Converting the selection to a sample gives the following values. The SDs for the green channels representing the read noise in ADUs is larger than before, with a mean of 1.52 ADUs.


My data from a previous analysis using ImagesPlus and Roger Clark's methodology are shown here. At low exposures, the graph departs from linearity as shown, due to truncation of the read noise. The minimum raw values include zero.


Taking the linear portion of the data and plotting the variance of the noise against the ADU values gives the following with the gain and read noises shown. The results are actually slightly better for read noise than the Rawdigger analysis of the optical black area.


Your expert comments are welcome.

Correction June 9, 2013, see post below.

Bill



« Last Edit: June 09, 2013, 09:51:24 am by bjanes »
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Optical Black RN versus Curve Fitted RN
« Reply #163 on: May 30, 2013, 05:56:18 pm »


Here is the raw histogram. 17K pixels are included for each channel, with 34K green values. The bell curve is skewed to the left. I don't know what this means.

If I read your heading correctly and the Exposure time is 2 seconds I guessing that could be the result of dark current/shot noise.

My data from a previous analysis using ImagesPlus and Roger Clark's methodology are shown here. At low exposures, the graph departs from linearity as shown, due to truncation of the read noise. The minimum raw values include zero.

Taking the linear portion of the data and plotting the variance of the noise against the ADU values gives the following with the gain and read noises shown. The results are actually slightly better for read noise than the Rawdigger analysis of the optical black area.


Your expert comments are welcome.

I assume you did not take two identical images and subracted them in order to get rid of PRNU which is proportional to signal, and that's probably why your curve above is not linear as the signal increases.  If so you may want to limit the curve fitting to data in the 0.5% to 5% range (that would be about ADU 80-800).

Jack
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: Optical Black RN versus Curve Fitted RN
« Reply #164 on: May 30, 2013, 07:06:48 pm »

If I read your heading correctly and the Exposure time is 2 seconds I guessing that could be the result of dark current/shot noise.

Jack, that is a good pickup. I repeated the optical black with an exposure of 1/125 sec and the read noise matched the long method even better. Since this is from the optical black area, there is no shot noise as no light hit the sensels.

I assume you did not take two identical images and subracted them in order to get rid of PRNU which is proportional to signal, and that's probably why your curve above is not linear as the signal increases.  If so you may want to limit the curve fitting to data in the 0.5% to 5% range (that would be about ADU 80-800).

No, I took duplicate frames and subtracted them as per Roger's method. IHMO the data are about as linear as one could expect.

Bill
« Last Edit: May 30, 2013, 07:09:47 pm by bjanes »
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Optical Black RN versus Curve Fitted RN
« Reply #165 on: May 31, 2013, 02:31:04 am »

Jack, that is a good pickup. I repeated the optical black with an exposure of 1/125 sec and the read noise matched the long method even better.

What about the shape?

Quote
Since this is from the optical black area, there is no shot noise as no light hit the sensels.

Yes, although I was referring to the black current (it comes with its own shot noise :-)

Quote
No, I took duplicate frames and subtracted them as per Roger's method. IHMO the data are about as linear as one could expect.

Bill

Yes, it looks like very good data, I am impressed.  I wonder why the slope does not change randomly, instead dicreasing monotonically as the signal increases.  Signal level has very little influence on igain k(e-/ADU) when read noise is negligible with respect to shot noise.  What would the slope be in the 0.5-5% range?

Jack
Logged

Bart_van_der_Wolf

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 8913
Re: Optical Black RN versus Curve Fitted RN
« Reply #166 on: May 31, 2013, 03:07:59 am »

Jack, that is a good pickup. I repeated the optical black with an exposure of 1/125 sec and the read noise matched the long method even better. Since this is from the optical black area, there is no shot noise as no light hit the sensels.

Hi Bill,

That's why I prefer to use the shortest possible exposure time the camera allows, and make sure that the viewfinder is blocked as well, specifically for read noise analysis. I also prefer to use a body-cap instead of a lens and lenscap, to avoid any possible influence from the lens electronics (motor noise and wide aperture gain boost).

For camera's like those from Canon that record with the Raw data read-noise offset, that allows to use the effective image area for sampling the read noise, and for e.g. Nikon cameras that clip the read noise in the effective image area, that still allows to use the masked pixel area without significant influence from e.g. dark current, even to the edge (a normal exposure can spill some light (!) to the first or second sensels directly adjacent to the effective image sensels).

Quote
No, I took duplicate frames and subtracted them as per Roger's method. IHMO the data are about as linear as one could expect.

It's the preferred method with real sensors. That will remove yet another unknown before it can pollute the dataset and possibly lead to wrong assumptions.

Cheers,
Bart
Logged
== If you do what you did, you'll get what you got. ==

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: Optical Black RN versus Curve Fitted RN
« Reply #167 on: May 31, 2013, 11:19:27 am »

That's why I prefer to use the shortest possible exposure time the camera allows, and make sure that the viewfinder is blocked as well, specifically for read noise analysis. I also prefer to use a body-cap instead of a lens and lenscap, to avoid any possible influence from the lens electronics (motor noise and wide aperture gain boost).

For camera's like those from Canon that record with the Raw data read-noise offset, that allows to use the effective image area for sampling the read noise, and for e.g. Nikon cameras that clip the read noise in the effective image area, that still allows to use the masked pixel area without significant influence from e.g. dark current, even to the edge (a normal exposure can spill some light (!) to the first or second sensels directly adjacent to the effective image sensels).

Bart,

Good suggestions, as usual for you. To investigate the effect of possible light spillage or blooming into the masked pixels, I took exposures at 1 second and 30 seconds of daylight sky at ISO 100 and examined the columns one by one going from the left to right of the masked areas.

The masked area is shown here.


Data for 1 second exposure


Data for 30 sec exposure


No spillage into the masked pixels is apparent. The longer exposure does show some evidence of dark current (thermal noise), and the columns show heterogeneity for reasons that are not apparent to me.

Turning on long exposure noise reduction produces a different appearance for the masked pixels as shown.


Examination of the data demonstrates removal of thermal noise as expected, but the residual noise is even less than the read noise, and the bright pair of columns just to the right of the masked pixels is no longer present.


My conclusion is that use of the masked pixels is a good method to determine read noise if one keeps the exposures less than 1 second or so, avoiding excessive thermal noise that contaminates the read noise. The result is the same as the more laborious regression method plotting the variance of noise against the ADU numbers and performing linear regression.

Regards,

Bill
Logged

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2370
    • The Last Word
Re: Optical Black RN versus Curve Fitted RN
« Reply #168 on: May 31, 2013, 11:33:53 am »

My conclusion is that use of the masked pixels is a good method to determine read noise if one keeps the exposures less than 1 second or so, avoiding excessive thermal noise that contaminates the read noise. The result is the same as the more laborious regression method plotting the variance of noise against the ADU numbers and performing linear regression.

Good work, Bill.

Jim

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Optical Black RN versus Curve Fitted RN
« Reply #169 on: June 01, 2013, 01:40:54 pm »

My conclusion is that use of the masked pixels is a good method to determine read noise if one keeps the exposures less than 1 second or so, avoiding excessive thermal noise that contaminates the read noise. The result is the same as the more laborious regression method plotting the variance of noise against the ADU numbers and performing linear regression.

Regards,

Bill


Yes, excellent work as always Bill.  There is something that doesn't sit right with me in your ImagePlus graph, though:



Isn't this graph in DN, and therefore so would be the square root of the y intercept (read noise) at 4.13 DN?  Converting it to e- at a k of 3.46 e-/DN, the read noise would be 14.3 e-, which doesn't jibe with other data we have on the 'e'.   On the other hand, if one takes the values in your table and concentrates in the .5-5% range (or 84-838DN) one minimizes any residual influence of PRNU and RN in the curve, getting a better overall fit (R^2=0.9999).  These values are all referred to a 14 bit DN:



Gain (K) is 3.24 and read noise is 3.2e-, just short of 1 DN.  These values appear to me to be more accurate.  What do you think?

Jack
[EDIT] I've updated the table with a little more information.  Notes:

1. Shot Noise is calculated subtracting the newly found Read Noise from the combined Shot+Read Noise in quadrature all the way down.
2. The PRNU column (above the Shot Noise Dominated Box, where Read noise is immaterial, 1 vs 60^2) calculates PRNU subtracting the combined Shot+Read noise from Total Noise in quadrature.
3. Below the Box (where PRNU is much smaller than Shot and Read Noise) Read Noise in DN is calculated by using fitted gain k to determine shot noise from the square root of the signal in DN times k, then subracting it from the combined Shot+Read noise in quadrature and re-converting it to DN.  The average value for Read Noise so calculated is 1.08 DNs or 3.5 e-.  We are still an order of magnitude away from SNR=1,  RN is too small a component of Shot+Read, and Shot Noise and Read+Shot Noise are too close, so any small measurement variation results in quite a large variance of results with this method.  PRNU also skews the estimates, even if marginally.  I normally would not use these levels for this purpose, but it gives us an indication.
4. The gain column K is simply the ratio of Signal to Shot Noise in 1 squared.
« Last Edit: June 01, 2013, 05:49:50 pm by Jack Hogan »
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: ETTR vs ISO
« Reply #170 on: June 01, 2013, 03:50:56 pm »

Jack,

Yes, I think you are correct about the read noise intercept. A major error on my part. I am traveling now and don't have access to my data. Will check when I get home.

Thanks,

Bill
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: ETTR vs ISO
« Reply #171 on: June 01, 2013, 06:03:40 pm »

Jack,

Yes, I think you are correct about the read noise intercept.

You know, thinking about it, the Variance/Signal plot seems to me to be an excellent and accurate way of determining gain k (as long as precautions are used to stay away from either end of the curve), but imho the y intercept method of determining Read Noise is very sensitive to changes in the slope: With k in hand, I think I would prefer to use the subtraction of Signal-determined shot noise in quadrature from the combined Shot+Read noise around signals of less than 5 DN.  I wonder if zero blocking might influence the statistics there, and if so whether that could be compensated for with a little math.

Jack
« Last Edit: June 01, 2013, 06:06:13 pm by Jack Hogan »
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: ETTR vs ISO
« Reply #172 on: June 05, 2013, 10:59:32 am »

You know, thinking about it, the Variance/Signal plot seems to me to be an excellent and accurate way of determining gain k (as long as precautions are used to stay away from either end of the curve), but imho the y intercept method of determining Read Noise is very sensitive to changes in the slope: With k in hand, I think I would prefer to use the subtraction of Signal-determined shot noise in quadrature from the combined Shot+Read noise around signals of less than 5 DN.  I wonder if zero blocking might influence the statistics there, and if so whether that could be compensated for with a little math.

Jack,

You are correct that the slope of the regression line is critical in determining read noise. Looking at my data, I see that there is non-linearity in the plot of variance vs data number at higher exposures. Since the regression uses least squares, the high DNs have a disproportionate influence on the regression line. Using the range of DNs from 2654 to 13, I get more reasonable data that agrees with the Rawdigger determined read noise and also with Bill Claff's data. When selecting the lower range for the DNs, it is important to exclude those where the read noise is clipped.

I'm not certain as how to apply your method. The variance of the subtracted duplicate images includes shot noise and read noise adding in quadrature. To determine the shot noise by subtracting the read noise in quadrature, one needs the read noise. Re-reading Emil's post on noise, I finally see how the regression line is determined:

 Thus, read and photon shot noise contribute as:

 N2 = R2 + S/g
, where N is the total noise, R is the read noise, S is the signal, and g is the gain.

Regards,

Bill
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: ETTR vs ISO
« Reply #173 on: June 05, 2013, 11:21:11 am »

Yes. Exposure settings for aperture and shutter are identical, the only difference is the ISO setting. And the differences in the amount of noise is significant as one would expect with ETTR (optimal exposure for raw).

I don't care about the Histogram! But what I see is a vast difference in noise whereby the ONLY setting change is ISO. So it's obviously affecting the degree of noise.

You saying increasing the ISO in this case is to reduce the read noise, this isn't also ETTR? Seems the number of photons collected should be the same since what he's calling exposure is fixed in both examples. The results however are clear in terms of the differences in noise and I understand that not all camera sensors respond as this Canon does.

Roger Clark has a nice post discussing ISO, exposure, and ETTR that clears up much confusion. He regards exposure in the usual sense as the number of photons collected. If one varies the ISO setting of the camera in manual mode and uses the same shutter speed and aperture, exposure is not affected. However, if one is in auto mode, changing the ISO does affect the exposure, since the shutter speed and/or the aperture are affected. He discusses ETTR for Canon cameras and concludes:

"ISO is a relative gain, varying by camera and has nothing to do with sensitivity or true exposure. ISO is simply a post sensor gain and digitization range."


True exposure is discussed in an add on post.

Regards,

Bill
« Last Edit: June 05, 2013, 11:27:48 am by bjanes »
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: ETTR vs ISO
« Reply #174 on: June 09, 2013, 05:44:17 am »

Looking at my data, I see that there is non-linearity in the plot of variance vs data number at higher exposures. Since the regression uses least squares, the high DNs have a disproportionate influence on the regression line.

Hi Bill,

Yes, and if you look closely, you will see that there is also a non-linearity at the lowest exposures in the Read+Shot noise variance plotted, making the slope deviate from the ideal.  Hence the suggestion to not use points below about 0.5% of full scale lest the slope of the fitted line be polluted and bent downwards, resulting in wrong values for k and RN.  I believe the ideal plot needs to be of shot noise variance only versus mean signal in order to determine k.  When plotting the same data in the form of SNR vs DN [noise in this case being Read+Shot] excluding data below 0.5% of FS and above 5% (or tighter), the slope of the resulting line needs to be -3dB/decade.  If it isn't, it is polluted.

Quote
I'm not certain as how to apply your method. The variance of the subtracted duplicate images includes shot noise and read noise adding in quadrature. To determine the shot noise by subtracting the read noise in quadrature, one needs the read noise.

Yes, and we have both (RN from the curve fitting variables), so we can simply calculate shot noise by:

Shot Noise = sqrt[[read+shot)^2 - RN^2]

Cheers,
Jack
« Last Edit: June 09, 2013, 05:59:36 am by Jack Hogan »
Logged
Pages: 1 ... 7 8 [9]   Go Up