D800e Read Noise with RawDigger

[bjanes]

Read noise is an important characteristic of a sensor and is often obtained simply by taking an exposure with the lens cap on and checking the standard deviation, which is the read noise since no photon noise is present. However, this method does not work with the D800e and other Nikon cameras which clip the read noise at zero, losing about a half of the normal distribution.

The D800e and many Nikon cameras have a masked pixel area where optical black has been applied so that no light reaches that part of the sensor. On the D800e this to to the extreme right of the frame and can be shown with RawDigger with zoom at maximum setting as shown below. The masked pixels start at column 7378 and extend to the right. The first 14 pixels are masked off and receive no light and can be used to get the read noise. The next two column are at saturation and the remaining pixels to the right are all at 1. This pixels are presumably for some type of calibration.



One can select the dark frame pixels using RawDigger's select by the numbers. I selected a 14 x 400 area as shown, and looked at the histogram. It is roughly bell shaped, indicating that the read noise is not clipped.





One then converts the selection to a sample and looks at the data. The standard deviations are the read noise.



One would think that the read noise would be constant, but when I looked at several frames taken at different exposures at the base ISO of 100, I noted that the read noise varies between 1.03 and 1.57 ADUs as shown in the table below, which is for the green channels only for simplicity.



Bill Claff also determines read noise by looking at the masked pixels with his proprietary program and obtains a 14 bit read noise of 1.261 ADUs for the base ISO of 100, somewhere in the midrange of my data. I am perplexed by why the read noise varies with exposure--does the camera do some type of preconditioning?

Regards,

Bill

[bns]

Depending on how much you varied the exposure, the effect you measure might be due to thermal (leakage) electrons.
In that case you would see a higher (read)noise figure at the longer exposure.

Boudewijn Swanenburg

[Bart_van_der_Wolf]

One would think that the read noise would be constant, but when I looked at several frames taken at different exposures at the base ISO of 100, I noted that the read noise varies between 1.03 and 1.57 ADUs as shown in the table below, which is for the green channels only for simplicity.

Hi Bill,

I'm not sure, but does shifting the left crop boundary from 7378 to 7379 change anything? I also wondered, since the exposure time seems to be in the 1 second region, if Nikon activates a darkframe subtraction (which reduces systematic/pattern noise, but increases random noise by a factor sqrt(2)?

Cheers,
Bart

[LKaven]

I am perplexed by why the read noise varies with exposure--does the camera do some type of preconditioning?

Do you remember a PM I sent you about 8 months ago?  Thermal noise is a very serious consideration with this camera, even at relatively high shutter speeds, and especially with live view.  At the time I was looking to you for some recommendations.  I'm looking forward to the results of your study.

[Fine_Art]

That band explains a few things for me. First of all I was wondering why a black stripe showed up when I convert a nef in images plus.

Second it explains why the frame size is larger on Sony cameras. I once tried to do a difference between the raw conversion in a standard raw converter and I Plus. It wouldn't let me as the frames were different dimensions. I had to align then crop off the edge. The difference on the Sony's is the pixels look normal (maybe I need to look closer, don't quote me on that).

Maybe its for Auto Whi-Bal and dark frame as Bart says.

I have heard a lot of griping in astro-photo groups about Nikon deleting data with their cameras. This explains it in more detail, thanks. Apparently Canon does not do this, they all like canon cameras.

[fdisilvestro]

One would think that the read noise would be constant, but when I looked at several frames taken at different exposures at the base ISO of 100, I noted that the read noise varies between 1.03 and 1.57 ADUs as shown in the table below, which is for the green channels only for simplicity.

I am perplexed by why the read noise varies with exposure--does the camera do some type of preconditioning?

I suspect that the "Hot Pixel Suppression" algorithm that is automatically applied at slow speeds (can't be disabled) might be the reason.
Attached is a sample of two dark frames from a Nikon D300 (I know, it is a different camera, and different Hot pixel suppression algorithm, but it shows that it is applied to the optical black pixels) where the only difference was the shutter speed, one at 1/5 and the other at 1/4 (Where the algorithm start to be applied). The high value of the standard deviation is due to the high ISO used, but note the difference for the image taken at 1/4, from 35 down to 28.

These frames where taken with High ISO NR Off, I tool also two frames with High ISO NR ON and the results for the data in the optical black pixels was the same.

As a side note, here there is an interesting article about micrographs of a Nikon D2H sensor, where the optical black region can be seen

Regards,
Francisco

[bjanes]

Hi Bill,

I'm not sure, but does shifting the left crop boundary from 7378 to 7379 change anything? I also wondered, since the exposure time seems to be in the 1 second region, if Nikon activates a darkframe subtraction (which reduces systematic/pattern noise, but increases random noise by a factor sqrt(2)?

Cheers,
Bart

Bart,

Thanks for your input. With the D800e dark frame subtraction can be turned off, and it was off for my tests. If it is on, dark frame subtraction is applied for exposures > 1 second.

As you suggested, I changed the crop to 7380 with a width of 10 pixels and the noise was lower. To investigate the effect of exposure times, I took some shots at 1/60 and 1/20 second as shown below. The differences in noise were still apparent.



To investigate further, I used a crop of 1 x 100 pixels going from 7378 to 7391, covering the 14 pixel band of the optical dark. As I understand the architecture of the Expeed 2 processor, it uses 12 parallel readouts and I presume the optical black columns correspond to each parallel readout. It looks like channel 1 has a higher readout noise. All channels show increased noise at higher exposure. How do you interpret these findings?



Best regards,

Bill

[bjanes]

I suspect that the "Hot Pixel Suppression" algorithm that is automatically applied at slow speeds (can't be disabled) might be the reason.
Attached is a sample of two dark frames from a Nikon D300 (I know, it is a different camera, and different Hot pixel suppression algorithm, but it shows that it is applied to the optical black pixels) where the only difference was the shutter speed, one at 1/5 and the other at 1/4 (Where the algorithm start to be applied). The high value of the standard deviation is due to the high ISO used, but note the difference for the image taken at 1/4, from 35 down to 28.

These frames where taken with High ISO NR Off, I tool also two frames with High ISO NR ON and the results for the data in the optical black pixels was the same.

As a side note, here there is an interesting article about micrographs of a Nikon D2H sensor, where the optical black region can be seen

Regards,
Francisco

Francisco,

With the D800e, dark frame subtraction can be turned off. The manual states that it is applied for exposures > 1 sec if it is turned on. See my reply to Bart. I took more exposures, keeping them no greater than 1/20 sec and observed the same results.

The micrographs of the D2H sensor are quite interesting. Thanks for the link.

Bill

[LKaven]

At 1/80th and ISO6400, thermal noise on the D800 is highly intrusive.  I am certain that long before it is intrusive, it is measurable.  I do my own dark frame subtraction for anything of this sort.  Live view exacerbates the condition significantly.  

So-called "long exposure" noise reduction should be an option full time for this camera.  I suspect that the amount of active electronics on the sensor in Exmor designs is at the root of it.

[bjanes]

At 1/80th and ISO6400, thermal noise on the D800 is highly intrusive.  I am certain that long before it is intrusive, it is measurable.  I do my own dark frame subtraction for anything of this sort.  Live view exacerbates the condition significantly. 

So-called "long exposure" noise reduction should be an option full time for this camera.  I suspect that the amount of active electronics on the sensor in Exmor designs is at the root of it.

PS -- Hmph.  :-)

Naturally, the thermal noise will be magnified at ISO 6400, but at normal exposures with base ISO of 100, it is negligible.

Here are histograms of a 5 minute exposure (top) and lens cap exposure (bottom), both at ISO 100. The statistics are in terms of 16 bit data numbers. Most of the noise at 5 min is read noise as is shown by comparison with the lens cap histogram. I'm not sure what your point is.

Regards,

Bill

[IliasG]

Francisco,

With the D800e, dark frame subtraction can be turned off. The manual states that it is applied for exposures > 1 sec if it is turned on. See my reply to Bart. I took more exposures, keeping them no greater than 1/20 sec and observed the same results.

The micrographs of the D2H sensor are quite interesting. Thanks for the link.

Bill

Francisko talks about HotPixelSuppression which is turned automatically on at >=1/4 sec exposures
http://actionphotosbymarianne.com/Tech/index.html

[IliasG]

Bart,

Thanks for your input. With the D800e dark frame subtraction can be turned off, and it was off for my tests. If it is on, dark frame subtraction is applied for exposures > 1 second.

As you suggested, I changed the crop to 7380 with a width of 10 pixels and the noise was lower. To investigate the effect of exposure times, I took some shots at 1/60 and 1/20 second as shown below. The differences in noise were still apparent.

http://bjanes.smugmug.com/Photography/Sensor-Analysis/D800e-Sensor/i-QgzDTZP/0/O/RD_TableB.png

To investigate further, I used a crop of 1 x 100 pixels going from 7378 to 7391, covering the 14 pixel band of the optical dark. As I understand the architecture of the Expeed 2 processor, it uses 12 parallel readouts and I presume the optical black columns correspond to each parallel readout. It looks like channel 1 has a higher readout noise. All channels show increased noise at higher exposure. How do you interpret these findings?

http://bjanes.smugmug.com/Photography/Sensor-Analysis/D800e-Sensor/i-4MhNTfF/0/O/Samples103_104.png

Best regards,

Bill

Bill, thanks for this investigation,

some thoughts of mine ...

I'll have to search at Dpreview forums for this, but if my memory serves me well, I think Bill Claff didn't use the 6 first optically black columns for his calculations because something goes wrong with them.

A sample  of only 100 pixels is rather smallish...
You should take care of the outliers (hot/dead pixels) which can strongly affect stdev when the sample is small.

[bjanes]

Bill, thanks for this investigation,

some thoughts of mine ...

I'll have to search at Dpreview forums for this, but if my memory serves me well, I think Bill Claff didn't use the 6 first optically black columns for his calculations because something goes wrong with them.

A sample  of only 100 pixels is rather smallish...
You should take care of the outliers (hot/dead pixels) which can strongly affect stdev when the sample is small.

Ilais,

A good point, so I repeated the sampling using the full height of the sensor. The results are similar but not identical. If you remember what Bill Claff found, please post. I will e-mail and ask him to comment on the present thread.



Regards,

Bill

[bjanes]

Francisko talks about HotPixelSuppression which is turned automatically on at >=1/4 sec exposures
http://actionphotosbymarianne.com/Tech/index.html

Ilias,

Thanks for the link to marianne's site. She is phenomenal and I will look at her program. Roger's method is laborious and one might ask why re-invent the wheel? One reason is that it is instructive to whoever takes the trouble.

Regards,

Bill

[IliasG]

Ilais,

A good point, so I repeated the sampling using the full height of the sensor. The results are similar but not identical. If you remember what Bill Claff found, please post. I will e-mail and ask him to comment on the present thread.

http://bjanes.smugmug.com/Photography/Sensor-Analysis/D800e-Sensor/i-73zWBqF/0/O/samples_103_extended.png

Regards,

Bill

It was my fault to suspect outliers since it is visible at your data that there aren't any. Although as a general rule it's good to have a big sample.

Sorry I cannot remember with confidence, there is a possibility those 6 columns were of D4 instead of D800.
As it looks now we have two pairs to exclude .. 1,2 and 9,10. It's the same with D800E samples from Imaging resource.

[bclaff]

Bill,

I'm a bit rusty on this and don't have the time to research it as fully as it deserves.
As I recall I only use two columns; the first two I believe.
Also, I do separate the channels to compensate for any differences in the different channel amplifiers.

Best Regards,
Bill