First, I fail to understand the procedure for doing a Blackframe (read noise) analysis.
Hi Mike,
No problem, I'll explain. A Blackframe is supposed to be black because it received no exposure. However, when we analyse it there is noise. The noise is produced by the camera electronics. When we take precautions to eliminate as many noise sources (e.g. thermal noise doubles for approx. each 6 degrees Celsius rise) as possible, we could assume that the remaining noise is unavoidable and linked to the action of reading out the sensor data, hence coined "Read noise".
A Blackframe is typically produced by setting the camera to it's shortest possible exposure time (to counteract thermal noise build-up), using a body cap instead of a lens (to avoid light leaks, electronic noise from the lens, and camera gain adjustments at certain apertures), and covering the eyepiece of the viewfinder (to avoid light leaking into the mirrorbox though the back).
The signal that is still recorded is the lowest signal possible and is usually random with a Gaussian distribution. It changes with the ISO (gain) setting. It is
not the same as a Darkframe, which is produced with a much longer (>1 sec. typically) exposure time, as used for Darkframe subtraction. By comparing a Darkframe and a Blackframe one can quantify the (mostly thermal) contribution.
If the Canon camera adds an offset of 1024 to all data, I would think that one needs to subtract 1024 from the data; particularly the mean since this is the significant datum in regard to read noise, and the S.D. is not of any importance.
The offset in most Canons cameras is part of the ADC quantization, so it is not added afterwards. That's why the noise has a Gaussian distribution centered at (usually) ADU 1024. There are also values below 1024 because of the Readnoise.
From what I have read (and perhaps misunderstood), in doing a black-frame (offset) analysis, one takes several images and computes the average of the means, or alternatively, the median value.
What you describe is a Darkframe (not Blackframe) noise reduction technique, commonly used in astrophotography where long exposure times are needed to collect enough photons to record faint signals. This is also why Canon cameras are often used in astrophotography, because the Readnoise improves predictably with averaging multiple frames and may reveal faint signals.
When doing a S/N analysis, I can understand that one wishes to exclude pixels outside the range of 0 to clipping level from influencing the S.D. However, since the S.D. (and the other statistics) are computed first and the offset added after the fact, the addition of an offset should not affect the S.D. It also eludes me how one recognizes the inclusion of clipped noise by examination of the stats.
You probably figured it out after the above explanation, but to make sure... When you subtract 2 noisy data sets with a mean value of e.g. 1024, then there is a 50% chance that an image has a value of 1024 or less. There is a equal chance of it being 1024 or higher. When we subtract an image with a higher data value from one with a lower data value we would get a negative number, which cannot be encoded in an integer number calculation, and thus result in a clipped noise distribution.
Therefore we add an offset to both datasets, which only changes the mean value but not the SD around that mean, and the result of the subtraction can be statistically evaluated. My choice of 1024 is not a must, one can use any number that doesn't add to the risk of integer value clipping, although it could also indicate an ADC problem. That's why I use the IRIS stat command after the subtraction, to check that there are no values that resulted in (probably clipped) zero despite the offset. If it would have a minimum of zero, then I redo the subtraction with a higher offset (for light exposure frames), but for Blackframes this is usually not needed (especially for lower ISO gain settings).
Hope that helped to understand the chosen procedure.
Cheers,
Bart