What an excellent article! And I'm thrilled to see so many people on this forum with a technical bent. Back in 2002 I found out about these concepts, and it totally changed the way I think about photography (and the way that I
do photography, as well).
My first camera with RAW capability was an Olympus E-20N. For me, this was a doorway into experimentation. I started out merely taking bias, dark, and flat frames, and writing programs to average them, subtract the bias, scale and subtract the dark, and divide by the flat. I used this to refine some heavily-stacked astrophotos. (The E-20N was a quasi-DSLR, with a beam-splitter instead of a mirror, a fixed zoom lens with a leaf shutter, and a 2/3" sensor. Needless to say it was much noiser than "true" DSLRs, and astrophotography required a whole bunch of stacked frames to beat down the noise.)
I came up with a way to plot variance (standard deviation squared) versus data level (ADU) with a huge amount of precision. The conventional technique is to measure noise locally within a region of solid color in one photo — this is not only subject to PRNU but assumes that the region doesn't have any tonal variation; and the number of data points is limited to the number of solid color swatches (or exposures at different light levels).
I decided I wanted to compare each pixel only with itself, across multiple exposures — camera on tripod, triggered remotely, photographing a scene with as much dynamic range as I could squeeze in, defocused to minimalize the effects of cumulative camera movement (ideally there'd be none of this, but even the teensiest subpixel movement threw off my data otherwise).
There was a major problem: when I took multiple exposures of the same thing, the brightness in each frame was different. I didn't know whether this resulted more from variations in my light source or in my camera's shutter lengths. It didn't matter, because I had no way of eliminating either variation.
I realized that with only one light source, and constant or negligable thermal noise, the frames differing brightnesses could be averaged together to make a reference frame that still had a monotonic tone curve. So I took this "reference frame", with much lower noise than each frame that went into it, and sorted the pixels by brightness.
The sorted list is merely a list of x,y coordinate pairs, going from darkest to lightest. The upshot of this is that each individual source frame can then be processed, and pixels grouped by brightness; simply take a subregion of the sorted list, and all the pixels at those coordinates will be very close in
average brightness (using knowledge gleaned from the full set of frames) but will follow a Poisson distribution within the one frame. So the mean and the variance provide a data point for the graph; then the program can move on to the next set of pixels adjacent in the list ("adjacent" in the list, not adjacent spatially). (Actually, for extra precision my program pools the data from all the frames, using the sorted pixel list, before doing any calculations of mean and variance; but the way in which this is done is analogous to doing it on one frame.)
Some simple algebra gave me a formula to get a graph of gain from the graph of variance. Integrating this yields a graph of linearity.
It turns out that the graph of gain shows some interesting features when mapped with great precision. Here are the graphs I came up with:
Olympus E-20N @ ISO 80Olympus E-20N @ ISO 320Notice that at ISO 80, the gain varies quite significantly, going from 7.3 to 8.6 electrons per ADU. This results in a significantly nonlinear "tone curve". When I used this tone curve in my astrophotography stacking program, the division-by-flat-frame became more accurate, and the last trace of PRNU disappeared!
Here is the astrophotography experiment where I put all this to use; I was limited to photographing the sky around Polaris, because I don't have a tracking mount (I made my stacking program rotate the frames according to timestamp). PRNU was quite prominent in this, because that region of sky is quite light-polluted at my home. (I subtracted the light pollution using a windowed median function.)
Back when I first did this experiment, I tried to find other people to do it on their own cameras (especially true DSLRs) by posting on a dpreview.com forum. I didn't get many responses, and the one person I recruited to do it on his own camera didn't follow through. I think it will be different, now, on this forum!
Three weeks ago I finally got a DSLR, and have begun to do these experiments on it:
Canon Rebel XSi (450D) @ ISO 100(I was also shocked to realize that my new DSLR has different read noises at different ISOs. I expected it to have the same read noise (in electrons) at all ISOs, just like my old Olympus E-20; I expected to be able to stay at ISO 100 always, and simply underexpose when necessary, never having to worry about highlights being blown. But I suppose that'd be too easy!)
My experiment could use lots of refinement. For one thing, the way I arrange a highly-dynamic-ranged scene is pretty much seat-of-the-pants; I use my computer room as the scene (photographed at 18mm), and move objects around to "sculpt" the histogram. What is needed is a standard way to sculpt the histogram into being utterly linear. Photographing a print doesn't come close to being good enough, because the blacks aren't black enough. I'm thinking perhaps a 3-dimensional shape could have the proper reflective characteristics, but I'm not sure how to design it.
Sources of error also have to be wrangled. There's miniscule camera movements, which I don't think can be entirely eliminated; defocusing should be done in the direction that yields soft bokeh, and of course mirror lockup is a must. Dust in the air is a source of error I have no idea how to measure, and no idea of its significance; the only way to know would be to do an experiment both in a vacuum and at normal air pressure, with everything else equal. Variations in thermal noise are an issue if dim lighting is used (i.e., long exposures); using a flash as lighting may solve this, but it may also complicate things (there may be vertical variances in lighting due to a DSLR's focal plane shutter and possibly imperfect flash sync). Note that thermal noise is fine and dandy if it stays constant; excluding hot/dead pixels, it is just another source of shot noise which is indistinguishable from photon shot noise as far as my experiment is concerned.
There's a slight snag in my calculations, at the left edge of the graph of gain. Currently I have to input an "initial gain" to get it to come out nicely, but I doubt that this initial gain is accurate. Getting deeper shadows into the photographed scene (or intentionally varying the exposure) may improve this, but it may also require modifications in my model. In the meantime, I can only approximate the read noise in electrons, by using the average gain a little ways into the graph rather than at the beginning of it.
Also, I need to implement some curve-fitting to handle the near-saturation levels. As it is now, my graph cuts off there. This will of course also be necessary for Nikon cameras, which subtract the bias level resulting in some shadow clipping (do all Nikons do this?) I was actually hoping my new Canon would not do any highlight clipping but would instead allow the sensor to saturate naturally. Sigma Foveons do this, and as a result their RAWs have some noise in the highlights. That'd be great for my experiment.
For the preliminary Canon 450D graph I've posted, I took a subset of 36 contiguous frames from a series of 136 frames to get good data; something must have changed over time, as the full set of frames had much more variance. My room was lit with incandescent lighting — the frames were 10 second exposures; the level of thermal noise may have changed over time. Also, I defocused the lens in what I now realize was the wrong direction, giving hard bokeh. Nevertheless, the subset of 36 frames appear to be giving good data.
I tried a series of flash shots (off-camera external bounce flash), and while the shape of that graph is similar, it is a bit steeper (enough to change the projected saturation point from about 32000 electrons to about 31000). I'm not sure why, but I suspect that at 1/200 second flash sync, the shutter curtains may not be entirely clear at the moment the flash fires, resulting in slight vertical variations in sensor illumination.
Despite the complications described above, this experiment works as-is, giving interesting results. If properly interpreted, the bumps in the graph of gain may provide an intimate look into the inner workings of cameras' ADC units — I'd be extremely curious to see where the bumps are on other cameras (especially comparing ADCs of differing bit depth), and whether they all have such a flat gain graph (unlike my old Olympus E-20).
Who wants to do this experiment on your own RAW-capable camera(s)? I'm quite willing to share my source code and calculations, and go into more depth in my explanation. (I hope this post isn't too long already!) Comments and questions are quite welcome. Also, I'll bet some of you may have ideas on how to improve/refine the experiment and methodology!
EDIT: Emil, I just realized that you are both the topic-starter and the writer of the article! It's awesome to have you here on the forum, and I would
really appreciate your input on my experiment.
