Luminous Landscape Forum

Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: Jim Kasson on April 05, 2013, 11:52:54 pm

Title: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jim Kasson on April 05, 2013, 11:52:54 pm
I woke up this morning with an idea: calibrating out pixel-response nonlinearity (PRNU). First off, what is it? It’s a kind of noise in digital camera systems. It’s not actual noise, since it’s predictable. It stems from the fact that all photosites in a given color plane of a digital camera don’t have the same sensitivity to photons. If a group of photosites on the sensor of a digital camera are all exposed to the same photon flux for the same length of time, some will read higher values than others. If you consider the systematic errors to be a noise signal, the amplitude of the PRNU signal varies directly as the number of photons hitting the photosite, and therefore PRNU is greatest in the highlight areas of an image.

My idea for calibrating it out was bog-simple: construct a map of the pixel sensitivities, invert it, and multiply the values in the raw image by the inverted map. In the dim, distant future, cameras might do that themselves, with the maps generated by the manufacturers and burned into the camera before it is shipped.  In the medium term, raw developer coders could have you make a series of calibration images, and use them to correct each image from that camera as it’s “developed”.

Before I got too carried away with the idea, I thought I’d test a couple of cameras to discover the nature of PRNU. I made 256 exposures of a defocused white wall, first with the Nikon D4, and then with the Sony NEX-7. Those are the two cameras that I have handy that span the greatest range of photosite density.

Here’s how I got to 256 exposures. I figured that the PRNU and the shot noise at full scale might be comparable. If I wanted the PRNU data to be accurate, I should do enough averaging to reduce the shot noise by about a decimal order of magnitude. The square root of 256 is 8 [No, it's 16. Oops!], so that’s pretty close. Besides, I just couldn’t stomach the idea of making 1024 exposures. I did compute a corrected PRNU by subtracting the shot noise in quadrature, but it turned out to be a small correction; 256 images were enough. Whew!

I used the base ISO (100 on both cameras), and a shutter speed of 1/30 of a second at an aperture which gave me an exposure in the green channels of the raw images which was about one stop below clipping.  I saved the still-mosaiced raw images as tif’s, and averaged all 256 image for each camera. Then I wrote a program to extract each color plane from the averaged images and compute the mean and standard deviation of the data in a 200x200 pixel (that's 10,000 pixels in each of the four color planes) central region of each color plane.

I looked at the histogram of the data and verified that the PRNU appears Gaussian:

(http://www.kasson.com/ll/PRNUdist.PNG)

I brought the statistics into Excel and computed the standard deviation of the PRNU as a percentage of the signal. Averaged over all four channels, the D4 PRNU was .29% of the signal and NEX-7 PRNU was .41%. I then calculated the shot noise, and extrapolated it to what it’s be for a nearly full scale signal. Then I calculated the ratio of the PRNU over the shot noise at full scale. For the D4 it’s 97%, and for the NEX-7 it’s 70%.

Then I got a whole lot less excited about this project. I don’t hear a lot of people complaining about noise in the highlight values of digital images. Even if the PRNU is the same as the shot noise, calibrating the PRNU out will only reduce the overall highlight noise to 70% of what it was (because shot noise and PRNU are uncorrelated, you can’t just subtract the numbers). Doesn’t seem like it’s worth the effort.

I was taught that negative results were as valuable as positive ones, so I’m posting this. Also, I could certainly have made some errors in my thinking, and I’d like anyone who sees a problem with what I’ve just written to let me know.
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jack Hogan on April 06, 2013, 05:44:40 am
Very nice, Jim, especially the confirmation that PRNU has a gaussian distribution.  For the D4 I get 0.29% at ISO 100 and 0.32% at ISO 200 from DxO's SNR curves.  I typically prefer the ISO 200 values because I am more confident about having gotten a clear view of the shot noise tangent.

It'd be interesting to know how much of PRNU is due to stuff before silicon (lenses, filters, CFA) vs the rest.  As a reference, the D5200 (sensor made by Toshiba) appears to have sightly better PRNU than the D7100 with apparently the same sensor, 0.52% vs 0.62% resp. at ISO 100, 0.58 and 0.65% at ISO 200.  The D7100 apparently does not have an OLPAF/AA altogether, suggesting - I think - that we are seeing more clearly non uniformities created by and before the AA.

The situation is reversed with the D800e and the D800, which use the same sensor made by Sony - with .28 and .50% respectively at ISO 100 - and 0.36 and 0.53% at ISO 200.  They both have a 4 dot beam splitter in the light path as an OLPAF/AA, which is however reversed out in the 'e'.

[EDIT]Since I was at it I also checked the other camera pair with/without AA filter that I have data for, the Pentax K5II/s with sensors fabricated by Sony, I believe, of roughly the same pixel pitch as the D800/e.  Here again as for the the D800/e pair the K5IIS with no AA appears to have better PRNU performance than the K5II with AA (0,32%-0,32% vs 0.38%-0.39% at ISO 100 and 200 respectively).  Sony's fabrication process is obviously more precise, creating fewer non-uniformities than Toshiba's, although Toshiba's sensels are 30% smaller by area.[\EDIT]

Why does PRNU performance increase without an AA filter in the Sony made sensors?
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: bjanes on April 06, 2013, 08:19:43 am

I brought the statistics into Excel and computed the standard deviation of the PRNU as a percentage of the signal. Averaged over all four channels, the D4 PRNU was .29% of the signal and NEX-7 PRNU was .41%. I then calculated the shot noise, and extrapolated it to what it’s be for a nearly full scale signal. Then I calculated the ratio of the PRNU over the shot noise at full scale. For the D4 it’s 97%, and for the NEX-7 it’s 70%.

Then I got a whole lot less excited about this project. I don’t hear a lot of people complaining about noise in the highlight values of digital images. Even if the PRNU is the same as the shot noise, calibrating the PRNU out will only reduce the overall highlight noise to 70% of what it was (because shot noise and PRNU are uncorrelated, you can’t just subtract the numbers). Doesn’t seem like it’s worth the effort.

I was taught that negative results were as valuable as positive ones, so I’m posting this. Also, I could certainly have made some errors in my thinking, and I’d like anyone who sees a problem with what I’ve just written to let me know.


Jim,

Nice work! I agree that it may not be worth reducing PRNU, since it is not apparent in most images. I'm no visual psychologist, but I understand that the human perceptual system responds to relative differences in luminance and 0.3% is below the threshold of about 1% predicted by the Weber-Fechner law (http://Vhttp://www.normankoren.com/digital_tonality.html). See the Normal Koren link for a brief explanation.

Since PRNU is proportional to the signal, the next step in your research would be to derive a coefficient for each pixel that would equalize the gain of each photosite. For a 30MP sensor, this would require a large table, but our computers could easily handle the computation.

Regards,

Bill
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jim Kasson on April 06, 2013, 11:11:29 am
For the D4 I get 0.29% at ISO 100 and 0.32% at ISO 200 from DxO's SNR curves.  I typically prefer the ISO 200 values because I am more confident about having gotten a clear view of the shot noise tangent.

Jack,

The way I'm doing the calculation, I'd need 512 exposures at ISO 200 to get the same accuracy as I get with 256 at ISO 100. The PRNU noise would be half as much, measured in electrons, and the shot noise would be 0.707 as much, so I'd need twice as many exposures to get the averaged shot noise down to the same percentage of the PRNU. I suppose I could make that many exposures with the D4. With the NEX-7 it would be a pain. With the M9, it would be excruciating.

Why does PRNU as as percentage of signal vary with ISO? If the effect of PRNU is entirely on the number of electrons in the well, it shouldn't. There's probably a variation in gain that affects it as well. If we can measure that we'll have a partial answer to th question "does the PRNU all take place before the quantizing to electrons or not" question from the Unity Gain ISO thread.

I wonder if there's a way to compare the accuracy of your method for getting PRNU to mine? If you can describe it to me in detail, I can try both methods on the computer camera model and compare the results to the values programed into the model.

Jim
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jim Kasson on April 06, 2013, 11:21:47 am
Since PRNU is proportional to the signal, the next step in your research would be to derive a coefficient for each pixel that would equalize the gain of each photosite. For a 30MP sensor, this would require a large table, but our computers could easily handle the computation.

Bill, that was my intent going in. In fact, I have such a table for the D4 and the NEX-7. In the case of the D4, with the PRNU and the shot noise essentially equal at full scale at ISO 100, you don't need to calibrate out the PRNU to see what the image would look like with it calibrated out. Instead, you can average a bunch of images to drive the shot noise way down, and what you're left with is the PRNU. Since both are Gaussian, eliminating the PRNU and eliminating the shot noise should produce essentially the same visual effect.

In order to see the differences between the averaged and un-averaged images, you should probably judge the images on a 10-bit display system. If eight bits is enough for you, you can see some of my averaging results here (http://blog.kasson.com/?p=2854).

Jim
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Chris Warren on April 06, 2013, 12:08:08 pm
Hi Jim,

This is nice.  I have been doing some similar things with my D40 here:
http://www.luminous-landscape.com/forum/index.php?topic=61636.msg502869#msg502869
I have found that with a FW near 28,000 e and PRNU = 0.57%, shot noise (167 e) and PRNU 'noise' (160 e) are pretty similar at max signal.  From this, I went on to take 100 flat fields (remember sqrt(256) is not 8!) and convince myself that I could eliminate the PRNU and get to shot noise limited.  This was tedious, as I split out frames in IRIS and then did the processing and averaging in ImageJ; wished there was an easier way, like an in-camera button, etc.

However, the point is whether it is worthwile to do so?  I think it is.  I went into a discussion into some of this here:
http://www.dpreview.com/forums/post/38963273
I think that if we have a camera that is capable of 100:1 SNR for a uniform area, but the detail is only 10:1, then it would be good to go for as much SNR as possible, and try to raise this to 150:1 if we can, to try and pull out the details better.  IOW, if we pay good money for a sensor that is capable of 150 or 200:1 even, when shot noise limited, then why not have this capability or option?

Chris
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jim Kasson on April 06, 2013, 12:21:36 pm
(remember sqrt(256) is not 8!)

Chris, my face is red! Corrected.

Jim
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jim Kasson on April 06, 2013, 12:29:15 pm
However, the point is whether it is worthwhile to do so?  I think it is.  I went into a discussion into some of this here:
http://www.dpreview.com/forums/post/38963273

Chris, thanks for the pointer.  Given a few days to add mosaiced sensors and CFAs to my camera simulation, I should be able to model this. I can do it quicker for a Fovean-like camera. I'll take a look at it. I'll think about a target some more, but right now I kind of like the idea of a sin(x)/x radial pattern superimposed on a grey field.  Does that make sense to you?

Jim
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jim Kasson on April 06, 2013, 12:34:36 pm
I have been doing some similar things with my D40 here:
http://www.luminous-landscape.com/forum/index.php?topic=61636.msg502869#msg502869

I like this post, Chris. One thing that you found out is that PRNU hardly varies at all with ISO. That makes sense if the causes of PRNU are all before the point where the number of electrons in the well is determined.

Jim
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jack Hogan on April 06, 2013, 12:53:08 pm
Why does PRNU as as percentage of signal vary with ISO?

Good question, Jim, I've been wondering about that myself.  I calculate PRNU from DxO Total SNR data at 100%, subtracting the shot noise component quadratically.  I estimate shot noise at 100% (and therefore FWC) by finding an area on the SNR curve unaffected by read noise or PRNU and extrapolating from it: the 100% intercept of the 3dB/octave tangent to the SNR curve is shot noise at full scale assuming that the tangent is sitting on a portion of the curve where there is just shot noise.  This area is usually around .8-5% of full scale for clean (Sony sensored) cameras at ISO 100 and as gain/ISO is raised it naturally moves towards full scale until it gets overwhelmed by read noise and it becomes unusable for our purposes - usually around ISO 6400.

(http://i.imgur.com/4sLIzwF.jpg)

It doesn't work well at low ISOs for cameras with large read noise (and/or PRNU), such as the D4 or most Canons, which may not have a shot noise only portion in the ISO 100 SNR curve from which to properly extrapolate shot noise/FWC at full scale.  The tangent is lower than it should be, so shot noise at 100% and FWC are underestimated.  Here are tangents drawn on the D4 DxOmark.com full SNR curves at ISO 100 and 800.  Note how the heavy duty read noise at 100 ISO has pushed the tangent further up the curve than in the case of the cleaner RX-1 above so that now we are starting to enter in the PRNU polluted portion of the curve.  We may never get a reading of just shot noise with the D4 at ISO 100 (On the other hand at ISO 800 PRNU is too small for an accurate reading, so in these cases the sweetspot for PRNU measurement is usually around ISO 200, sometimes 400).

(http://i.imgur.com/kOPGWtO.jpg)

It is sometimes evident when the ISO 100 curve does not have a shot noise only portion because FWC, average Absolute QE and PRNU are lower than expected compared to ISO 200 and 400.  This is what one gets by reading off tangents and other data for the D4 in the graph above, for instance:

(http://i.imgur.com/pfIYIjX.jpg)

Note FWC, QE and PRNU as outliers at ISO 100.  This suggests that the area of the ISO 100 curve where the tangent rests is polluted by other sources of noise.  By raising FWC to bring PRNU in line with ISO 200, QE starts to look right, and relative gains as calculated through Ssat and FWC fall into place.  We have achieved symmetry - but this does not say much about the accuracy of the figures.

(http://i.imgur.com/k3JBoGG.jpg)

Jack
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jim Kasson on April 06, 2013, 01:48:57 pm
I'll think about a target some more, but right now I kind of like the idea of a sin(x)/x radial pattern superimposed on a grey field. 

This?

(http://www.kasson.com/ll/sinhjpg.jpg)

Or this?

(http://www.kasson.com/ll/sinf.jpg)

Or maybe even this (produced while debugging)?

(http://www.kasson.com/ll/sinbugjpg.jpg)

Jim
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Bart_van_der_Wolf on April 06, 2013, 02:08:30 pm
Hi Jim,

A few suggestions/questions to clarify the procedure.

My idea for calibrating it out was bog-simple: construct a map of the pixel sensitivities,

Normalize to an average of 1.0 ,

Quote
invert it, and multiply the values in the raw image by the inverted map.

Quote
In the dim, distant future, cameras might do that themselves, with the maps generated by the manufacturers and burned into the camera before it is shipped.  In the medium term, raw developer coders could have you make a series of calibration images, and use them to correct each image from that camera as it’s “developed”.

This is similar to what Astrophotographers do when they remove the vignetting from an image, shoot a number of 'flats' and average them (they also subtract the Bias), normalize to a maximum of 1 (same for all channels or separate per channel), and divide the image by that, all while still in linear gamma space and preferably before demosaicing.
 
Quote
Here’s how I got to 256 exposures. I figured that the PRNU and the shot noise at full scale might be comparable. If I wanted the PRNU data to be accurate, I should do enough averaging to reduce the shot noise by about a decimal order of magnitude. The square root of 256 is 8, so that’s pretty close.

Probably a typo, the square root of 256 is 16, but maybe you intended to say something else, 8 stops? I saw Chris caught it as well.

Quote
Besides, I just couldn’t stomach the idea of making 1024 exposures. I did compute a corrected PRNU by subtracting the shot noise in quadrature, but it turned out to be a small correction; 256 images were enough. Whew!

I get the subtraction in quadrature to reduce pattern noise and other non-random effects, but what do you mean with a small correction? The subtraction gives the shot noise, then what did you do to determine the PRNU, subtract the sum of the earlier averaged signal result and the averaged shot noise from the individual files?

Quote
I looked at the histogram of the data and verified that the PRNU appears Gaussian:

(http://www.kasson.com/ll/PRNUdist.PNG)

Just to be sure, this is the patternnoise residual (see above)?

Quote
I brought the statistics into Excel and computed the standard deviation of the PRNU as a percentage of the signal. Averaged over all four channels, the D4 PRNU was .29% of the signal and NEX-7 PRNU was .41%. I then calculated the shot noise, and extrapolated it to what it’s be for a nearly full scale signal. Then I calculated the ratio of the PRNU over the shot noise at full scale. For the D4 it’s 97%, and for the NEX-7 it’s 70%.

Is that 97% and 70% PNRU? Seems high ...

Quote
I was taught that negative results were as valuable as positive ones, so I’m posting this.

Yes, much appreciated. The rejection of a hypothesis also gives valuable information.

Cheers,
Bart
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Fine_Art on April 06, 2013, 10:09:00 pm
Its news to me that this is not normalized as a QC step before the camera leaves the factory. I know they go though a process for removing hot/stuck pixels which is burned into the pixel map. I imagined this as test shots against a screen that should produce a set number in the RAW values. I assumed there would be tests for values of R,G,B high, medium, low, to ensure the cameras are outputting what they should.

The fact you are getting this variability means their QC tests are something not covering this. Anyone know what kinds of tests are done before a camera ships?
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: MichaelEzra on April 07, 2013, 05:29:44 am
You could try this technique by using raw Flat Field correction in RawTherapee by setting the blur radius to 0.
If you place all your flat fields in the same directory, point to it in the preferences, RawTherapee will average flat fields automatically, matching by camera manufacturer, camera model, lens, foal length, aperture and then apply the flat field correction.
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Bart_van_der_Wolf on April 07, 2013, 06:50:38 am
You could try this technique by using raw Flat Field correction in RawTherapee by setting the blur radius to 0.
If you place all your flat fields in the same directory, point to it in the preferences, RawTherapee will average flat fields automatically, matching by camera manufacturer, camera model, lens, foal length, aperture and then apply the flat field correction.

Hi Michael,

That RawTherapee will average flat fields automatically is new to me, and that averaging is essential to avoid noise amplification with a small or zero blur radius. AFAIK it isn't mentioned in the Manual.

According to the manual:
Quote from: RawTherapee manual
Blur Radius
Blur radius controls the degree of blurring of the flat field data. Default value
of 32 is usually sufficient to get rid of localised variations of raw data due to
noise.  Setting the  Blur  Radius  to  0 skips  blurring process  and allows  to
correct for dust and other debris on the sensor (as long as their position has
not changed) at the expense of carrying noise from the flat field file into the
corrected image. If such correction is desired, it is advisable to create flat field
files with minimum amount of noise at the lowest ISO setting and optimal light
exposure.

Could you confirm the averaging behavior, and does it apply to Dark frames as well? That would be great news!

Cheers,
Bart
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: MichaelEzra on April 07, 2013, 07:07:07 am
Hi Bart, yes, both flat field and dark frame use averaging if more than a single matching frame is found.

See line 107
http://code.google.com/p/rawtherapee/source/browse/rtengine/ffmanager.cc

See line 110
http://code.google.com/p/rawtherapee/source/browse/rtengine/dfmanager.cc
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Bart_van_der_Wolf on April 07, 2013, 07:33:13 am
Hi Bart, yes, both flat field and dark frame use averaging if more than a single matching frame is found.

Michael, that's marvelous. How is the match determined?

The remark in the program code says "averaging of flatfields if more than one is found matching the same key". What's the trigger/key? I may find it by studying on the program code, but perhaps you can beat me to it with a further explanation.

Cheers,
Bart
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: MichaelEzra on April 07, 2013, 07:48:54 am
For flat fields    (ffInfo::key):  camera manufacturer, camera model, lens, focal length, aperture

  The search for the best match is twofold:
  if perfect matches by key are found, then the list is scanned for lesser distance in time
  otherwise if no match is found, the whole list is searched for lesser distance in lens and aperture


For dark frames (dfInfo::key): camera manufacturer, camera model, ISO, shutter speed

  The search for the best match is twofold:
  if perfect matches by key are found, then the list is scanned for lesser distance in time
  otherwise if no match is found, the whole list is searched for lesser distance in iso and shutter
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Bart_van_der_Wolf on April 07, 2013, 08:47:58 am
For flat fields    (ffInfo::key):  camera manufacturer, camera model, lens, focal length, aperture

  The search for the best match is twofold:
  if perfect matches by key are found, then the list is scanned for lesser distance in time
  otherwise if no match is found, the whole list is searched for lesser distance in lens and aperture


For dark frames (dfInfo::key): camera manufacturer, camera model, ISO, shutter speed, aperture

  The search for the best match is twofold:
  if perfect matches by key are found, then the list is scanned for lesser distance in time
  otherwise if no match is found, the whole list is searched for lesser distance in iso and shutter

Hi Michael,

Thanks for the clarification of this seemingly undocumented (I didn't notice it in the manual) feature. It's brilliant, and as it should be implemented (EXIF driven with a cost function to automatically pick the most appropriate inputs). Kudos to the developers.

RawTherapee was already one of the top Raw converters in my book, this feature (averaging multiple Flats/Darks) can make a real difference for those who want to get the most out of their source files.

Cheers,
Bart
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: MichaelEzra on April 07, 2013, 09:06:16 am
Hi Bart, I just added this info to the "Auto matching logic" sections of Dark Frame and Flat Field sections of the online manual. Thanks for pointing that it was missing (its nice to see that manual is being read! BTW, RT manual went through comprehensive updates recently).

Please note that I made a correction to "Key for dark frames" in the post above, it does not include aperture.
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Bart_van_der_Wolf on April 07, 2013, 10:02:40 am
Hi Bart, I just added this info to the "Auto matching logic" sections of Dark Frame and Flat Field sections of the online manual. Thanks for pointing that it was missing (its nice to see that manual is being read! BTW, RT manual went through comprehensive updates recently).

Thanks, I'm the one who downloaded it, and read it ;)

Quote
Please note that I made a correction to "Key for dark frames" in the post above, it does not include aperture.

Noted, aperture shouldn't make any difference for a dark frame.

Cheers,
Bart
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jim Kasson on April 07, 2013, 12:17:06 pm
I get the subtraction in quadrature to reduce pattern noise and other non-random effects, but what do you mean with a small correction? The subtraction gives the shot noise, then what did you do to determine the PRNU, subtract the sum of the earlier averaged signal result and the averaged shot noise from the individual files?

Here's an example, from one of the green channels of the NEX-7. The standard deviation of the averaged image was 7.95 ADUs. The computed shot noise, which I got be taking the gain at ISO 100, using it to figure out the average number of electrons in the well, taking the square root of that, multiplying by the gain to get back to ADUs, and dividing by sqrt(256) (calculated by Excel, not by me; we've already shown that taking the square root of 256 is something I can't reliably do) to take into account the averaging. All that resulted in a shot noise number of 1.03 ADUs. Subtracting that in quadrature from 7.95 gave a corrected PRNU standard deviation of 7.88 ADUs. Thus, the correction was 0.07 ADUs, what I was calling a small correction.


Just to be sure, this is the patternnoise residual (see above)?

The graph is actually a graph of the histogram of one plane of the raw averaged image. I made no attempt to compensate for the residual shot noise left after the 256 image averaging operation.

Is that 97% and 70% PNRU? Seems high ...

Those numbers are the ratios of the PRNU to the shot noise of a single exposure at full scale. So, for the D4, the PRNU and the shot noise are about equal at full scale. For the NEX-7, the PRNU is somewhat less than the shot noise at full scale.

Thanks,

Jim

Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Chris Warren on April 07, 2013, 12:45:37 pm
Hi Jim,

No problem!  Bright side is you have a more accurate flat field to work with with 256!  Looks like you  have some targets to work with for trying out, which makes sense to me.  One of the references that I referred to show a pretty cool before and after about how the author (Janesick) cleaned up a sine wave image by removing PRNU; for real world pictures one would have to experiment.  I'll have to try some more with this as well; it looks like nice news about the functionality of RawTherapee, which I would like to try as well.

Some side notes.  The flat-fielding process relies upon sensor linearity to work, which I think/hope is the case for our sensors.  I got pretty good linearity with the D40.  Also, flat-fielding can be used to do photon transfer measurements (measuring k, read noise, FW, PRNU) on 1 frame of uniform data, instead of the 2-frame differencing method.  Both methods negate the effects of PRNU, and both can be done for comparison.  Also, I would think that tables, as Bill notes, would have to be built up, which are going to be different vs F# if we also want to negate vignetting, but processing power being what it is I wouldn't imagine it being too bad.  Pretty exciting.

Chris
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jim Kasson on April 08, 2013, 02:28:14 pm
Chris, I did the testing with a target like the one you were talking about. I made three images. The first one was of the target itself, a 64-bit floating point image converted to a 16-bit, 2.2 gamma, unsigned integer TIFF RGB file. The second one was a simulated exposure in a Nikon D4 (but one with a Fovean-like sensor -- no Bayer CFA) at ISO 100 near full scale. I used a D65 source white balance, and corrected it afterwards by multiplying the red and green planes by the appropriate white balance coefficients. The third image was like the second, but the simulated D4 had no PRNU at all.

The images look virtually identical to me.

Showing 8-bit JPEGs wouldn't help, so here's a link to a 16-bit psd file with three layers, one for each image (http://www.kasson.com/ll/PRNUTestD4.psd).

Jim
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: IliasG on April 08, 2013, 08:12:28 pm
Hi Jim,

No problem!  Bright side is you have a more accurate flat field to work with with 256!  Looks like you  have some targets to work with for trying out, which makes sense to me.  One of the references that I referred to show a pretty cool before and after about how the author (Janesick) cleaned up a sine wave image by removing PRNU; for real world pictures one would have to experiment.  I'll have to try some more with this as well; it looks like nice news about the functionality of RawTherapee, which I would like to try as well.

Some side notes.  The flat-fielding process relies upon sensor linearity to work, which I think/hope is the case for our sensors.  I got pretty good linearity with the D40.  Also, flat-fielding can be used to do photon transfer measurements (measuring k, read noise, FW, PRNU) on 1 frame of uniform data, instead of the 2-frame differencing method.  Both methods negate the effects of PRNU, and both can be done for comparison.  Also, I would think that tables, as Bill notes, would have to be built up, which are going to be different vs F# if we also want to negate vignetting, but processing power being what it is I wouldn't imagine it being too bad.  Pretty exciting.

Chris

Hi Chris,

Can you upload a picture of Janesic's before-after cleanup ?. Was his procedure on Bayer mosaiced data ?.
I ask because I am thinking of a possibility that removing all pattern noise could help the demosaicing algo to make better decisions regarding interpolation directions, so we could take a better - sharper - more detailed result.

A comment on this by an expert on demosaic algorithms (Emil ??) would be nice .. before I (we) start taking thousands of flat field shots just to investingate ... 
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: MichaelEzra on April 09, 2013, 04:07:10 pm
Since we are talking about pixel-level flattening (blur radius=0), I'd like to mention that it is reasonable to expect that consecutive flat frames will be displaced by more than 1 pixel even if camera is on the tripod. The illumination hitting the sensor has to be extremely uniform to alleviate this issue and avoid having to align the flat fields before their averaging.

since this topic is about the sensor-related non-uniformity only, I suppose, that dark frame correction with averaging of the dark frames would be more appropriate here. There is also no alignment issue with dark frames.
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Chris Warren on April 13, 2013, 12:25:58 pm
Hi Jim,

Thanks for doing this!  I thought we'd see some differences but I guess not.  I looked at data for the D4, and between Dxomark and Sensorgen, I see that it's FW capability is 117,813 e, and so with PRNU removed we could anticipate an SNR = 343:1 at max signal, and without the PRNU removal (I estimate PRNU = 0.31 %), the SNR = 234:1.  Do those parameters sound about right to you?  Maybe these are some really high SNRs whose effects can't be seen, even with lower contrast targets?  I'll have to mull this over some more and maybe try to see if I can do something with my camera.

Chris
Title: Re: Calibrating out Pixel Response Nonuniformity, or not?
Post by: Jim Kasson on April 13, 2013, 03:15:44 pm
I looked at data for the D4, and between Dxomark and Sensorgen, I see that it's FW capability is 117,813 e, and so with PRNU removed we could anticipate an SNR = 343:1 at max signal, and without the PRNU removal (I estimate PRNU = 0.31 %), the SNR = 234:1.  Do those parameters sound about right to you? 

Yes. Here are the numbers I used in the simulation, all gotten from my testing:

fullWellCount = 111404
pixelNUStd = 0.003
D65RedCorrection = 280.53/540.95
D65BlueCorrection = 326.108/540.95
D65RedWBCoef = 1 / D65RedCorrection
D65BlueWBCoef = 1 / D65BlueCorrection

The last four numbers come into play because, at D65, which was the white point I used for the simulation, the different color planes have different average values, which are compensated for during the raw development to get the white point of the final image back to D65. The effect of that is to reduce the average PRNU from what it would be the way I think you did it, which is assuming that all three (or four, depending on how you count) have the same average value.


Maybe these are some really high SNRs whose effects can't be seen, even with lower contrast targets? 

At least not by me.

Jim