Pages: 1 ... 4 5 [6] 7 8 ... 10   Go Down

Author Topic: Computing Unity Gain ISO from a single exposure  (Read 157380 times)

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Computing Unity Gain ISO from a single exposure
« Reply #100 on: March 28, 2013, 12:59:37 pm »

Jack, there aren't any photon-created electrons in the test image; the inside of the camera was as near to a photon-free zone as I could make it. I think we're just looking at electrical noise, exacerbated by turning up the gain with the ISO control. You could consider the signal to be zero, and that bucket is so full that I had to cut it off the histogram so that you could see the other buckets. I could have used the log scale for the y axis, but sometimes that's confusing to people.

Plenty of dithering power, then :)   I believe that the buckets around zero are so full because Nikon does not zero-offset the ADC, so it also records as positive the negative portion of the gaussian, skewing results around zero (see figure 11 here).  You can correct for that with some math to figure out the 'real' read noise.

So, I'm not surprised. I'll be working on some images today with real photons, and not very many of them. Maybe we'll see gaps in the D4, but I'm betting the M9 noise will prevent that.

Also, I don't think we need to have SNRs greater than one in the deep shadows (a necessary condition for gaps) for Unity Gain ISO to make sense. But let's wait on that until I figure out what's going on with the D800E and the two Sonys.

Thanks for getting me started on this.

Jim

My pleasure, looking forward to coming along for the ride.

Jack
Logged

Bart_van_der_Wolf

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 8913
Re: Computing Unity Gain ISO from a single exposure
« Reply #101 on: March 28, 2013, 01:05:29 pm »

I looked at DCRAW, and went to a couple of the websites that offered compiled code, and was put off by the extra stuff that seemed to some with it. I don't want to buy a C compiler, and I'm -- probably unjustifiably so -- scared of using a free compiler that I don't know for sure is without side effects.

Call me chicken.

Hi Jim,

No problem, BTW I like chicks ;)

I simply use the precompiled binaries that are offered on several sites (e.g. http://www.insflug.org/raw/). The benefit over MathLab and some other commercial offerings is that DCRaw is freely available for everybody. I like it when everybody can participate and expand on each others findings.

Cheers,
Bart
Logged
== If you do what you did, you'll get what you got. ==

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Computing Unity Gain ISO from a single exposure
« Reply #102 on: March 28, 2013, 01:09:41 pm »

Jack, no matter what you think about the dithering effect of noise in the system, we do agree on one thing: there's a point at which you don't improve the SNR by turning up the ISO knob. If you think that noise dithering swamps out the theoretical gain in SNR of Unity Gain ISO, then you think that point comes sooner (at a lower ISO) than you do if you buy the whole Unity Gain ISO worldview. But either way, you stop twisting the knob.

Right?

Absolutely.  My thinking is that you stop twisting it when increasing analog gain no longer gives you a benefit in SNR in the shadows or you hit the ceiling with desirable highlights.  This is how Bill Claff puts it.  Unity gain may be another way to achieve this or a similar objective, but I haven't quite understood how it would do it yet since there is no mention of noise in that equation, so I look forward to your tests and thought process. 
« Last Edit: March 28, 2013, 01:19:37 pm by Jack Hogan »
Logged

Vladimirovich

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1311
Re: Computing Unity Gain ISO from a single exposure
« Reply #103 on: March 28, 2013, 01:21:40 pm »

Thanks, Bart, but I think I'm going to use Matlab for the image processing once I get the raw image planes. Not that it will be better than a specialized image processing program, but it's a program that I know (reasonably) well. I can't create those incredibly dense APL-like constructions, so real Matlab experts probably would look down on me, but the trade-off is that anyone with C++/Java experience can make sense of my code.

I looked at DCRAW, and went to a couple of the websites that offered compiled code, and was put off by the extra stuff that seemed to some with it. I don't want to buy a C compiler, and I'm -- probably unjustifiably so -- scared of using a free compiler that I don't know for sure is without side effects.

Call me chicken.

Jim

Jim, the same authors as for RawDigger are offering dcraw refactoring (dcraw -> C++ library) Libraw and there is a command line binaries compiled for you that do what you want :

http://www.libraw.org = http://www.libraw.org/download (there you can download a zip file and extract the content with precompiled binaries - in examples folder)

see for command line parameters = http://www.libraw.org/docs/Samples-LibRaw-eng.html
« Last Edit: March 28, 2013, 01:23:22 pm by Vladimirovich »
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Computing Unity Gain ISO from a single exposure
« Reply #104 on: March 28, 2013, 03:13:05 pm »

Still doing that, but while Googling, found this link which was highly relevant to high ISO and particularly the answer by jrista addressed the subject of unity gain very well and is quite relevant to this discussion. IMHO.

http://photo.stackexchange.com/questions/29675/what-is-so-special-about-iso-1600

Hi Ted, I believe this is the paragraph that you are referring to:

Quote
Gain is the conversion ratio of electrons (e-) to digital units (DU). A camera that converts exactly one e- to one DU has "unity gain".
Most cameras achieve unity gain at some exact (but possibly non-selectable) ISO setting.
More frequently, gain is fractional, such as 5.7 e- to every DU.  For every stop increase in ISO, gain drops by the same factor.
If you have a gain of 5.7 e-/DU at ISO 100, you would have 2.85 e-/DU at ISO 200, 1.425 e-/DU at ISO 400, .7125 e-/DU at ISO 800, and 0.35625 e-/DU at ISO 1600.

As you increase ISO, you lose signal to noise ratio (S/N). A lower S/N is never really a good thing...

Alright, here is an example.  We are at base ISO in manual mode and Exposure is already as big as it can be, chosen according to dof and blur constraints.  We take a test shot, look at our (fictitious) Raw histogram and realize that the brightest highlight we would like to keep is still two stops below clipping.  Do we increase ISO?

If one were to follow that quote above blindly one would not.  And they'd be leaving IQ at the scene.  Because for a fixed exposure, when you raise ISO the input referred read noise usually goes down (except in ISOless cameras) improving the shadows' SNR and overall DR - but only up to a point.  The key is determining the ISO past which the IQ no longer improves, because if you increase it past that point then all you are doing is waste space and possibly blow more highlights.

I believe that point is determined mainly by the physical characteristics of the analog 'amplification' in the conversion chain*.  Unity Gain exponents introduce an additional element unrelated to noise and suggest that in any case one should stop raising ISO when the chain produces 1 ADU for each electron from the sensor.  I don't quite understand why that should be a limit, probably because I am missing a piece of the puzzle.

Jack
*PS Bill Claff has developed a marvellous chart http://home.comcast.net/~NikonD70/Charts/PDR_Shadow.htm#D800E,EOS 5D Mark III that shows graphically the improvement that one can expect from raising ISO in such situations for various cameras.
« Last Edit: March 28, 2013, 03:32:56 pm by Jack Hogan »
Logged

xpatUSA

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 390
    • Blog
Re: Computing Unity Gain ISO from a single exposure
« Reply #105 on: March 28, 2013, 04:00:47 pm »

Hi Ted, I believe this is the paragraph that you are referring to:

Quote
Gain is the conversion ratio of electrons (e-) to digital units (DU). A camera that converts exactly one e- to one DU has "unity gain".
Most cameras achieve unity gain at some exact (but possibly non-selectable) ISO setting.
More frequently, gain is fractional, such as 5.7 e- to every DU.  For every stop increase in ISO, gain drops by the same factor.
If you have a gain of 5.7 e-/DU at ISO 100, you would have 2.85 e-/DU at ISO 200, 1.425 e-/DU at ISO 400, .7125 e-/DU at ISO 800, and 0.35625 e-/DU at ISO 1600.

As you increase ISO, you lose signal to noise ratio (S/N). A lower S/N is never really a good thing...

Alright, here is an example.  We are at base ISO in manual mode and Exposure is already as big as [we care to set] according to dof and blur constraints.  We take a test shot, look at our (fictitious) Raw histogram and realize that the brightest highlight we would like to keep is still two stops below clipping.  Do we increase ISO?

I would say we increase ISO from base - simply because the scene is currently under-exposed by the said two stops.

Quote
If one were to follow that quote above blindly one would not.  And they'd be leaving IQ at the scene.  Because for a fixed exposure, when you raise ISO the input referred read noise usually goes down (except in ISOless cameras) improving the shadows' SNR and overall DR - but only up to a point.  The key is determining the ISO past which the IQ no longer improves, because if you increase it past that point then all you are doing is waste space and possibly blow more highlights.

Since we're talking numbers, how are you defining "IQ", SNR, DR or some combination thereof? This is a serious question, not a dig!

Quote
I believe that point is determined mainly by the physical characteristics of the analog 'amplification' in the conversion chain.

Sorry, the sentence is not very precise in engineering terms. However I do think I know what is meant and, since it encompasses the [entire] conversion chain, regretfully I can not comment.

Quote
Unity Gain exponents introduce an additional element unrelated to noise and suggest that, in any case, one should stop raising ISO when the chain produces 1 ADU for each electron from the sensor.  I don't quite understand why that should be a limit, probably because I am missing a piece of the puzzle.

Another question related to the first paragraph. Am I right or wrong in thinking that the 2 stops increase from base ISO in your example took us into Unity Gain+ territory?

A fascinating subject . . . well, for three of us anyway . . .
« Last Edit: March 28, 2013, 04:03:32 pm by xpatUSA »
Logged
best regards,

Ted

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Computing Unity Gain ISO from a single exposure
« Reply #106 on: March 28, 2013, 04:18:09 pm »

I would say we increase ISO from base - simply because the scene is currently under-exposed by the said two stops.

I am going to stop you right here: we are not underexposed because Exposure (determined solely by shutter speed and f/number) is as good as it gets and fixed: it's the biggest Exposure we could manage within our artistic constraints of dof (f/number) and blur (ss).  In other words we are not going to get any more photons/electrons/signal than this.  What we have is those photons/electrons being mapped to the Raw data so that the desirable highlights are recorded two stops below clipping.  Even so we can easily record scene information in all its glory because bit depth is only loosely linked to the amount of information recorded thanks to dithering ;).  The question is, is it the best we can do?

Jack

PS: IQ in this context is objective, quantifiable parameters like DR, SNR and blown vs retained desirable highlights.  And sure, let's assume that the two stops got you beyond unity gain. That doesn't mean you should get there :)
« Last Edit: March 28, 2013, 04:46:33 pm by Jack Hogan »
Logged

xpatUSA

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 390
    • Blog
Some Foveon results using ImageJ
« Reply #107 on: March 28, 2013, 05:05:02 pm »


ISO 100 (base) at left, ISO 1600 at right. Linear 16-bit TIFFs from dcraw, opened in ImageJ and split into R, G & B images:



Interesting that the condition: (mean = SD^2) would occur at less than 1600 ISO whereas Clark's method gives ISO 1880 for Unity Gain.

Any comment, Jim?

Logged
best regards,

Ted

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: Computing Unity Gain ISO from a single exposure
« Reply #108 on: March 28, 2013, 05:07:14 pm »

I believe that point is determined mainly by the physical characteristics of the analog 'amplification' in the conversion chain*.  Unity Gain exponents introduce an additional element unrelated to noise and suggest that in any case one should stop raising ISO when the chain produces 1 ADU for each electron from the sensor.  I don't quite understand why that should be a limit, probably because I am missing a piece of the puzzle.

Jack
*PS Bill Claff has developed a marvellous chart http://home.comcast.net/~NikonD70/Charts/PDR_Shadow.htm#D800E,EOS 5D Mark III that shows graphically the improvement that one can expect from raising ISO in such situations for various cameras.

Raising the ISO is helpful with cameras having poorly matched electronics where the read noise increases at low ISO, as Emil shows in Fig 15a and the accompanying text in his article on sensor noise and dynamic range. By raising the ISO, read noise is decreased initially and then levels off beyond a certain point. Increasing the ISO further does not decrease the read noise and only limits highlight headroom. The point of diminishing returns can be estimated by looking at the DXO plot of DR vs ISO and determining the ISO where the DR begins to drop by 1 stop for each doubling of the ISO.

Some in this thread have suggested that the unity gain is useful in determining this point of diminishing returns where increasing the ISO no longer decreases the read noise. However, unity gain has little to do with read noise. Many of the newer cameras using the Sony Exmor sensors are ISO less in that read noise does not vary with ISO and one does not need to bother with increasing the ISO on the camera, but can merely increase exposure in the raw converter. Unity gain has little utility with these cameras.

I determined unity gain for my D800e by a brute force, using Roger Clark's method for sensor analysis. For details, see Roger's post, but in brief, I took duplicate images at various ISOs of a uniform target illuminated by daylight coming in from a south facing window on a clear cloudless day where the illumination did not vary. I exposed at 1 EV over the meter reading get data where read noise and dark noise do not contribute significantly to the total noise, using a 300 mm lens at f/8 to reduce light falloff. I used ImagesPlus to isolate the 200x200 central area of the image and extract the green channel, and then subtracted duplicate images to obtain the photon noise. The number of captured photoelectrons equals the square of the signal:noise and one can determine the gain by dividing the number of electrons by the 14 bit data number. The results are shown. The unity gain can be estimated by the graph and occurs at about ISO 320.

Regards,

Bill
Logged

xpatUSA

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 390
    • Blog
Re: Computing Unity Gain ISO from a single exposure
« Reply #109 on: March 28, 2013, 05:13:12 pm »

I am going to stop you right here: we are not underexposed because Exposure (determined solely by shutter speed and f/number) is as good as it gets and fixed.

OK. Thanks for the link.
« Last Edit: March 28, 2013, 05:16:20 pm by xpatUSA »
Logged
best regards,

Ted

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Computing Unity Gain ISO from a single exposure
« Reply #110 on: March 28, 2013, 05:26:37 pm »

Some in this thread have suggested that the unity gain is useful in determining this point of diminishing returns where increasing the ISO no longer decreases the read noise. However, unity gain has little to do with read noise. Many of the newer cameras using the Sony Exmor sensors are ISO less in that read noise does not vary with ISO and one does not need to bother with increasing the ISO on the camera, but can merely increase exposure in the raw converter. Unity gain has little utility with these cameras.

Hi Bill, I agree.  In fact I'd extend your comment even to cameras that are not ISO less.  For instance, the 5DMIII has a Unity Gain of around ISO 500, but imho it would benefit greatly from raising ISO well into 2500 and above - as long as desirable highlights were not clipped.  Nevertheless I am sure that there is some merit to the unity-gain argument, and I would like to learn more to understand what that is.

I determined unity gain for my D800e by a brute force, using Roger Clark's method for sensor analysis. For details, see Roger's post, but in brief, I took duplicate images at various ISOs of a uniform target illuminated by daylight coming in from a south facing window on a clear cloudless day where the illumination did not vary. I exposed at 1 EV over the meter reading get data where read noise and dark noise do not contribute significantly to the total noise, using a 300 mm lens at f/8 to reduce light falloff. I used ImagesPlus to isolate the 200x200 central area of the image and extract the green channel, and then subtracted duplicate images to obtain the photon noise. The number of captured photoelectrons equals the square of the signal:noise and one can determine the gain by dividing the number of electrons by the 14 bit data number. The results are shown. The unity gain can be estimated by the graph and occurs at about ISO 320.

Your results agree perfectly with the analysis of full SNR data from DxO that I produced in the table in page three of this thread, where unity gain would have been attained at ISO 3.21*100=321.

Jack
« Last Edit: March 28, 2013, 05:31:26 pm by Jack Hogan »
Logged

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2370
    • The Last Word
Re: Computing Unity Gain ISO from a single exposure
« Reply #111 on: March 28, 2013, 07:07:39 pm »

Rather than use dark noise as a stimulus as in the preceding post, I have made a series of measurements of the histograms of the Nikon D4 and D800E, and the Sony RX-1 and NEX-7 when presented with a featureless surface eight or nine stops below clipping. I did not include the Leica M9 because I expect that it will perform pretty much like the D4, except with a lot more noise; if anyone would like to see that camera tested, let me know and I’ll oblige.

Each camera was measured with the camera ISO setting at all the whole stops from ISO 100 through ISO 6400. Shutter speeds were kept at 1/60 and faster to avoid any in-camera processing that might take place at slow shutter speeds. The central 200x200 pixels (10,000 pixels in each color plane) were used for the histogram. These tests have demonstrated to me that the histogram combing observed at high ISO settings in the previous post are due to two reasons:

    ADCs that, although they are specified as 14-bit devices, are not delivering 14 bits of resolution

    Digital gain applied by the camera manufacturers to the real raw data before it is written to the raw file.

The digital gain seems to be applied at very high ISOs, where there is so much noise that the l0ss in resolution of taking a  14-bit unsigned integer and multiplying it by a number less than 16 to yield another 14-bit unsigned integer probably does not adversely impact image quality.

One thing that surprised me is the Gaussian look to the noise in all cases. I had thought that the noise had longer tails than that from my dark noise tests.

Summary for the four cameras:

The Nikon D4 uses all 14 of its bits all the time except for those lost to digital white balance. There is no evidence of histogram periodicities that would indicate the elusive electron quanta, but with a sample that big, we've probably got several ADCs and several analog amplifiers involved. Details here.

The Nikon D800E is at 14-bit device with all codes present except for those lost to digital white balance until the ISO knob gets to 3200. Then the gain of the analog amplifier stops increasing and the output of the ADC is shifter one bit to the left, giving thirteen bits of resolution. There is another one bit leftward shift and concomitant loss of resolution to 12 bits at ISO 6400. Details here.

The Sony RX-1 is never a 14-bit instrument. It starts at 13 bits at ISO 100, and loses another bit in two of the channels at ISO 6400. Details here.

The Sony NEX-7 starts out a 12-bit camera at ISO 100, is 11 bits at ISO 3200, and 10 bits at ISO 6400.  Details here.

Jim



« Last Edit: March 28, 2013, 07:13:47 pm by Jim Kasson »
Logged

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2370
    • The Last Word
Re: Computing Unity Gain ISO from a single exposure
« Reply #112 on: March 28, 2013, 07:17:52 pm »

I determined unity gain for my D800e by a brute force, using Roger Clark's method for sensor analysis. For details, see Roger's post, but in brief, I took duplicate images at various ISOs of a uniform target illuminated by daylight coming in from a south facing window on a clear cloudless day where the illumination did not vary. I exposed at 1 EV over the meter reading get data where read noise and dark noise do not contribute significantly to the total noise, using a 300 mm lens at f/8 to reduce light falloff. I used ImagesPlus to isolate the 200x200 central area of the image and extract the green channel, and then subtracted duplicate images to obtain the photon noise. The number of captured photoelectrons equals the square of the signal:noise and one can determine the gain by dividing the number of electrons by the 14 bit data number. The results are shown. The unity gain can be estimated by the graph and occurs at about ISO 320.

And that's the result I got using the one-shot method that got me to start this post. That seems to be one thing we can all agree on. In the case of the two Sony's, I don't believe that they're 14-bit cameras anymore, and that means that the Unity Gain ISO for the RX-1 get's doubled, and the Unity Gain ISO for the NEX-7 gets quadrupled.

Jim
« Last Edit: March 28, 2013, 07:20:38 pm by Jim Kasson »
Logged

Bart_van_der_Wolf

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 8913
Re: Computing Unity Gain ISO from a single exposure
« Reply #113 on: March 28, 2013, 07:58:56 pm »

Nevertheless I am sure that there is some merit to the unity-gain argument, and I would like to learn more to understand what that is.


Hi Jack,

The group of people who work with unity gain most are probably those involved with Astrophotography. They are faced with two major challenges (besides light-pollution).

The first is a lack of photons, so they would like to expose longer (collect more photons) to get the faint stars to get significantly brighter than the (background) noise floor. That brings us to the second issue, exposure time must be kept as short as possible to avoid atmospheric turbulence and perhaps residual tracking errors despite their special equatorial mounts. They also don't want to overexpose the brighter stars.

So they are caught in the middle, between exposure times that must be as long as possible but short enough to avoid motion issues. That's where ISO, or rather Gain, makes a difference. They take multiple exposures to reduce the noise by (amongst others) averaging, and boost the gain to amplify the weak signals and allow shorter exposure times. However, there is not much to be gained once those weak signals are sufficiently above the (lowered) noise floor, raising the gain further will increasingly add more amplifier and thermal noise which will make them again less visible.

In other words, they have little use for the situation where it takes several converted photons to change the ADU by only a single unit because that would require an unnecessarily long exposure time and an exponential growth of the dark current. There is also not much use for too much gain that separates single photon conversions by more than a single ADU, because it only risks getting more amplifier noise and temperature which won't improve accuracy one bit (pun intended).

That's why many of them aim for a setting of approximately unity gain or slightly above, because it's as accurate as useful, yet as low noise as possible.

Cheers,
Bart
Logged
== If you do what you did, you'll get what you got. ==

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2370
    • The Last Word
Re: Some Foveon results using ImageJ
« Reply #114 on: March 28, 2013, 08:10:27 pm »

Any comment, Jim?

Ted,

This isn't working very well for you, is it?

It looks to me that your ISO 100 histograms are broader than I would expect, and also not as smoothly Gaussian, which is what I would think they'd look like if the noise were all photon noise. Are you selecting the central 200x200 (or so) of an image that has no detail at all? I start with something that's flat-looking, and defocus the heck out of the lens to make sure. Also, as Bill Janes suggested earlier, a mean of 1000 would have less of a contribution from pixel response nonuniformity.  

Your ISO 1600 histograms have too low a mean, and are probably affected a great deal by read noise. They, too, should have a mean of about 1000. Here's the thing that confuses me, though; the SDs look too low, not too high.

Neither ISO is producing numbers that make sense. If you would, could you try again with a very flat subject, like a white wall? I'd be surprised if there were something about the Foveon that kept this method from working.

Sorry,

Jim



Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2370
    • The Last Word
Re: Computing Unity Gain ISO from a single exposure
« Reply #115 on: March 28, 2013, 08:18:10 pm »

The group of people who work with unity gain most are probably those involved with Astrophotography. They are faced with two major challenges (besides light-pollution).

Bart,

Astrophotographers often cool their sensors, don't they? As the read noise level drops, Unity Gain ISO assumes greater importance. As the noise goes up and up, at some point one or two photons either way need a lot of processing to be detected. I'm trying to sort out the visual effects, but I'll have to devise some new test to see if single electron changes in the sensor well can translate to visible differences in normal photographic images. I'm learning a lot with this exploration.

Jim

xpatUSA

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 390
    • Blog
Re: Some Foveon results using ImageJ
« Reply #116 on: March 28, 2013, 08:54:44 pm »

Ted,

This isn't working very well for you, is it?

That's OK. I happened to have some X3F raw files of a Macbeth card shot for a different purpose and used those just to play with ImageJ. I selected the white patch in each case, and the mean values are what ImageJ said they were. In other words I hadn't adjusted the exposure to get the magic ~1000. I was actually on my way to delete the post anyway, so no harm done. I thought the Foveon images might be of interest with them not having two green channels to complicate matters.

I too was unable to make any sense of the relationship of the SD's to the means but, on re-examination, the distributions look skewed and there's quite a long 'tail' on the left . .

[added] X3F raw files are a trip in themselves - numbers bigger than 4095, weird meta data to do with normalizing the channels, channel saturation values that aren't, most folks just accept it and enjoy the images.

Thanks for your comments,

« Last Edit: March 28, 2013, 09:35:45 pm by xpatUSA »
Logged
best regards,

Ted

Bart_van_der_Wolf

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 8913
Re: Computing Unity Gain ISO from a single exposure
« Reply #117 on: March 28, 2013, 09:55:35 pm »

Bart,

Astrophotographers often cool their sensors, don't they?

Hi Jim,

Not necessarily/specifically, although cold nights generally do provide for lower atmospheric turbulence and they cool the setup down (and that causes condensation and battery capacity issues, but I digress). Many non-professional Astrophotographers use normal, non-adjusted DSLRs, some with only the IR filter removed from the IR+AA filter package.

Quote
As the read noise level drops, Unity Gain ISO assumes greater importance.

Absolutely. And by using things like averaging (just one of several steps), even random noise can be reduced a lot (a factor of 1/sqrt(N), where N is the number of exposures, often in the range of 16). Systematic noise (dead/hot pixels, dark current, other temporal noise, non-flatness of field) is removed by some of the other steps.

Quote
As the noise goes up and up, at some point one or two photons either way need a lot of processing to be detected. I'm trying to sort out the visual effects, but I'll have to devise some new test to see if single electron changes in the sensor well can translate to visible differences in normal photographic images. I'm learning a lot with this exploration.

While essential for Astrophotographers, the only benefit of using 'unity-gain' for regular photography is when we want to optimize the balance between noise quality AND short exposure time. Otherwise, just increase the number of Photons that get converted and the S/N ratio will offer a better image quality, or shorten the exposure time and get the shot to begin with. In the latter case, it may help to use a unity gain ISO setting, and underexpose if one can boost 'exposure' in post-processing (instead of cranking up the ISO setting).

Cheers,
Bart
Logged
== If you do what you did, you'll get what you got. ==

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Computing Unity Gain ISO from a single exposure
« Reply #118 on: March 29, 2013, 04:45:01 am »

Each camera was measured with the camera ISO setting at all the whole stops from ISO 100 through ISO 6400. Shutter speeds were kept at 1/60 and faster to avoid any in-camera processing that might take place at slow shutter speeds. The central 200x200 pixels (10,000 pixels in each color plane) were used for the histogram. These tests have demonstrated to me that the histogram combing observed at high ISO settings in the previous post are due to two reasons:

    ADCs that, although they are specified as 14-bit devices, are not delivering 14 bits of resolution

    Digital gain applied by the camera manufacturers to the real raw data before it is written to the raw file.

The digital gain seems to be applied at very high ISOs, where there is so much noise that the l0ss in resolution of taking a  14-bit unsigned integer and multiplying it by a number less than 16 to yield another 14-bit unsigned integer probably does not adversely impact image quality.

Good stuff.  I assume that  just above you are referring to demosaicing and rendering, and of course I agree.  Another source of noise that I would assume would make the gaps imperceptible even at ISO 100 (as in the case of the two Sonys) is photon noise - shot noise inherent in the light hitting our retinas.

Quote
One thing that surprised me is the Gaussian look to the noise in all cases. I had thought that the noise had longer tails than that from my dark noise tests.

Perhaps with the shorter shutter speed dark noise is not as apparent, or perhaps it's corrected for.  Any clues as to what those tiny entries next to the main codes could be in the earlier histograms?

Quote
Summary for the four cameras:

The Nikon D4 uses all 14 of its bits all the time except for those lost to digital white balance. There is no evidence of histogram periodicities that would indicate the elusive electron quanta, but with a sample that big, we've probably got several ADCs and several analog amplifiers involved. Details here.

I guess that's the typical real world situation with a CoC.  Would it be worthwhile to try to isolate an ADC/amplifier pair, perhaps with a smallish sample? I think that they are arranged in columns.  I'd be really interested to see whether we can approximately show one for one conversion.

Quote
The Nikon D800E is at 14-bit device with all codes present except for those lost to digital white balance until the ISO knob gets to 3200. Then the gain of the analog amplifier stops increasing and the output of the ADC is shifter one bit to the left, giving thirteen bits of resolution. There is another one bit leftward shift and concomitant loss of resolution to 12 bits at ISO 6400. Details here.

The Sony RX-1 is never a 14-bit instrument. It starts at 13 bits at ISO 100, and loses another bit in two of the channels at ISO 6400. Details here.

The Sony NEX-7 starts out a 12-bit camera at ISO 100, is 11 bits at ISO 3200, and 10 bits at ISO 6400.  Details here.

The NEX-7 is surpirsing to me because it came out shortly after the D7k and K5 which used on-board 14-bit ADCs for the first time, so I thought it would use the same technology.  However its Raw data reaches full scale at only around DN4000 (it is spec'd as 12 bits).  Since at ISO 100 3/4 of the codes are missing, could we say that it is instead a 10-bit device?

Thanks for all your hard work and insights, this is great!

Jack
« Last Edit: March 29, 2013, 04:46:32 am by Jack Hogan »
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Computing Unity Gain ISO from a single exposure
« Reply #119 on: March 29, 2013, 04:54:47 am »


Hi Jack,

The group of people who work with unity gain most are probably those involved with Astrophotography. They are faced with two major challenges (besides light-pollution).

The first is a lack of photons, so they would like to expose longer (collect more photons) to get the faint stars to get significantly brighter than the (background) noise floor. That brings us to the second issue, exposure time must be kept as short as possible to avoid atmospheric turbulence and perhaps residual tracking errors despite their special equatorial mounts. They also don't want to overexpose the brighter stars.

So they are caught in the middle, between exposure times that must be as long as possible but short enough to avoid motion issues. That's where ISO, or rather Gain, makes a difference. They take multiple exposures to reduce the noise by (amongst others) averaging, and boost the gain to amplify the weak signals and allow shorter exposure times. However, there is not much to be gained once those weak signals are sufficiently above the (lowered) noise floor, raising the gain further will increasingly add more amplifier and thermal noise which will make them again less visible.

In other words, they have little use for the situation where it takes several converted photons to change the ADU by only a single unit because that would require an unnecessarily long exposure time and an exponential growth of the dark current. There is also not much use for too much gain that separates single photon conversions by more than a single ADU, because it only risks getting more amplifier noise and temperature which won't improve accuracy one bit (pun intended).

That's why many of them aim for a setting of approximately unity gain or slightly above, because it's as accurate as useful, yet as low noise as possible.

Thank you Bart, that explains it.  Since they are able to reduce noise substantially through a combination of longish exposures and stacking, they lose its dithering action and therefore they are effectively in the ideal no-noise situation where one for one does indeed make sense.

Jack
« Last Edit: March 29, 2013, 04:58:26 am by Jack Hogan »
Logged
Pages: 1 ... 4 5 [6] 7 8 ... 10   Go Up