Pages: 1 [2] 3   Go Down

Author Topic: The useful bit depth  (Read 28400 times)

EricWHiss

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2639
    • Rolleiflex USA
The useful bit depth
« Reply #20 on: November 30, 2007, 05:09:41 pm »

Quote
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157333\"][{POST_SNAPBACK}][/a]


Maybe but you almost have to be with digital.  The price of all this stuff forces everyone to do a lot of research.  It's all still pretty new to most of us.    I shoot almost every day, but I find myself on the forums almost every day too.  Still trying to learn how to get the best images out of all this.
Logged
Rolleiflex USA

EgillBjarki

  • Guest
The useful bit depth
« Reply #21 on: November 30, 2007, 05:15:16 pm »

Quote
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157333\"][{POST_SNAPBACK}][/a]

Hey man, its all part of the game... With film there was the zone system and with digital there is the bits and sensors. Nothing wrong with getting to know your equipment as mutch as possible.
Logged

jpop

  • Newbie
  • *
  • Offline Offline
  • Posts: 20
The useful bit depth
« Reply #22 on: November 30, 2007, 06:09:35 pm »

Quote
However, your analogy doesn't work.  DR has nothing to do with absolute size, or breakdown.  It's about the ratio of maximum recordable signal, to the lowest usable signal.  The lowest usable *could*, theoretically, be limited by the number of levels, but in reality, it is mainly determined by analog noise.
[a href=\"index.php?act=findpost&pid=157237\"][{POST_SNAPBACK}][/a]


The quantity of charge which can be retained after light hits a CCD certainly has to do with it's dynamic range while I'm in absolute agreement that noise only detracts the usefulness of quantity.  A teaspoon of pepper in a 55 gallon drum filled with water has a different effect than a teaspoon of pepper in a glass.
Logged
John Popp
Dodd Camera Professional Divis

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
The useful bit depth
« Reply #23 on: November 30, 2007, 07:30:13 pm »

Quote
The full well capacity and read noise for the D200 are supplied by Roger Clark and are shown in the table below. By this definition, the D200 has 11.7 stops of DR.



However, this DR is overoptimistic for photographic purposes.[a href=\"index.php?act=findpost&pid=157253\"][{POST_SNAPBACK}][/a]

I don't think it's even correct for the traditional audio engineering definition of DR.  There is no way that the D200 has only 10 electrons read noise at its lowest ISO.  That's most likely an error on Roger's part.  The D200 ISO 100 RAW files I have seen have a read noise of 18 electrons, almost a full stop more than Roger's figure.

As far as the usefulness of DR values are concerned, obviously they must be taken in the context of their over-simplicity.  The engineering definition simply tells us how far down below saturation the image starts to rapidly break up from read noise.  It's just a reference point, from which one can offset their own standards.  More useful, for any camera at a given ISO, is a set of figures; the engineer's DR, the point at which 1:1 SNR is achieved, along with other values, such as 3:1 and 10:1.  The relative strengths of read and shot noise at various RAW levels are independent, so one camera can have more DR from saturation down to 10:1, while the other has more down to 1:1.
Logged

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
The useful bit depth
« Reply #24 on: November 30, 2007, 07:36:16 pm »

Quote
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157333\"][{POST_SNAPBACK}][/a]

If that were true, it would be perfectly fine, legal, and none of your business.
Logged

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
The useful bit depth
« Reply #25 on: December 01, 2007, 03:43:29 am »

Quote
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.

Perhaps you'll learn someday that knowing your equipment is one of the things that makes the difference between a photographer and a mere button-presser, just like intimately knowing the effects of tire pressure and wheel alignment and engine tuning and tire temperature and track surface conditions makes the difference between a winning racing team and a losing one. Knowing how to get every possible bit of performance out of your gear in challenging circumstances can often make the difference between getting a usable shot and not. Knowing the exact relationship between the camera histogram and the point at which the RAW file is clipped and where on the camera histogram the noise level becomes great enough to ruin shadow detail at a particular ISO is critical, especially when shooting in challenging lighting situations. Perhaps you think Ansel Adams was an idiot for inventing the Zone System?
Logged

rainer_v

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1194
    • http://www.tangential.de
The useful bit depth
« Reply #26 on: December 01, 2007, 04:31:30 am »

Quote
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157427\"][{POST_SNAPBACK}][/a]

and i would like to know if you think that you are one of these three giants ?
« Last Edit: December 01, 2007, 04:32:12 am by rainer_v »
Logged
rainer viertlböck
architecture photograp

Chris Livsey

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 807
The useful bit depth
« Reply #27 on: December 01, 2007, 09:13:01 am »

Quote
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157333\"][{POST_SNAPBACK}][/a]

The "only three" argument may be usefully extended to cover the number who actually understand, really understand, digital.
 
The English astronomer Sir Arthur Stanley Eddington was asked whether it was true that only three people understood Einstein’s theory of gravitation. He is supposed to have hesitated because he was trying to think who could be the third!
Logged

Snook

  • Guest
The useful bit depth
« Reply #28 on: December 01, 2007, 09:34:56 am »

Quote
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157333\"][{POST_SNAPBACK}][/a]
Looks like anthony needs to be out shootiing and working on building up his portfolio. and not in here. After he shoots a lot then he may be back in here asking tech questions..:+}
Anthony did you read the forum topic?
Snook
Logged

samuel_js

  • Guest
The useful bit depth
« Reply #29 on: December 01, 2007, 11:13:36 am »

Quote
Perhaps you'll learn someday that knowing your equipment is one of the things that makes the difference between a photographer and a mere button-presser, just like intimately knowing the effects of tire pressure and wheel alignment and engine tuning and tire temperature and track surface conditions makes the difference between a winning racing team and a losing one. Knowing how to get every possible bit of performance out of your gear in challenging circumstances can often make the difference between getting a usable shot and not. Knowing the exact relationship between the camera histogram and the point at which the RAW file is clipped and where on the camera histogram the noise level becomes great enough to ruin shadow detail at a particular ISO is critical, especially when shooting in challenging lighting situations. Perhaps you think Ansel Adams was an idiot for inventing the Zone System?
[a href=\"index.php?act=findpost&pid=157427\"][{POST_SNAPBACK}][/a]
Adams wasn't even present at his children's birth because he was out doing real photography. Do you think he spent his time at home reading graphs?

Quote
Looks like anthony needs to be out shootiing and working on building up his portfolio. and not in here. After he shoots a lot then he may be back in here asking tech questions..:+}
Anthony did you read the forum topic?
Snook
[a href=\"index.php?act=findpost&pid=157461\"][{POST_SNAPBACK}][/a]

Most photographers here don't even show their "work", some prefer to show graphs to tell us how photography should be. The majority of the crowd here will probably go throught their entire life before they can get one image in the same class as Anthony's.

You'll learn photography making photos and prints. Most of this theory thing is useless in real world photography and will never make you a better photographer.

But of course, this is an open forum and people can talk about whatever they want and all opinions should be respected.  
« Last Edit: December 01, 2007, 12:00:06 pm by samuel_js »
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
The useful bit depth
« Reply #30 on: December 01, 2007, 11:19:19 am »

Quote
I don't think it's even correct for the traditional audio engineering definition of DR.  There is no way that the D200 has only 10 electrons read noise at its lowest ISO.  That's most likely an error on Roger's part.  The D200 ISO 100 RAW files I have seen have a read noise of 18 electrons, almost a full stop more than Roger's figure.
[a href=\"index.php?act=findpost&pid=157369\"][{POST_SNAPBACK}][/a]

Roger's method of determining the read noise may not be accurate. As Emil Martinec has shown, the D3 clips the black point. My own tests with the D200 also show the black point is clipped, as shown in this Iris histogram of one of the green channels:

[attachment=4094:attachment]

Whether this is half Gaussian or some other distribution is not clear. If so, one could multiply the standard deviation by 1.66. It would be preferable to use Emil's regression method, but this would require additional data. If one multiplied Roger's read noise by 1.66 to correct for the clipping, then the read noise would be 17, more or less in agreement with your figures.

Quote
As far as the usefulness of DR values are concerned, obviously they must be taken in the context of their over-simplicity.  The engineering definition simply tells us how far down below saturation the image starts to rapidly break up from read noise.  It's just a reference point, from which one can offset their own standards.  More useful, for any camera at a given ISO, is a set of figures; the engineer's DR, the point at which 1:1 SNR is achieved, along with other values, such as 3:1 and 10:1. 
[a href=\"index.php?act=findpost&pid=157369\"][{POST_SNAPBACK}][/a]

That is the whole point of the Imatest plot, which shows noise in terms of f/stops rather than S:N. The average person would have trouble translating a S:N to what is seen visually.
« Last Edit: December 01, 2007, 11:22:14 am by bjanes »
Logged

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
The useful bit depth
« Reply #31 on: December 01, 2007, 12:33:46 pm »

Quote
Roger's method of determining the read noise may not be accurate. As Emil Martinec has shown, the D3 clips the black point. My own tests with the D200 also show the black point is clipped, as shown in this Iris histogram of one of the green channels:

[attachment=4094:attachment]

Whether this is half Gaussian or some other distribution is not clear.

(EDIT - the red text is a mistake; ignore it - I explain what I should have wrote in another reply to bjanes)

The biggest clue is when you subtract the number of positive values from the number of zero values.  The result should be equal to the number of positive values, if the RAW was really zeroed at photonic black (with a short exposure with no significant dark current noise, of course).  Here's a case where 14 bits would be convenient; the accuracy of this estimation would be better with more levels, even if the extra 2 did not contain significant usable signal.

Quote
If so, one could multiply the standard deviation by 1.66. It would be preferable to use Emil's regression method, but this would require additional data.

Well, I get my figure by subtracting shot noise from total noise in quadrature from a section of smooth, out-of-focus near-black.  For example, with a mean level of 8 ADU (with almost no zeros in the sample), I get a sigma of 2.55 ADU.  With an assumed 32K electrons at 4095, a mean of 8 ADU represents a mean of 8/4095 * 32000 =   62.5 electrons.  The sigma of 2.55 ADU represents a sigma of 2.55/4095 * 3200 = 19.9 electrons.  (19.9^2 - 62.5)^0.5 = 18.26 electrons total read noise, or 18.26/32000 * 4095 = 2.34 ADU.  10 electrons (, as Roger reports, would make the D200 the best DSLR except maybe for the K10D, for deep shadows at ISO 100.

Edit - that text in red was not meant to be in my reply; I thought I had deleted it.  10 electrons would put it right in line with most of the DSLRs with the lowest read noise at ISO 100, not better.

Reverse-engineering the effects of clipping (from the difference between the number of zeros and the number of positive values), and working with a blackframe, however might be preferable, as then you could get the noise of the entire RAW, as opposed to single color channel (considering green to be two channels, of course).  It would take some experimentation to see of this is practical.  Basically, you would have to clip different noise levels from a full gaussian curve at different points in the curve, and see if the relationship has a direct mapping.

Quote
If one multiplied Roger's read noise by 1.66 to correct for the clipping, then the read noise would be 17, more or less in agreement with your figures.
That is the whole point of the Imatest plot, which shows noise in terms of f/stops rather than S:N. The average person would have trouble translating a S:N to what is seen visually.
[a href=\"index.php?act=findpost&pid=157480\"][{POST_SNAPBACK}][/a]

Well, a full graph of SNR at all RAW levels is the most useful, especially when comparing cameras on the same graph.  Of course, white-balancing must be taken into consideration for practical use, possibly by charting the weakest channel, adjusted for its scaling factor.  SNR in tungsten light or deep desert shade is obviously going to be worse than SNR in magenta light, if they are all to be fully WB'ed.
« Last Edit: December 02, 2007, 04:44:00 pm by John Sheehy »
Logged

Eric Myrvaagnes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 22814
  • http://myrvaagnes.com
    • http://myrvaagnes.com
The useful bit depth
« Reply #32 on: December 01, 2007, 01:51:15 pm »

Quote
phase-leaf-hasselblad-sinar have 16bit and use 14bit. two bits are "unused".
[a href=\"index.php?act=findpost&pid=157176\"][{POST_SNAPBACK}][/a]
They're spare bits, in case any of the other bits get broken.  
Logged
-Eric Myrvaagnes (visit my website: http://myrvaagnes.com)

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
The useful bit depth
« Reply #33 on: December 02, 2007, 10:17:03 am »

Quote
The biggest clue is when you subtract the number of positive values from the number of zero values.  The result should be equal to the number of positive values, if the RAW was really zeroed at photonic black (with a short exposure with no significant dark current noise, of course).
[a href=\"index.php?act=findpost&pid=157497\"][{POST_SNAPBACK}][/a]

That is an interesting concept, so let us examine it. Shown below are the raw data as reported by Iris for the green2 CFA:



The data may be shown as a histogram:



The black frame should have all values at zero, but the read process introduces an error, and the resulting values can be expressed as a mean of 0 ± the error, which could be expressed in standard deviations. If the camera is clipping the noise at zero, then the negative values would be clipped to zero and their values added to the zero column of the histogram. Since the normal distribution is symmetrical, every positive bar in the histogram has a negative mirror image, which has been clipped and added to the zero bar. We can reconstruct the non-clipped data by copying each positive bar to the negative side and subtracting that amount from the zero bar.

The arithmetic is shown below. The original data are on the left. The sum of the positive non-zero data are shown at the bottom, and the result of subtracting the non-zero values from the zero values is also shown. The resulting histogram is also shown.





The normal distribution is continuous and the probability of getting any value exactly equal to zero is zero, since on a continuum there are infinite possibilities. For the zero bar, we are actually using 0±0.5. The ADC does the rounding here.

We can now examine some normal distributions generated in Excel. The mean is 0 and N = 10^6. Histograms for σ =0.5, 1, 2 and 5 are shown.









The integral 0..∞ in all cases should contain half the population, but with integers, data values less than 0.5 in the tail are clipped to zero and lost. Looking at the σ = 0.5 histogram one could construct the clipped histogram by adding the positive non-zero values to the zero bar and deleting the negative values. If we subtract the positive non-zero values from the new zero bar, we merely recover the original zero bar. The result is greater than the sum of the non-zero bars. For the σ = 5 case, the reverse is true. Examination of the other histograms is left to the reader. John's criterion does not seem to work.  

Quote
Well, I get my figure by subtracting shot noise from total noise in quadrature from a section of smooth, out-of-focus near-black.  For example, with a mean level of 8 ADU (with almost no zeros in the sample), I get a sigma of 2.55 ADU.  With an assumed 32K electrons at 4095, a mean of 8 ADU represents a mean of 8/4095 * 32000 =   62.5 electrons.  The sigma of 2.55 ADU represents a sigma of 2.55/4095 * 3200 = 19.9 electrons.  (19.9^2 - 62.5)^0.5 = 18.26 electrons total read noise, or 18.26/32000 * 4095 = 2.34 ADU.  10 electrons (, as Roger reports, would make the D200 the best DSLR except maybe for the K10D, for deep shadows at ISO 100.
[a href=\"index.php?act=findpost&pid=157497\"][{POST_SNAPBACK}][/a]

That seems reasonable, and is in agreement with the result obtained by assuming that the clipped distribution is half-Gaussian.
« Last Edit: December 02, 2007, 10:25:43 am by bjanes »
Logged

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
The useful bit depth
« Reply #34 on: December 02, 2007, 04:39:46 pm »

Quote
The integral 0..∞ in all cases should contain half the population, but with integers, data values less than 0.5 in the tail are clipped to zero and lost. Looking at the σ = 0.5 histogram one could construct the clipped histogram by adding the positive non-zero values to the zero bar and deleting the negative values. If we subtract the positive non-zero values from the new zero bar, we merely recover the original zero bar. The result is greater than the sum of the non-zero bars. For the σ = 5 case, the reverse is true. Examination of the other histograms is left to the reader. John's criterion does not seem to work.
[a href=\"index.php?act=findpost&pid=157653\"][{POST_SNAPBACK}][/a]

I looked back in my post that you replied to, and I see that I didn't express my criterion correctly.  It had both a typo and a conceptual problem (probably confusing this with something else that I had been doing), and I remember now what it was I was looking for, and it is not as exacting.

Here's the real criterion.  

In an unclipped histogram where:
T = total population
Zo = original zero population
N = negative population
P = positive population

Then: T = N+Z+P

When clipping occurs, and when N = P (blackpoint is correctly assumed), then clipping results in T = Zc + P where
Zc = zero population after clipping
and Zc = Zo+N.

One of the most obvious things to look for is that P is not greater than Zc.  If P is greater than Zc, then the black point must be clipped too low for a mid-histogram clip.  P should always be smaller than Zc.

If you mirror the individual positive populations and use them as negative populations, and subtract P from Zc, then you have a hypothetical Zh, which you can then compare to the rest of the curve, to see how well it fits.  If Zh is too high to fit the curve, then black was clipped too high.  If Zh is too low, then black was clipped too low.

Now, I remember a few months ago experimenting with ways of calculating how far off the black might be, and I remember that it helped when the noise was high, in ADUs, as the difference in population between successive values should be small, and then Zh could be seen as roughly multiples of those populations for a rough estimate of how far off the blackpoint is from zero.   You can see the difference that I am talking about between your sigma=0.5 and 5.0 histograms; if the 5.0 was available only in clipped form, and you had to estimate how may ADUs black was off from zero, the potential for error would be much less than with the 0.5 histogram.
Logged

EricWHiss

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2639
    • Rolleiflex USA
The useful bit depth
« Reply #35 on: December 02, 2007, 05:58:08 pm »

BJanes and John,
You guys are scaring me!  But let me ask you (collectively) a question.   Why can't a camera/sensor combo that uses 16 bits get a full 16 stops DR?   Seems like the most I have seen is 12 stops.  Will we have to wait until the sensor and camera manufacturers go to 32bits?

Thanks,
Eric
« Last Edit: December 02, 2007, 05:58:28 pm by EricWHiss »
Logged
Rolleiflex USA

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
The useful bit depth
« Reply #36 on: December 02, 2007, 06:08:08 pm »

Quote
BJanes and John,
You guys are scaring me!  But let me ask you (collectively) a question.   Why can't a camera/sensor combo that uses 16 bits get a full 16 stops DR?   Seems like the most I have seen is 12 stops.  Will we have to wait until the sensor and camera manufacturers go to 32bits?
[a href=\"index.php?act=findpost&pid=157726\"][{POST_SNAPBACK}][/a]

To use an audio analogy, the cameras have too much hiss.

Clever readout circuitry, such as used in Canon DSLRs and the Nikon D3, can read out a fraction of the sensor's DR (IOW< at high ISO) with less hiss, relative to absolute signal, but when reading out the entire sensor DR, at the base ISO, the hiss is stronger relative to signal.  If it weren't for this hiss, a camera could just count electrons and give an optimal signal, limited only by total sensor electron capacity.  Shadows are much, much more aesthetically pleasing when low electron counts are the only "noise", because blacks get close to really being black, so there is more contrast between near-black and blacker.  When the "hiss" or read noise is present, it clouds over the darkest areas in a blanket.
Logged

eronald

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6642
    • My gallery on Instagram
The useful bit depth
« Reply #37 on: December 02, 2007, 06:12:49 pm »

Quote
The "only three" argument may be usefully extended to cover the number who actually understand, really understand, digital.
 
The English astronomer Sir Arthur Stanley Eddington was asked whether it was true that only three people understood Einstein’s theory of gravitation. He is supposed to have hesitated because he was trying to think who could be the third!
[a href=\"index.php?act=findpost&pid=157454\"][{POST_SNAPBACK}][/a]

Actually, if I really try I guess I can follow most of the arguments here. Having an engineering degree with a VLSI design major certainly helps

That doesn't mean I would want to be doing the talking, as of now. I'll leave that to people who make the technology tick on a day to day basis.

I don't know whether this stuff is really interesting or not. My back has stripes at high ISO, I'm told they come from underexposure, I'm still not convinced. I'm starting to get interested in being able to quantify the performance of a camera so as to be able to determine objectively whether the equipment I have is as good as the next guy's.

Edmund
« Last Edit: December 02, 2007, 06:23:50 pm by eronald »
Logged
If you appreciate my blog posts help me by following on https://instagram.com/edmundronald

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
The useful bit depth
« Reply #38 on: December 02, 2007, 08:44:21 pm »

Quote
You can see the difference that I am talking about between your sigma=0.5 and 5.0 histograms; if the 5.0 was available only in clipped form, and you had to estimate how may ADUs black was off from zero, the potential for error would be much less than with the 0.5 histogram.
[a href=\"index.php?act=findpost&pid=157717\"][{POST_SNAPBACK}][/a]

Here is the clipped sigma 5 histogram. It does appear quite different.

Bill


[attachment=4104:attachment]
Logged

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
The useful bit depth
« Reply #39 on: December 03, 2007, 09:10:16 am »

Quote
I don't know whether this stuff is really interesting or not. My back has stripes at high ISO, I'm told they come from underexposure, I'm still not convinced. I'm starting to get interested in being able to quantify the performance of a camera so as to be able to determine objectively whether the equipment I have is as good as the next guy's.
[a href=\"index.php?act=findpost&pid=157731\"][{POST_SNAPBACK}][/a]

What is needed is a repository of sample files from multiple specimens of the same camera, so that comparisons can be made, so one can decide if the issue is by (passive) design or if the individual specimen is a lemon.  For MF backs, we're talking about a lot of data.  The RAWs need to converted in a repeatable, homogenous manner, or be RAW, but most people don't know how to look directly at RAW data.

A doctored version of DCRAW that looked for common problems and profiled them would be useful, too, and could keep the data to a minimum.  You'd have to feed it things like blackframes, OOF flat areas, etc.
Logged
Pages: 1 [2] 3   Go Up