Luminous Landscape Forum
Equipment & Techniques => Medium Format / Film / Digital Backs – and Large Sensor Photography => Topic started by: Panopeeper on November 29, 2007, 09:01:49 pm
-
There is a new topc on the Digital Cameras, Backs and Shooting Techniques forum about 16-bit DSLRs. Someone stated, that even MF cameras don't utilize the 16 bit range.
Has someone conducted a thorough test to determine, if the full bit depth of the camera is in fact useful?
Alternatively, does anyone have images with very high dynamic range and the full usage of the bit depth, suitable for a pixel peeping analysis?
Such a demo image should contain very fine, well defined structure in the bright as well as dark areas and exposed to the right ( and then +4EV :-). I found silk flowers particularly suitable for such analysis.
-
Here's a crazy idea, let's go out and do some actual photography instead
-
There is a new topc on the Digital Cameras, Backs and Shooting Techniques forum about 16-bit DSLRs. Someone stated, that even MF cameras don't utilize the 16 bit range.
Has someone conducted a thorough test to determine, if the full bit depth of the camera is in fact useful?
Alternatively, does anyone have images with very high dynamic range and the full usage of the bit depth, suitable for a pixel peeping analysis?
Such a demo image should contain very fine, well defined structure in the bright as well as dark areas and exposed to the right ( and then +4EV :-). I found silk flowers particularly suitable for such analysis.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=157135\")
Lacking test data, why not just look at the published specs from Kodak and Dalsa, who make the chips used in these backs. They claim 70-72 db or 12 f/stops. The Nikon D3 can do around 11.6 stops in tests by [a href=\"http://forums.dpreview.com/forums/read.asp?forum=1021&message=25627719]EJ Martin[/url].
He quite properly notes, "Beware that other criterion for evaluating dynamic range are used by some. For instance, dpreview and imaging-resource photograph step charts and analyze the result after running the image through raw conversion. While perhaps more applicable to the end user's aims, this approach convolves the properties of the camera with those of the raw conversion software, and so you have to ask how much of the result is attributable to the camera and how much to the raw conversion software being used. To my mind, direct measurement of the raw data without raw conversion (which is what I did) gives an unambiguous measure of the camera's capabilities."
-
They claim 70-72 db or 12 f/stops
This says nothing about the actually used bit depth. If we apply the "1% law", then 12 stops could utilize even 17 bits (ignoring the question,how one would use the result).
-
This says nothing about the actually used bit depth. If we apply the "1% law", then 12 stops could utilize even 17 bits (ignoring the question,how one would use the result).
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=157147\")
To encode a DR of 12 stops with integer gamma 1 encoding only takes 12 bits. That would leave only 1 level in the darkest f/stop. You could use a 17 bit ADC, and then 12th f/stop would contain 32 levels, well short of your goal. However, the dynamic range is determined by the noise floor in most situations, rather then by running out of levels. As you state, the result might not be usable because of noise.
With linear encoding, you can not get a 1% error across the entire range, since the relative error is not constant, but increases at the lower end of the scale. For a constant 1% error, you would need to use log encoding. See this [a href=\"http://www.anyhere.com/gward/hdrenc/hdr_encodings.html]post[/url].
For example, an error of 1 pixel value would have minimal significance in the highlights where the average pixel value might be 250 and the relative error 0.4%. In the shadows at a pixel value of 10, it would be 10%.
-
With linear encoding, you can not get a 1% error across the entire range
With the "1% law" I was referring to the claim regarding the minimum difference between lightnesses in order to be perceivable.
-
Here's a crazy idea, let's go out and do some actual photography instead
[a href=\"index.php?act=findpost&pid=157141\"][{POST_SNAPBACK}][/a]
HEAVENS NO!!!!
-
phase-leaf-hasselblad-sinar have 16bit and use 14bit. two bits are "unused".
-
Here's another wild idea. How about people that are interested in dynamic range and bit-depth go out and pick up a book to learn something about the subject before discussing it on a message board. Once they discover that analog dynamic range and digital bit-depth are two completely separate things (one being a measurement of data and the other a resolution scale applied to collected data) they will understand why you want 16-bits of resolution in a 12-stop dynamic range image.
Simply put–you get more tonal levels in each f-stop of image data. This becomes increasingly important as you move down the brightness scale and the number of levels become increasingly smaller in the shadow areas. The "unused" bits of data are the ones that you don't want to use because they have the fewest number of levels and is where banding occurs.
-
HEAVENS NO!!!!
[a href=\"index.php?act=findpost&pid=157168\"][{POST_SNAPBACK}][/a]
-
Simply put–you get more tonal levels in each f-stop of image data. This becomes increasingly important as you move down the brightness scale and the number of levels become increasingly smaller in the shadow areas. The "unused" bits of data are the ones that you don't want to use because they have the fewest number of levels and is where banding occurs.
[a href=\"index.php?act=findpost&pid=157191\"][{POST_SNAPBACK}][/a]
Perhaps even a little simpler is that Dynamic Range is how big your Pizza is and Bit Depth is simply how many slices you cut it into. Cutting a 6" pizza in half and again cutting the half in half and so on continually leaves a smaller and smaller slice. A larger pizza (Dynamic Range) will leave you with larger slices through the same process and something that might fill your belly up.
-
Here's another wild idea. How about people that are interested in dynamic range and bit-depth go out and pick up a book to learn something about the subject before discussing it on a message board. Once they discover that analog dynamic range and digital bit-depth are two completely separate things (one being a measurement of data and the other a resolution scale applied to collected data) they will understand why you want 16-bits of resolution in a 12-stop dynamic range image.
Simply put–you get more tonal levels in each f-stop of image data. This becomes increasingly important as you move down the brightness scale and the number of levels become increasingly smaller in the shadow areas. The "unused" bits of data are the ones that you don't want to use because they have the fewest number of levels and is where banding occurs.
[a href=\"index.php?act=findpost&pid=157191\"][{POST_SNAPBACK}][/a]
There is no countour banding occuring in any digital camera's RAW data, if it is at least 12 bits. There is far too much noise for bit depth to be a significant limitation. All countour banding or posterization you see comes from conversion routines and limitations of the display.
There really is no real world place for your theory to apply, except in computer-generated graphics.
-
Perhaps even a little simpler is that Dynamic Range is how big your Pizza is and Bit Depth is simply how many slices you cut it into. Cutting a 6" pizza in half and again cutting the half in half and so on continually leaves a smaller and smaller slice. A larger pizza (Dynamic Range) will leave you with larger slices through the same process and something that might fill your belly up.
[a href=\"index.php?act=findpost&pid=157219\"][{POST_SNAPBACK}][/a]
You have succeeded in making me hungry.
However, your analogy doesn't work. DR has nothing to do with absolute size, or breakdown. It's about the ratio of maximum recordable signal, to the lowest usable signal. The lowest usable *could*, theoretically, be limited by the number of levels, but in reality, it is mainly determined by analog noise.
-
Here's another wild idea. How about people that are interested in dynamic range and bit-depth go out and pick up a book to learn something about the subject before discussing it on a message board. Once they discover that analog dynamic range and digital bit-depth are two completely separate things (one being a measurement of data and the other a resolution scale applied to collected data) they will understand why you want 16-bits of resolution in a 12-stop dynamic range image.
Simply put–you get more tonal levels in each f-stop of image data. This becomes increasingly important as you move down the brightness scale and the number of levels become increasingly smaller in the shadow areas. The "unused" bits of data are the ones that you don't want to use because they have the fewest number of levels and is where banding occurs.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=157191\")
An excellent post. Analog DR and bit depth are two separate things, but they are related in that to record a given DR, a minimum bit depth is needed. The number of levels required in the darkest f/stop of a given DR is also determined by bit depth. [a href=\"http://www.normankoren.com/digital_tonality.html]Norman Koren[/url] discusses these concepts on his web site. Since the eye is less sensitive to levels in the darkest f/stop, he suggests that 8 levels are need there to prevent banding.
This chart illustrates these concepts for 12, 14, and 16 bit images with a linear tone curve. For a 12 bit image the maximum DR is 12 stops, but there is only one level in the darkest f/stop and the data here would not be usable. If 8 levels are required in the darkest f/stop for a usable image, then the DR is 9 stops max. The f/stops where there are insufficient levels are shown in red in my chart. Usable DR is related to bit depth, but the bit depth will be of no avail if noise obscured any visible detail in the shadows.
(http://bjanes.smugmug.com/photos/227100439-XL.gif)
However, the engineering definition of DR is full well capacity/read noise, usually expressed in electrons, but since sensors are linear, one could also use ADU (analog to digital units, i.e. raw data numbers).
An example may help to clarify things. I will use the Nikon D200 for which I have data, but the same considerations would apply to medium format digital cameras for which the chip manufacturers supply data.
The full well capacity and read noise for the D200 are supplied by Roger Clark (http://www.clarkvision.com/imagedetail/evaluation-nikon-d200/index.html) and are shown in the table below. By this definition, the D200 has 11.7 stops of DR.
(http://bjanes.smugmug.com/photos/227100438-M.gif)
However, this DR is overoptimistic for photographic purposes. First of all, the DR of the image convolves the DR of the camera and that of the raw converter, and some DR is lost during the conversion. Secondly, the photographic DR is determined by how much noise you can tolerate in the shadows. One can photograph a Stouffer step wedge and determine DR with Imatest Stepchart (http://www.imatest.com/docs/tour_q13.html). The results are shown below.
(http://bjanes.smugmug.com/photos/227100443-O.gif)
The camera can detect 11.6 stops of DR, which is consistent with the engineering definition. However, for a high quality photographic image the effective DR is only 6.51 stops.
Data for the KAF 39000 chip used in some high end medium format backs are published by Kodak (http://www.kodak.com/US/en/dpq/site/SENSORS/name/KAF-39000_product/show/KAF-39000_productSpecifications).
Full well is 60K electrons and read noise is 16 electrons, giving a DR of 71.4 db or 11.9 f/stops, not that much different from the D200. The DR for excellent quality photographic images would be less. I have not seen actual test results, but from a consideration of theory, I think the DR advantages of medium format may be overblown. This does not take the noise spectrum into account. With a 39K pixel image, the noise will be at a higher frequency and less objectionable to the eye than with a 10K pixel image, where the noise pattern will be coarser. In the end, one must confirm the image quality with actual photographic tests.
Bill
-
... By this definition, the D200 has 11.7 stops of DR.
However, this DR is overoptimistic for photographic purposes.
Full well is 60K electrons and read noise is 16 electrons, giving a DR of 71.4 db or 11.9 f/stops, not that much different from the D200.
Bill
[a href=\"index.php?act=findpost&pid=157253\"][{POST_SNAPBACK}][/a]
Hi Bill,
Definitely lot's of information, but the point to drive home is the 'useful' DR is rarely the max theoretical amount and often more like half that amount. This is where MFDB excels over DSLR since more of the max DR is usable. Perhaps up to 3-4 stops more in fact.
I have a stoufer transmission step wedge and Imatest, and have tested my canon 5D, and friends 1DIII, Leica DMR. The leica has a 1.5 stop real useable advantage over the canon 1DIII and perhaps 2 over the 5D. My phase p20 which I haven't tested but can see from side by side images taken in studio must have about a 1.5 stop advantage over the Leica which means perhaps 3 usable stops over the Canon 1DIII (when both cameras are set to base ISO).
Eric
-
Hi Bill,
Definitely lot's of information, but the point to drive home is the 'useful' DR is rarely the max theoretical amount and often more like half that amount. This is where MFDB excels over DSLR since more of the max DR is usable. Perhaps up to 3-4 stops more in fact.
I have a stoufer transmission step wedge and Imatest, and have tested my canon 5D, and friends 1DIII, Leica DMR. The leica has a 1.5 stop real useable advantage over the canon 1DIII and perhaps 2 over the 5D. My phase p20 which I haven't tested but can see from side by side images taken in studio must have about a 1.5 stop advantage over the Leica which means perhaps 3 usable stops over the Canon 1DIII (when both cameras are set to base ISO).
Eric
[a href=\"index.php?act=findpost&pid=157285\"][{POST_SNAPBACK}][/a]
and what about the tests where are over 11 stops measured for the canon 1d3 + 1ds3 ?
-
The "unused" bits of data are the ones that you don't want to use because they have the fewest number of levels and is where banding occurs
There are no "unused bits" but unused levels. For example the Phase One P45+ fills all bits if exposed properly, i.e. it creates 65535 levels.
Of course most of those levels will be mapped on the same value.
Re banding: I have yet to see a (raw) image, which shows banding due to the lack of levels.
Example: the Canon 1DMkII (12bit) often showed banding in the sky, and there were hopes that the 1DMkII with 14bit will solve this. I found, that the banding was not due to the lack of levels. The MkIII too shows banding.
-
and what about the tests where are over 11 stops measured for the canon 1d3 + 1ds3 ?
[a href=\"index.php?act=findpost&pid=157286\"][{POST_SNAPBACK}][/a]
It's a real number but not so useful for real life photography. It's kind of like saying that a 10 mega pix camera is equivalent to say a 10 mega pix Leica M8. Sure the amount of pixels are the same, but that's where the comparison ends. A whole lot of cameras and most MFDB have max DR in the 11-12 stop range - and what I am saying is those numbers are misleading. The numbers are calculated according the technical definition of the word, which was not developed with photography in mind.
That's why Imatest reports DR in 3 different sets of numbers - the highest or max will compare to the theoretical specs but is not useful for photographers as Bill notes in his post above.
In my own tests the 1DIII had more max DR than the Leica DMR but fully 1.5 stops less 'usable' DR. You actually can see this amount of difference in the images pretty easily. I can not say anything about the 1DsIII because I haven't tested it - I am only guessing it will be similar.
-
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
-
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
Here is a surprize: this section of the forums is called Equipment & Techniques.
-
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157333\"][{POST_SNAPBACK}][/a]
Maybe but you almost have to be with digital. The price of all this stuff forces everyone to do a lot of research. It's all still pretty new to most of us. I shoot almost every day, but I find myself on the forums almost every day too. Still trying to learn how to get the best images out of all this.
-
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157333\"][{POST_SNAPBACK}][/a]
Hey man, its all part of the game... With film there was the zone system and with digital there is the bits and sensors. Nothing wrong with getting to know your equipment as mutch as possible.
-
However, your analogy doesn't work. DR has nothing to do with absolute size, or breakdown. It's about the ratio of maximum recordable signal, to the lowest usable signal. The lowest usable *could*, theoretically, be limited by the number of levels, but in reality, it is mainly determined by analog noise.
[a href=\"index.php?act=findpost&pid=157237\"][{POST_SNAPBACK}][/a]
The quantity of charge which can be retained after light hits a CCD certainly has to do with it's dynamic range while I'm in absolute agreement that noise only detracts the usefulness of quantity. A teaspoon of pepper in a 55 gallon drum filled with water has a different effect than a teaspoon of pepper in a glass.
-
The full well capacity and read noise for the D200 are supplied by Roger Clark (http://www.clarkvision.com/imagedetail/evaluation-nikon-d200/index.html) and are shown in the table below. By this definition, the D200 has 11.7 stops of DR.
(http://bjanes.smugmug.com/photos/227100438-M.gif)
However, this DR is overoptimistic for photographic purposes.[a href=\"index.php?act=findpost&pid=157253\"][{POST_SNAPBACK}][/a]
I don't think it's even correct for the traditional audio engineering definition of DR. There is no way that the D200 has only 10 electrons read noise at its lowest ISO. That's most likely an error on Roger's part. The D200 ISO 100 RAW files I have seen have a read noise of 18 electrons, almost a full stop more than Roger's figure.
As far as the usefulness of DR values are concerned, obviously they must be taken in the context of their over-simplicity. The engineering definition simply tells us how far down below saturation the image starts to rapidly break up from read noise. It's just a reference point, from which one can offset their own standards. More useful, for any camera at a given ISO, is a set of figures; the engineer's DR, the point at which 1:1 SNR is achieved, along with other values, such as 3:1 and 10:1. The relative strengths of read and shot noise at various RAW levels are independent, so one camera can have more DR from saturation down to 10:1, while the other has more down to 1:1.
-
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157333\"][{POST_SNAPBACK}][/a]
If that were true, it would be perfectly fine, legal, and none of your business.
-
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
Perhaps you'll learn someday that knowing your equipment is one of the things that makes the difference between a photographer and a mere button-presser, just like intimately knowing the effects of tire pressure and wheel alignment and engine tuning and tire temperature and track surface conditions makes the difference between a winning racing team and a losing one. Knowing how to get every possible bit of performance out of your gear in challenging circumstances can often make the difference between getting a usable shot and not. Knowing the exact relationship between the camera histogram and the point at which the RAW file is clipped and where on the camera histogram the noise level becomes great enough to ruin shadow detail at a particular ISO is critical, especially when shooting in challenging lighting situations. Perhaps you think Ansel Adams was an idiot for inventing the Zone System?
-
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157427\"][{POST_SNAPBACK}][/a]
and i would like to know if you think that you are one of these three giants ?
-
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157333\"][{POST_SNAPBACK}][/a]
The "only three" argument may be usefully extended to cover the number who actually understand, really understand, digital.
The English astronomer Sir Arthur Stanley Eddington was asked whether it was true that only three people understood Einstein’s theory of gravitation. He is supposed to have hesitated because he was trying to think who could be the third!
-
I'm convinced that there are MAYBE three photographers on this entire forum, and one of them runs the board. It seems that everyone else is a gearhead techno scientist.
[a href=\"index.php?act=findpost&pid=157333\"][{POST_SNAPBACK}][/a]
Looks like anthony needs to be out shootiing and working on building up his portfolio. and not in here. After he shoots a lot then he may be back in here asking tech questions..:+}
Anthony did you read the forum topic?
Snook
-
Perhaps you'll learn someday that knowing your equipment is one of the things that makes the difference between a photographer and a mere button-presser, just like intimately knowing the effects of tire pressure and wheel alignment and engine tuning and tire temperature and track surface conditions makes the difference between a winning racing team and a losing one. Knowing how to get every possible bit of performance out of your gear in challenging circumstances can often make the difference between getting a usable shot and not. Knowing the exact relationship between the camera histogram and the point at which the RAW file is clipped and where on the camera histogram the noise level becomes great enough to ruin shadow detail at a particular ISO is critical, especially when shooting in challenging lighting situations. Perhaps you think Ansel Adams was an idiot for inventing the Zone System?
[a href=\"index.php?act=findpost&pid=157427\"][{POST_SNAPBACK}][/a]
Adams wasn't even present at his children's birth because he was out doing real photography. Do you think he spent his time at home reading graphs?
Looks like anthony needs to be out shootiing and working on building up his portfolio. and not in here. After he shoots a lot then he may be back in here asking tech questions..:+}
Anthony did you read the forum topic?
Snook
[a href=\"index.php?act=findpost&pid=157461\"][{POST_SNAPBACK}][/a]
Most photographers here don't even show their "work", some prefer to show graphs to tell us how photography should be. The majority of the crowd here will probably go throught their entire life before they can get one image in the same class as Anthony's.
You'll learn photography making photos and prints. Most of this theory thing is useless in real world photography and will never make you a better photographer.
But of course, this is an open forum and people can talk about whatever they want and all opinions should be respected.
-
I don't think it's even correct for the traditional audio engineering definition of DR. There is no way that the D200 has only 10 electrons read noise at its lowest ISO. That's most likely an error on Roger's part. The D200 ISO 100 RAW files I have seen have a read noise of 18 electrons, almost a full stop more than Roger's figure.
[a href=\"index.php?act=findpost&pid=157369\"][{POST_SNAPBACK}][/a]
Roger's method of determining the read noise may not be accurate. As Emil Martinec has shown, the D3 clips the black point. My own tests with the D200 also show the black point is clipped, as shown in this Iris histogram of one of the green channels:
[attachment=4094:attachment]
Whether this is half Gaussian or some other distribution is not clear. If so, one could multiply the standard deviation by 1.66. It would be preferable to use Emil's regression method, but this would require additional data. If one multiplied Roger's read noise by 1.66 to correct for the clipping, then the read noise would be 17, more or less in agreement with your figures.
As far as the usefulness of DR values are concerned, obviously they must be taken in the context of their over-simplicity. The engineering definition simply tells us how far down below saturation the image starts to rapidly break up from read noise. It's just a reference point, from which one can offset their own standards. More useful, for any camera at a given ISO, is a set of figures; the engineer's DR, the point at which 1:1 SNR is achieved, along with other values, such as 3:1 and 10:1.
[a href=\"index.php?act=findpost&pid=157369\"][{POST_SNAPBACK}][/a]
That is the whole point of the Imatest plot, which shows noise in terms of f/stops rather than S:N. The average person would have trouble translating a S:N to what is seen visually.
-
Roger's method of determining the read noise may not be accurate. As Emil Martinec has shown, the D3 clips the black point. My own tests with the D200 also show the black point is clipped, as shown in this Iris histogram of one of the green channels:
[attachment=4094:attachment]
Whether this is half Gaussian or some other distribution is not clear.
(EDIT - the red text is a mistake; ignore it - I explain what I should have wrote in another reply to bjanes)
The biggest clue is when you subtract the number of positive values from the number of zero values. The result should be equal to the number of positive values, if the RAW was really zeroed at photonic black (with a short exposure with no significant dark current noise, of course). Here's a case where 14 bits would be convenient; the accuracy of this estimation would be better with more levels, even if the extra 2 did not contain significant usable signal.
If so, one could multiply the standard deviation by 1.66. It would be preferable to use Emil's regression method, but this would require additional data.
Well, I get my figure by subtracting shot noise from total noise in quadrature from a section of smooth, out-of-focus near-black. For example, with a mean level of 8 ADU (with almost no zeros in the sample), I get a sigma of 2.55 ADU. With an assumed 32K electrons at 4095, a mean of 8 ADU represents a mean of 8/4095 * 32000 = 62.5 electrons. The sigma of 2.55 ADU represents a sigma of 2.55/4095 * 3200 = 19.9 electrons. (19.9^2 - 62.5)^0.5 = 18.26 electrons total read noise, or 18.26/32000 * 4095 = 2.34 ADU. 10 electrons (, as Roger reports, would make the D200 the best DSLR except maybe for the K10D, for deep shadows at ISO 100.
Edit - that text in red was not meant to be in my reply; I thought I had deleted it. 10 electrons would put it right in line with most of the DSLRs with the lowest read noise at ISO 100, not better.
Reverse-engineering the effects of clipping (from the difference between the number of zeros and the number of positive values), and working with a blackframe, however might be preferable, as then you could get the noise of the entire RAW, as opposed to single color channel (considering green to be two channels, of course). It would take some experimentation to see of this is practical. Basically, you would have to clip different noise levels from a full gaussian curve at different points in the curve, and see if the relationship has a direct mapping.
If one multiplied Roger's read noise by 1.66 to correct for the clipping, then the read noise would be 17, more or less in agreement with your figures.
That is the whole point of the Imatest plot, which shows noise in terms of f/stops rather than S:N. The average person would have trouble translating a S:N to what is seen visually.
[a href=\"index.php?act=findpost&pid=157480\"][{POST_SNAPBACK}][/a]
Well, a full graph of SNR at all RAW levels is the most useful, especially when comparing cameras on the same graph. Of course, white-balancing must be taken into consideration for practical use, possibly by charting the weakest channel, adjusted for its scaling factor. SNR in tungsten light or deep desert shade is obviously going to be worse than SNR in magenta light, if they are all to be fully WB'ed.
-
phase-leaf-hasselblad-sinar have 16bit and use 14bit. two bits are "unused".
[a href=\"index.php?act=findpost&pid=157176\"][{POST_SNAPBACK}][/a]
They're spare bits, in case any of the other bits get broken.
-
The biggest clue is when you subtract the number of positive values from the number of zero values. The result should be equal to the number of positive values, if the RAW was really zeroed at photonic black (with a short exposure with no significant dark current noise, of course).
[a href=\"index.php?act=findpost&pid=157497\"][{POST_SNAPBACK}][/a]
That is an interesting concept, so let us examine it. Shown below are the raw data as reported by Iris for the green2 CFA:
(http://bjanes.smugmug.com/photos/227767314-M.gif)
The data may be shown as a histogram:
(http://bjanes.smugmug.com/photos/227767297-O.gif)
The black frame should have all values at zero, but the read process introduces an error, and the resulting values can be expressed as a mean of 0 ± the error, which could be expressed in standard deviations. If the camera is clipping the noise at zero, then the negative values would be clipped to zero and their values added to the zero column of the histogram. Since the normal distribution is symmetrical, every positive bar in the histogram has a negative mirror image, which has been clipped and added to the zero bar. We can reconstruct the non-clipped data by copying each positive bar to the negative side and subtracting that amount from the zero bar.
The arithmetic is shown below. The original data are on the left. The sum of the positive non-zero data are shown at the bottom, and the result of subtracting the non-zero values from the zero values is also shown. The resulting histogram is also shown.
(http://bjanes.smugmug.com/photos/227767317-O.gif)
(http://bjanes.smugmug.com/photos/227767299-O.gif)
The normal distribution is continuous and the probability of getting any value exactly equal to zero is zero, since on a continuum there are infinite possibilities. For the zero bar, we are actually using 0±0.5. The ADC does the rounding here.
We can now examine some normal distributions generated in Excel. The mean is 0 and N = 10^6. Histograms for σ =0.5, 1, 2 and 5 are shown.
(http://bjanes.smugmug.com/photos/227767305-O.gif)
(http://bjanes.smugmug.com/photos/227767306-O.gif)
(http://bjanes.smugmug.com/photos/227767308-O.gif)
(http://bjanes.smugmug.com/photos/227767311-O.gif)
The integral 0..∞ in all cases should contain half the population, but with integers, data values less than 0.5 in the tail are clipped to zero and lost. Looking at the σ = 0.5 histogram one could construct the clipped histogram by adding the positive non-zero values to the zero bar and deleting the negative values. If we subtract the positive non-zero values from the new zero bar, we merely recover the original zero bar. The result is greater than the sum of the non-zero bars. For the σ = 5 case, the reverse is true. Examination of the other histograms is left to the reader. John's criterion does not seem to work.
Well, I get my figure by subtracting shot noise from total noise in quadrature from a section of smooth, out-of-focus near-black. For example, with a mean level of 8 ADU (with almost no zeros in the sample), I get a sigma of 2.55 ADU. With an assumed 32K electrons at 4095, a mean of 8 ADU represents a mean of 8/4095 * 32000 = 62.5 electrons. The sigma of 2.55 ADU represents a sigma of 2.55/4095 * 3200 = 19.9 electrons. (19.9^2 - 62.5)^0.5 = 18.26 electrons total read noise, or 18.26/32000 * 4095 = 2.34 ADU. 10 electrons (, as Roger reports, would make the D200 the best DSLR except maybe for the K10D, for deep shadows at ISO 100.
[a href=\"index.php?act=findpost&pid=157497\"][{POST_SNAPBACK}][/a]
That seems reasonable, and is in agreement with the result obtained by assuming that the clipped distribution is half-Gaussian.
-
The integral 0..∞ in all cases should contain half the population, but with integers, data values less than 0.5 in the tail are clipped to zero and lost. Looking at the σ = 0.5 histogram one could construct the clipped histogram by adding the positive non-zero values to the zero bar and deleting the negative values. If we subtract the positive non-zero values from the new zero bar, we merely recover the original zero bar. The result is greater than the sum of the non-zero bars. For the σ = 5 case, the reverse is true. Examination of the other histograms is left to the reader. John's criterion does not seem to work.
[a href=\"index.php?act=findpost&pid=157653\"][{POST_SNAPBACK}][/a]
I looked back in my post that you replied to, and I see that I didn't express my criterion correctly. It had both a typo and a conceptual problem (probably confusing this with something else that I had been doing), and I remember now what it was I was looking for, and it is not as exacting.
Here's the real criterion.
In an unclipped histogram where:
T = total population
Zo = original zero population
N = negative population
P = positive population
Then: T = N+Z+P
When clipping occurs, and when N = P (blackpoint is correctly assumed), then clipping results in T = Zc + P where
Zc = zero population after clipping
and Zc = Zo+N.
One of the most obvious things to look for is that P is not greater than Zc. If P is greater than Zc, then the black point must be clipped too low for a mid-histogram clip. P should always be smaller than Zc.
If you mirror the individual positive populations and use them as negative populations, and subtract P from Zc, then you have a hypothetical Zh, which you can then compare to the rest of the curve, to see how well it fits. If Zh is too high to fit the curve, then black was clipped too high. If Zh is too low, then black was clipped too low.
Now, I remember a few months ago experimenting with ways of calculating how far off the black might be, and I remember that it helped when the noise was high, in ADUs, as the difference in population between successive values should be small, and then Zh could be seen as roughly multiples of those populations for a rough estimate of how far off the blackpoint is from zero. You can see the difference that I am talking about between your sigma=0.5 and 5.0 histograms; if the 5.0 was available only in clipped form, and you had to estimate how may ADUs black was off from zero, the potential for error would be much less than with the 0.5 histogram.
-
BJanes and John,
You guys are scaring me! But let me ask you (collectively) a question. Why can't a camera/sensor combo that uses 16 bits get a full 16 stops DR? Seems like the most I have seen is 12 stops. Will we have to wait until the sensor and camera manufacturers go to 32bits?
Thanks,
Eric
-
BJanes and John,
You guys are scaring me! But let me ask you (collectively) a question. Why can't a camera/sensor combo that uses 16 bits get a full 16 stops DR? Seems like the most I have seen is 12 stops. Will we have to wait until the sensor and camera manufacturers go to 32bits?
[a href=\"index.php?act=findpost&pid=157726\"][{POST_SNAPBACK}][/a]
To use an audio analogy, the cameras have too much hiss.
Clever readout circuitry, such as used in Canon DSLRs and the Nikon D3, can read out a fraction of the sensor's DR (IOW< at high ISO) with less hiss, relative to absolute signal, but when reading out the entire sensor DR, at the base ISO, the hiss is stronger relative to signal. If it weren't for this hiss, a camera could just count electrons and give an optimal signal, limited only by total sensor electron capacity. Shadows are much, much more aesthetically pleasing when low electron counts are the only "noise", because blacks get close to really being black, so there is more contrast between near-black and blacker. When the "hiss" or read noise is present, it clouds over the darkest areas in a blanket.
-
The "only three" argument may be usefully extended to cover the number who actually understand, really understand, digital.
The English astronomer Sir Arthur Stanley Eddington was asked whether it was true that only three people understood Einstein’s theory of gravitation. He is supposed to have hesitated because he was trying to think who could be the third!
[a href=\"index.php?act=findpost&pid=157454\"][{POST_SNAPBACK}][/a]
Actually, if I really try I guess I can follow most of the arguments here. Having an engineering degree with a VLSI design major certainly helps
That doesn't mean I would want to be doing the talking, as of now. I'll leave that to people who make the technology tick on a day to day basis.
I don't know whether this stuff is really interesting or not. My back has stripes at high ISO, I'm told they come from underexposure, I'm still not convinced. I'm starting to get interested in being able to quantify the performance of a camera so as to be able to determine objectively whether the equipment I have is as good as the next guy's.
Edmund
-
You can see the difference that I am talking about between your sigma=0.5 and 5.0 histograms; if the 5.0 was available only in clipped form, and you had to estimate how may ADUs black was off from zero, the potential for error would be much less than with the 0.5 histogram.
[a href=\"index.php?act=findpost&pid=157717\"][{POST_SNAPBACK}][/a]
Here is the clipped sigma 5 histogram. It does appear quite different.
Bill
[attachment=4104:attachment]
-
I don't know whether this stuff is really interesting or not. My back has stripes at high ISO, I'm told they come from underexposure, I'm still not convinced. I'm starting to get interested in being able to quantify the performance of a camera so as to be able to determine objectively whether the equipment I have is as good as the next guy's.
[a href=\"index.php?act=findpost&pid=157731\"][{POST_SNAPBACK}][/a]
What is needed is a repository of sample files from multiple specimens of the same camera, so that comparisons can be made, so one can decide if the issue is by (passive) design or if the individual specimen is a lemon. For MF backs, we're talking about a lot of data. The RAWs need to converted in a repeatable, homogenous manner, or be RAW, but most people don't know how to look directly at RAW data.
A doctored version of DCRAW that looked for common problems and profiled them would be useful, too, and could keep the data to a minimum. You'd have to feed it things like blackframes, OOF flat areas, etc.
-
Here is the clipped sigma 5 histogram. It does appear quite different.
[a href=\"index.php?act=findpost&pid=157773\"][{POST_SNAPBACK}][/a]
Yes, but it doesn't demonstrate what I was talking about. The unclipped histogram shows populations for -1, 0, and 1 that are very close for the sigma=5.0, but are radically different for sigma=0.5. The 5.0 allows for simple interpolation; IOW, if the difference between P and Zc were 10,000, and the population of value 1 was 5000, you'd know that the blackpoint was about 1 ADU off. This is where my first, mistated criterion comes from, now that I think of it.
-
That seems reasonable, and is in agreement with the result obtained by assuming that the clipped distribution is half-Gaussian.
[a href=\"index.php?act=findpost&pid=157653\"][{POST_SNAPBACK}][/a]
BTW, while I correctly gave the contraction in width for a half-gaussian vs a full gaussian as ~1.66, the actual state of affairs is of course a half gaussian on the right plus a delta function at zero containing the weight of the negative values that have been clipped to zero. For the latter case the factor is ~1.71.
But the problem is that one cannot know precisely where the clipping has been done, and the noise spectrum is not a pure gaussian anyway. So I have some doubt as to whether applying such correction factors will give a reliable result. For instance in the D300 the signal is clipped well to the right of the mean of the noise distribution (over 90% of the raw values are zero up to about ISO 800). This is why methods using near-black but unclipped noise histograms extrapolated to zero give the most accurate results; no assumptions are being made.
-
Here are ISO 400 histograms of noise vs lower exposure for a D200; green channel data of a Kodak IT8 chart underexposed by three stops. These are histograms of the grayscale patches along the bottom of the chart. The growing spike at zero as the mean level is reduced is the aggregate weight of the histogram to the left of zero being lumped into a delta function *at* zero by the clipping of black:
(http://theory.uchicago.edu/~ejm/pix/20d/posts/tests/D300_40D_tests/D200_iso400_green_histos.jpg)
The darkest patch (22) approaches the endpoint, which is roughly a half gaussian of unaffected raw values with positive read noise, plus a delta function incorporating all the pixels whose negative raw value has been set to zero by clipping.
From this camera I was getting about 14 electrons of read noise at ISO 100, somewhat in between John's figure and Roger's.
-
But the problem is that one cannot know precisely where the clipping has been done, and the noise spectrum is not a pure gaussian anyway. So I have some doubt as to whether applying such correction factors will give a reliable result. For instance in the D300 the signal is clipped well to the right of the mean of the noise distribution (over 90% of the raw values are zero up to about ISO 800). This is why methods using near-black but unclipped noise histograms extrapolated to zero give the most accurate results; no assumptions are being made.
[a href=\"index.php?act=findpost&pid=158467\"][{POST_SNAPBACK}][/a]
Emil,
Thanks for the update. Certainly you have convinced me of the validity of your method. However, some will remain unconvinced.
Bill