Luminous Landscape Forum
Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: Ray on December 14, 2007, 02:07:13 am
-
As I suspected, using the legibility of large B&W text as a criterion for DR assessment results in a really high figure.
Unfortunately, I made the mistake of not continuing far enough with the underexposures, so it's not clear at what point the B&W text would cease to be legible. At least another 2/3rds of a stop I would think, giving a maximum DR of 12 stops.
Here are two shots 11.33 stops apart. I'm assuming with this method that one should take the interval in f stops between the longest and shortest exposure and not count the actual number of exposures taken at one stop intervals.
The exposures range from 2 secs to 1/1250th.
[attachment=4240:attachment] [attachment=4241:attachment]
-
Using the white text legibility is really a not a valid method, especially if you let the white text clip on the high end of the exposure series. If you do that, you're comparing apples and oranges and the results you get will be inflated by the density difference between the white text and the text in the quadrants.
But even if you use the white text for clipping and legibility, the DR figure you've arrived at is only valid when shooting high-contrast white-on-black subjects, which isn't really very useful. Most of the time we photographers are trying to pull low-contrast tonal subtleties from the shadows--woodgrain texture from a piece of furniture, the texture of the subject's hair, etc. and the text in the quadrants is a much better predictor of how well a particular camera + RAW converter will accomplish that. What results do you get using the quadrant text?
I updated the target image, please download the new one. It fixes the slight clipping present in the blue quadrant.
-
One other thing; it appears that the white square is bigger than 100 pixels across in your test shot. Maintaining a consistent size is important, because the bigger the letters are at 100%, the easier it is to read them. If you shoot the chart too large it will skew the results.
-
As I suspected, using the legibility of large B&W text as a criterion for DR assessment results in a really high figure.
Unfortunately, I made the mistake of not continuing far enough with the underexposures, so it's not clear at what point the B&W text would cease to be legible. At least another 2/3rds of a stop I would think, giving a maximum DR of 12 stops.
Here are two shots 11.33 stops apart. I'm assuming with this method that one should take the interval in f stops between the longest and shortest exposure and not count the actual number of exposures taken at one stop intervals.
The exposures range from 2 secs to 1/1250th.
[a href=\"index.php?act=findpost&pid=160579\"][{POST_SNAPBACK}][/a]
Ray,
The legibility test is one way of determining the noise floor, but the question remains: is that noise level acceptable? For high quality work, many would set the noise floor higher, and this is a subjective matter.
Bill
-
Using the white text legibility is really a not a valid method, especially if you let the white text clip on the high end of the exposure series. If you do that, you're comparing apples and oranges and the results you get will be inflated by the density difference between the white text and the text in the quadrants.
Jonathan,
(1)There's no indication of clipping of the whites in my ETTR at 2secs. I printed the target on matte paper and the texture of the paper is evident in the white square in the centre and the white border of the A4 sheet, ie. the values fluctuate between 214,214,214 and 218,218,218. The fluctuations of the white border are greater, varying from 221 to 228.
(2) Not only is the B&W text clearly readable 11.3 stops down but also the black cross in the centre. Now obviously such broad-brush detail in such degraded shadows is not much use to a fine-art landscaper but it would be to a spy who's life depended on decoding a message in the 12th stop of a Canon 5D image , or indeed an amateur astronomer searching for a new discovery that could get him a name in history.
I haven't made much attempt to determine a more useful DR for ordinary photography because I'm at a severe disadvantage regarding the status of my printer. Since I'm at present travelling, I'm using Epson's bottom-of-the-range 4-ink printer which I shall probably discard when I move from my current lodgings.
It's really quite primitive. One generic profile for everything, it makes a noise like a steam engine (clickety clack, clickety clack) and has a tendency to stop printing whenever a light is switched on or off, either in my room or the adjoining room, or whenever the air-conditioner is switched on or off. I've wasted a few prints as a result.
I'll try printing a glossy of your new target, but I don't think that Absolute Colorimetric has any meaning for my printer's color management. On the matte paper I can hardly read the smallest, faintest numbers.
By the way, why are you standardising the size of the target in terms of pixels? This is like dpreview's method of comparing noise. You get results like, the D60 has less noise than the 1Ds up to ISO 400. Surely what's important is the dynamic range of the whole camera.
Also, you didn't answer my question regarding the method of counting stops. Is one stop the interval between one exposure and the next exposure at half the value? On that basis, the extreme DR of the 5D is 11.3 stops. However, if I count all the exposures with a one stop interval between them, I get 12.3 stops.
I'll try this again with my best glossy print of your new target .
-
Ray,
1. I don't think the glossy paper is a good idea,
2. if you post the raw images, I can verify the clipping and the differences in stops, as well as the standard deviation on the darkest areas.
-
Ray,
1. I don't think the glossy paper is a good idea,
2. if you post the raw images, I can verify the clipping and the differences in stops, as well as the standard deviation on the darkest areas.
[a href=\"index.php?act=findpost&pid=160799\"][{POST_SNAPBACK}][/a]
Panopeeper,
Nor did I, which is why I initially printed the target on matte. But the fact is, if I can hardly read the smallest and faintest numbers before I've photographed the target, what chance have I of reading them after they've been photographed.
I've just discovered that one can join Yousendit free for 1GB per month of uploads.
I'll see if I can work out how to use it and take up your offer. I tried sending Guillermo a RAW file once, through my ISP in Australia, but failed.
-
But the fact is, if I can hardly read the smallest and faintest numbers before I've photographed the target, what chance have I of reading them after they've been photographed.
That's the whole point of the exercise, photograph something with subtle detail that isn't glaringly blatant. You've already demonstrated that using the legibility of the large white-on-black letters skews the results unrealistically high. And the smallest letters are distinguishable in the screenshot you posted already, so your camera is capable of capturing them legibly. It wouldn't be cheating to view the test frames at >100% in ACR, you know.
By the way, why are you standardising the size of the target in terms of pixels? This is like dpreview's method of comparing noise. You get results like, the D60 has less noise than the 1Ds up to ISO 400. Surely what's important is the dynamic range of the whole camera.
And it's equally silly to rate a high-pixel-count digicam better than a DSLR simply because it can throw more pixels at the subject. The larger the letters are, the easier it is to pick them out of the shadow noise. If you fill the frame with the chart, a camera with more megapixels will have an unfair advantage on that basis alone, and cameras with greater pixel counts will get a higher DR rating than they are actually entitled to. Keeping things constant across the board will give you a standardized evaluation of pixel quality. What's the point of increasing pixel count if you're not simultaneously increasing your ability to resolve fine details?
Also, you didn't answer my question regarding the method of counting stops. Is one stop the interval between one exposure and the next exposure at half the value? On that basis, the extreme DR of the 5D is 11.3 stops. However, if I count all the exposures with a one stop interval between them, I get 12.3 stops.
Given a 1/3-stop exposure interval, count the number of exposures where the quadrant text is legible and divide by 3. If you shot 21 frames at 1/3-stop intervals where the quadrant text is legible, then photographically useful DR is 7 stops.
-
That's the whole point of the exercise, photograph something with subtle detail that isn't glaringly blatant.
Jonathan,
In that case the quality of the print in conjunction with the quality of the lens will affect the result significantly.
And it's equally silly to rate a high-pixel-count digicam better than a DSLR simply because it can throw more pixels at the subject.
I doubt whether a high pixel digicam, say a 12mp Canon G9, could record finer detail at say 6 stops underexposure, than a 6mp DSLR, but if it could, it wouldn't be silly, it would be bloody marvelous .
I don't see why this should be a competition like a horse race where the stronger horse is handicapped. We're concerned about reality here, aren't we? Refer to the title of your thread, 'Real world DR testing'. If my camera exhibits greater DR in practice, in real world shooting, because it has more pixels, then so be it. Maybe that's one of the reasons I bought the camera.
If you shot 21 frames at 1/3-stop intervals where the quadrant text is legible, then photographically useful DR is 7 stops.
I shot 35 frames including the 2 I've shown above plus all in-between exposures at 1/3rd stop intervals. That makes the extreme outer limit of the 5D 11.67 stops of DR then.
I haven't examined closely the quadrant text yet, but I expect that the faintest and smallest text will becom illegible very quickly.
-
If my camera exhibits greater DR in practice, in real world shooting, because it has more pixels, then so be it. Maybe that's one of the reasons I bought the camera.
But that's the problem; it wouldn't. Consider this "thought experiment" for a moment. Imagine you conduct my test with a prime lens on your camera and frame each shot so that the square is 100 pixels wide. Now re-run the test with everything the same except you move the camera closer, so that the square is 500 pixels wide. Are you going to get exactly the same results? No. Did your camera magically increase its DR just because you moved it closer to the subject? Absolutely not. In order for the test results to be consistent and meaningful, the number of pixels resolving the target must be standardized. Varying the pixel count on-target will skew the test and make it less relevant to real-world results.
In that case the quality of the print in conjunction with the quality of the lens will affect the result significantly.
Lens quality affects everything, including real-world image quality; for best results, use the best lens you can. Regarding the quality of the test target, printing on decent matte paper with a good quality, properly profiled printer using Absolute Colorimetric rendering and the best quality print mode will give you consistency within the accuracy of the profile. I designed the target image to avoid excessively saturated colors so that gamut issues shouldn't be a problem. If you shoot so that the white square is 100 pixels across, the dithering patterns of the printer will be below the resolution of the sensor unless you have a really cheap printer.
-
Consider this "thought experiment" for a moment. Imagine you conduct my test with a prime lens on your camera and frame each shot so that the square is 100 pixels wide. Now re-run the test with everything the same except you move the camera closer, so that the square is 500 pixels wide. Are you going to get exactly the same results?
Yes, of course not. I understand that. I'm not recommending the square be any specified number of pixels. I'm recommending the width of the target fit the width of the sensor. In other words, test the DR of the camera, not one of its pixels.
Regarding the quality of the test target, printing on decent matte paper with a good quality, properly profiled printer using Absolute Colorimetric rendering and the best quality print mode will give you consistency within the accuracy of the profile. I designed the target image to avoid excessively saturated colors so that gamut issues shouldn't be a problem. If you shoot so that the white square is 100 pixels across, the dithering patterns of the printer will be below the resolution of the sensor unless you have a really cheap printer.
I have the cheapest printer that Epson sell; the 4-ink C90. It's not properly profiled, I'm using non-Epson paper and I'm working from a poorly calibrated laptop. Those are my current circumstances, which is why I haven't spent too much time examining the intermediate shots.
Panopeeper offered to examine my RAW files to see if the 2-second exposure is clipped or not, so courtesy YouSendIt the links are below for anyone who wants to have a look.
http://www.yousendit.com/download/www/YVJZc2ZFMVhVbS9IRGc9PQ (http://www.yousendit.com/download/www/YVJZc2ZFMVhVbS9IRGc9PQ)
http://www.yousendit.com/download/www/YVJaZFhuT2JFd2ZIRGc9PQ (http://www.yousendit.com/download/www/YVJaZFhuT2JFd2ZIRGc9PQ)
-
Yes, of course not. I understand that. I'm not recommending the square be any specified number of pixels. I'm recommending the width of the target fit the width of the sensor. In other words, test the DR of the camera, not one of its pixels.
And that's exactly what you're NOT doing, for the reasons I brought up in my previous post. Changing the number of pixels used to resolve the text changes the legibility threshold and invalidates the consistency of results. Or do you actually believe that chopping a sensor in half (if you could do so without ruining it and still read out the remaining pixels) would reduce the DR of the remaining pixels? Or that adding additional identical sensors to a camera will increase its DR? That's preposterous.
Velvia's DR is the same whether it is loaded into a 35mm camera or an 8x10. By your logic, the 8x10 would have more DR than the 35mm, even if loaded with the same film stock.
-
Panopeeper offered to examine my RAW files to see if the 2-second exposure is clipped or not, so courtesy YouSendIt the links are below for anyone who wants to have a look.
[a href=\"index.php?act=findpost&pid=160878\"][{POST_SNAPBACK}][/a]
The 2 second:
The white areas are all clipped in the blue and green channels. This means that they are either barely clipped in true "daylight", or are very clipped in a bluish light source (window from blue sky, or a so-called "daylight" bulb, which is usually bluer than real daylight). The red only clips in specular highlights in the lower right, just to the right of the letter "g" in "Target".
Mean: 1758.4 Median: 2019
Sigma: 1176.2
Maxi.: 3692.0 Mini.: 186.0
Clipping is at 3692, obviously.
A minimum of 186 says that in the weakest channel (red), the blackest thing in that image is only log((3692-128)/(186-128))/log(2) = 5.9 stops below saturation, and that is where the negative noise goes the deepest, so the mean is actually slightly higher than that.
The 0.001 second:
Nothings clipped (but that goes without saying).
Mean: 128.7 Median: 129
Sigma: 2.0
Maxi.: 158.0 Mini.: 117.0
ISO 100 is not where you want to do this thing. I don't remember the exact figure for the 5D, but banding noise is much higher at ISO 100, in electrons; something like 20x as high as ISO 1600. (Later Edit: Sorry, that figure is not right. for the 20D it is about 3x as high for horizontal, and 12x for vertical; the 5D is similar, IIRC). 1D banding noise does not decrease at the rate of 2D banding noise with small viewing or downsampling; it is a very horrible noise to deal with. With pure shot noise, I think you'd be able to read the larger texts in the triangles.
-
Changing the number of pixels used to resolve the text changes the legibility threshold and invalidates the consistency of results. Or do you actually believe that chopping a sensor in half (if you could do so without ruining it and still read out the remaining pixels) would reduce the DR of the remaining pixels?
Chopping a sensor in half would increase the total noise in an image of a given size, if I've understood the significance of dpreview's noise comparisons between the D60 and the 1Ds.
I'm simply using that analogy as it may apply to DR. I know DR is not the same as noise, but I would have thought that any reduction in noise either at the pixel level or at the 'total image' level has the effect of increasing DR.
In other words, if DR is equal at the pixel level in two different cameras, the camera with the greatest number of pixels will be capable of delivering an image of greater dynamic range. Is this not true? I could be wrong .
Velvia's DR is the same whether it is loaded into a 35mm camera or an 8x10. By your logic, the 8x10 would have more DR than the 35mm, even if loaded with the same film stock.
The same per unit area, yes. But not the same per unit picture of same FoV. Medium and large format have always been known for their smoother tonality (compared to 35mm) which I suspect also translates to a greater dynamic range in the sense that more detail is there in both highlights and shadows (as well as transitions), simply because there are more film grains available to describe such detail.
-
Ray,
here are the raw histograms. The 2s shot is good for this purpose, but the 1/1250s is far out of the range. The pixel values are between 119 and 140 (apart from a few stray pixels), while the values of masked pixels, which are used to determine the level of "black current" is between 120 and 130. In other words, much of the image must be indistinguishable from one shot with the lens cap on (35% of the reds and 25% of the greens and blues are under 128).
-
The 2 second:
The white areas are all clipped in the blue and green channels. This means that they are either barely clipped in true "daylight", or are very clipped in a bluish light source (window from blue sky, or a so-called "daylight" bulb, which is usually bluer than real daylight). The red only clips in specular highlights in the lower right, just to the right of the letter "g" in "Target".
John,
That's interesting. The target was illuminated by the natural light of a blue sky through the window. I suppose ACR has done a fair job in reconstructing the clipped channels. What would you say, 2/3rds of a stop overexposed? That brings the extreme DR limit of my 5D down to only 11 stops. Damn!
ISO 100 is not where you want to do this thing. I don't remember the exact figure for the 5D, but banding noise is much higher at ISO 100, in electrons; something like 20x as high as ISO 1600. 1D banding noise does not decrease at the rate of 2D banding noise with small viewing or downsampling; it is a very horrible noise to deal with. With pure shot noise, I think you'd be able to read the larger texts in the triangles.
Since this is supposed to be a DR test, it's difficult to understand how I would get a better result at a higher ISO. Is DR likely to be better at say ISO 200?
-
Ray,
here are the raw histograms. The 2s shot is good for this purpose, but the 1/1250s is far out of the range. The pixel values are between 119 and 140 (apart from a few stray pixels), while the values of masked pixels, which are used to determine the level of "black current" is between 120 and 130. In other words, much of the image must be indistinguishable from one shot with the lens cap on (35% of the reds and 25% of the greens and blues are under 128).
[a href=\"index.php?act=findpost&pid=160904\"][{POST_SNAPBACK}][/a]
The 2s shot is good for this purpose,
Panopeeper,
I notice that both the green and blue channels in your histogram show a cliff edge on the far right. Is this the clipping that John Sheehy is referring to?
In other words, much of the image must be indistinguishable from one shot with the lens cap on (35% of the reds and 25% of the greens and blues are under 128).
But some very relevant detail there, nevertheless, which you definitely wouldn't get with the lens cap on. There's no doubt, for example, that the subject is a Dynamice Range Test Chart because we can clearly read that it is.
-
Ray and Jonathan seem to be advocating two different forms of comparison (relevant to both noise levels and DR I think) that might each have their place.
One I call "per pixel"; the other "per print" or "per image".
Jonathan's approach is "per pixel" assessment is looking at an equal number of pixels, relevant for example if one plans to print at equal PPI, so getting a larger image from a sensor with more photo-sites. After all, you might well have paid for a camera with more pixels to get sharper and/or bigger prints, not so that you could down-sample or otherwise reduce resolution in NR processing to get acceptable noise levels and DR.
Ray's approach is "per image", comparisons relevant to making prints of the same size from different cameras and viewing from the same distance (or otherwise viewing at equal apparent image size) so that extra pixels can be used to lower the per pixel noise floor and thus increase DR, perhaps simply by the dithering effect of printing at a higher PPI, or perhaps with some spatial averaging that roughly equalizes total resolution.
-
I notice that both the green and blue channels in your histogram show a cliff edge on the far right. Is this the clipping that John Sheehy is referring to?
Right; that's in the 2 second shot. The saturation level at ISO 100 is 3692 (i.e. not the entire 12-bit range is used). Note, that the saturation level may change with ISO and even with the individual copy, but the latter is really negligable.
But some very relevant detail there, nevertheless, which you definitely wouldn't get with the lens cap on
Yes, of course, as many of the pixels are over the black level. However, the noise is inevitably very high with so many pixels under the black level. I guess you made other shots with higher exposures, those should be exemined. The point would be, that some of the texts are clean, while others are too noisy.
Anyway, I still favour the step wedge, like in DPReview's tests, but in pure raw format.
-
Since this is supposed to be a DR test, it's difficult to understand how I would get a better result at a higher ISO. Is DR likely to be better at say ISO 200?
[a href=\"index.php?act=findpost&pid=160905\"][{POST_SNAPBACK}][/a]
No, I was merely pointing out how bad banding (especially vertical) is in ISO 100 under-exposures. BTW, there was a mistake in my post, look at it again with the correction.
-
And it's equally silly to rate a high-pixel-count digicam better than a DSLR simply because it can throw more pixels at the subject. The larger the letters are, the easier it is to pick them out of the shadow noise. If you fill the frame with the chart, a camera with more megapixels will have an unfair advantage on that basis alone, and cameras with greater pixel counts will get a higher DR rating than they are actually entitled to.
[a href=\"index.php?act=findpost&pid=160829\"][{POST_SNAPBACK}][/a]
Entitled to for what? Resolution and individual pixel quality work together towards image noise and DR. Can you show a case where a line of sensors became less able to resolve detail by packing more pixels?
You seem to use the pixel-centric paradigm, popularized by Roger Clark and others, where DR automatically refers to the abilities of a single pixel as a measuring device. A pixel, just like the cones and rods in our eyes, is an artificial structure designed to bin photon strikes from an area into a single collection of photons. The real world of photons is one of infinite resolution, and sparse photon strikes near any single point (tremendous shot noise for hypothetic infinitesimal pixels). It is ultimately pointless to bin twice, once in the medium, and then again by the eye, but technology forces us to, but it should do as little binning as possible. Ray's ISO 110,000 version of your test chart proves this point. If you look at just the green channel in the RAW image, you can barely make out the biggest line in the green quadrant, and even more faintly, the second-biggest line. The next one is more like, "I think I see something there, but maybe not". Anyway, if you load it into photoshop, at 100% zoom, and stand back 15 feet from the monitor, you can make out the biggest line well enough to see the individual numbers somewhat distinctly. Come back near the monitor, and it is less distinct, as individual bright pixels obscure what's around them. Now, simulate bigger pixels by using Photoshop's pixelate|mosaic feature. Set it to Preview, and start with 2x2, then 3x3, then 4x4, etc. Instead of getting clearer of noise, but with lower resolution, resolution reduces with no reduction of noise. The clearest numbers are had at 100% (tiny pixels), viewed from a distance, because your eyes and brain get to do all the work, which they do better with more resolution, despite more noise per pixel.
-
Panopeeper,
I notice that both the green and blue channels in your histogram show a cliff edge on the far right. Is this the clipping that John Sheehy is referring to?
Yes. Gabor's histogram program is assuming that the RAW data can go to 4095, but it can only go to 3692 in your camera at ISO 100.
But some very relevant detail there, nevertheless, which you definitely wouldn't get with the lens cap on.
If you make a selection in photoshop, manually, of each area that you know has the same level throughout, and run the "Blur|Average" filter, you will get nice, clean areas distinct from each other in tonal level (unless they're supposed to be the same). The ability to capture accurate means at extremely low light levels is tremendous, barring the bands (Pun intended). If done with negative noise footroom, the response of means near black is totally linear from Canons. Despite the high levels of noise in each pixel, the mean of noise is infinitesimal, and one individual photon over an a large area does affect the average.
There's no doubt, for example, that the subject is a Dynamice Range Test Chart because we can clearly read that it is.
[a href=\"index.php?act=findpost&pid=160908\"][{POST_SNAPBACK}][/a]
What about mass hallucination?
-
Yes. Gabor's histogram program is assuming that the RAW data can go to 4095, but it can only go to 3692 in your camera at ISO 100
The camera specific parameters of the program (compiled, not user-parameters) include the saturation levels per pixelcolor and ISO, as well as the histogram range. If there is a power of 2 very close to the respective saturation level, then I specify that "round" value, otherwise some rounded value close to the top (there are variations between copies, so I rather give a slightly higher value than the top I see).
However, I don't know these values for many cameras. As the matter of fact, my program worked only with DNG until last week, and now I am about implementing the processing of native raw files (messy, messier, messiest, as if there were a competition between manufacturers). Adoba's DNG converter inserted these values, but very unreliably, I use those values only if I don't know anything better.
Anyway, if you know such values, pls tell me, or send me raw files.
Btw, we were discussing the saturation levels in ACR. As far as I see, the Canon 5D is better supported in ACR than the 40D: I saw DNG files (which reflect ACR) with different white level for different shots by 5D, the same ISO.
Thomas Knoll posted once to my related question, that the white level is based on the camera, ISO and image content. This is obviously not always the case, but they make the effort sometimes.
However, there is only a single value for the saturation per image, though the different pixels of the CFA can have different saturation levels. Ok, the difference is usually not large, but principally there should be separate values. As the matter of fact, my solution is not perfect either, because the two greens of a CFA have in some cases different saturation levels, and I can specify only one per color. If and when my program becomes a raw converter, then I will make the effort to evaluate it in runtime.
-
Ray and Jonathan seem to be advocating two different forms of comparison (relevant to both noise levels and DR I think) that might each have their place.
One I call "per pixel"; the other "per print" or "per image".
I really don't buy that argument. Let's take a look at the behavior of a randomly-selected pixel on a camera sensor over a series of 100 exposures. The number of photons required to saturate that pixel completely at a given ISO setting is going to be fairly consistent, as is the minimum number of photons required to read that pixel and say with confidence that one has a signal above noise level. The minimum and maximum number of photons that pixel can consistently detect is NOT going to change just because the photographer alters the crop of the images AFTER the exposures were made and changes the number of that pixel's brothers that are included in the final image. The decision to crop or not crop has zero effect on the dynamic range or noise level of a given capture.
Stated another way, the number of pixels included in a print has NOTHING to do with the engineering definition of DR (signal-to-noise ratio), and I categorically reject the idea that it should be included when attempting to define the subset of that range that is photographically useful. Otherwise you enter a murky realm where all the definitions change depending on how many of the captured pixels make it into the final print and you can redefine the meaning of "is" however you like.
If you think of image quality as being (pixel quality) * (pixel quantity), then you must first accurately and consistently analyze the behavior of each pixel, and then factor in differences in pixel quantity, or else you cannot meaningfully compare the performance of imaging devices with unequal pixel count.
-
Entitled to for what? Resolution and individual pixel quality work together towards image noise and DR. Can you show a case where a line of sensors became less able to resolve detail by packing more pixels?
I can think of some models of digicam that were replaced by a higher pixel count model and the new model performed worse than the old one overall because noise levels went up so much...
-
I really don't buy that argument. [a href=\"index.php?act=findpost&pid=160949\"][{POST_SNAPBACK}][/a]
I do, although the advantages of more pixels and greater resolution might be better explained by John Sheehy. At the level of pure physics (ie. excluding the effects of human perception), as far as I can see, any increase in DR would flow from a lower shot noise, not lower in the absolute sense, but lower in terms of S/N.
Lets consider the following thought experiment. (And one that BJL will appreciate, I'm sure .)
Let's imagine that at the same time the 5mp Olympus E-1 came out, Olympus also introduced a 20mp camera of the same aspect ratio but with a sensor 4x the area (roughly equivalent to the 1Ds3).
Because the technology used was very similar for both cameras, the DR of each individual pixel was identical in each camera. The photodiodes were the same, the on chip processing for each photodiode was the same, the ISO rating was the same and the read noise per pixel was the same.
If these two cameras were tested using your target per your instructions of getting the centre square 100 pixels wide, then both cameras would appear to have the same DR according to you.
Let's examine what would happen in reality when both cameras are used to shoot a scene, artificial target or general view, keeping the FoV the same for both cameras.
I'm sure you would agree, if you were to take any specified crop of that scene, however small, there would be 4x as many pixels in the crop from the larger format camera.
Let's make the crop so small that it is exactly the same size as a single pixel from the 5mp E-1, and let's say that the intensity of light is so great in that pixel-size part of the scene that it fills the well of the E-1 photodiode, producing a full ETTR.
Let's now examine the identical crop from the same scene shot with the larger format 20mp camera. The light that fills the well of one single E-1 photodiode now falls on 4 pixels from the larger camera. Each pixel receives 1/4 the amount of light, but that includes the pixels also that are gathering light from the dark part of the scene.
In order to fill all the wells in both cameras to an equal level, the camera with 4x the number of pixels needs 4x the exposure, 4x the amount of light. Either the scene needs to be 4x brighter or the aperture used in the larger camera needs to be 2 stops wider.
Now, if it were not for shot noise, there might appear to be no dynamic range advantage here, but with 4x the number of photons arriving at the sensor, I would think that shot noise, as a proportion of total signal level (S/N) would be half. I might be wrong here, though.
It might also be the case that other sources of noise completely obscure any improvement in shot noise to the point where such improvement is insignificant.
-
Let's imagine that at the same time the 5mp Olympus E-1 came out, Olympus also introduced a 20mp camera of the same aspect ratio but with a sensor 4x the area
In other words, the sensor pixel sites (sensels) are the same size.
The light that fills the well of one single E-1 photodiode now falls on 4 pixels from the larger camera. Each pixel receives 1/4 the amount of light
I think you need to rework your thought experiment.
-
In other words, the sensor pixel sites (sensels) are the same size.
[a href=\"index.php?act=findpost&pid=160958\"][{POST_SNAPBACK}][/a]
Exactly!
I think you need to rework your thought experiment.
Perhaps you can tell me why. The only factor which I see as being dubious is the significance of the photonic shot noise. It's supposed to vary with the square root of the photon count. Increase the amount of light illuminating the entire scene by a factor or 4, then for the entire scene photonic shot noise should be half, although it is the same at the pixel level for both cameras. Is this not so?
-
The minimum and maximum number of photons that pixel can consistently detect is NOT going to change just because the photographer alters the crop of the images AFTER the exposures were made and changes the number of that pixel's brothers that are included in the final image.[a href=\"index.php?act=findpost&pid=160949\"][{POST_SNAPBACK}][/a]
I was not thinking abut cropping, but processes like down-sampling or "binning" or fancier options like NR processing, so that the final output _pixels_ are produced using data from several _photosites_ The output pixels can end up with higher DR than the photosites signals used to make them.
[Aside on film. The "photosites" of film, silver halide crystals, have only two output states, so individually have very low DR. Dithering the output of billions of these "one-bit photosites" generates the DR and tonal gradations of a print, and no one cares about measuring the DR of those individual photosites.]
A monochrome sensor with 20 million _photosites_ can have output downsampled to 5MP by summing 2x2 blocks of photosite output, and with uncorrelated noise this will give pixels with S/N twice as high as for simple "one pixel per photosite" output.
[Aside on film. The fine tonal gradations of a silver halide print which under a microscope is pure black dots on a pure white background indicate that "visual averaging" of the light reflected off a print can be enough to increase DR in this way. DR also seems to improve with lower degrees of enlargement, such as when making prints of the same size from larger formats. You effectively increase the output DR of prints made with the same emulsion by adding more "photosites" and printing at the same size.]
-
If these two cameras were tested using your target per your instructions of getting the centre square 100 pixels wide, then both cameras would appear to have the same DR according to you.
Let's examine what would happen in reality when both cameras are used to shoot a scene, artificial target or general view, keeping the FoV the same for both cameras.
I'm sure you would agree, if you were to take any specified crop of that scene, however small, there would be 4x as many pixels in the crop from the larger format camera.
DR also seems to improve with lower degrees of enlargement, such as when making prints of the same size from larger formats. You effectively increase the output DR of prints made with the same emulsion by adding more "photosites" and printing at the same size.
Here's the fallacy in your arguments: any technique you use to give the higher-resolution camera a DR advantage over the other camera (or the larger format of film) will negate the resolution advantage by a corresponding amount, and require additional exposure to boot. Yes, you can bin 4 pixels together to reduce noise levels, but then you lose the resolution gain, and still have to increase exposure by a factor of 4. I could probably get better DR out of a typical digicam than that of any MFDB pixel if I binned the whole sensor output down to a single pixel, but that would be a completely impractical and retarded thing to do. There is no free lunch here. The whole point of the exercise is to find out what kind of DR performance you can get without giving up resolution. You aren't gaining anything if you increase pixel quality by decreasing pixel quantity.
-
... and still have to increase exposure by a factor of 4. I could probably get better DR out of a typical digicam than that of any MFDB pixel if I binned the whole sensor output down to a single pixel, but that would be completely impractical and retarded thing to do. There is no free lunch here.[a href=\"index.php?act=findpost&pid=160964\"][{POST_SNAPBACK}][/a]
There's no free lunch. The consequence of using a larger format is always a requirement for greater exposure. The consequence of a greater exposure is a better signal-to-shot-noise, with my method, but not with your method.
As I understand it, the binning that takes place in relation to the sensor with the higher pixel count is in the viewing of equal size images, not at the exposure stage.
-
Perhaps you can tell me why
If the sensels are of equal size, then why would the same amount of light be distributed among four pixels?
-
There's no free lunch. The consequence of using a larger format is always a requirement for greater exposure. The consequence of a greater exposure is a better signal-to-shot-noise, with my method, but not with your method.
And you only get that increase in DR with your method at the cost of resolution. If you're measuring DR with a test that effectively throws away 75% or more of actual resolution, the results are bullshit. If I have a 16MP camera and want to test DR, I really don't care what the DR is if I reduce resolution to 4MP or 1MP or 1 pixel. Remember that the whole point of the test is to evaluate real-world, photographically meaningful DR, and binning an image down to 1/4 of its original pixel count is not real-world photographic practice.
-
If the sensels are of equal size, then why would the same amount of light be distributed among four pixels?
[a href=\"index.php?act=findpost&pid=160970\"][{POST_SNAPBACK}][/a]
For images of equal FoV at equal exposure, the same amount of light would be distributed amongst the four pixels of the larger format.
With Jonathan's method, you'd be using the larger format (in my example) at the same exposure and same physical aperture, same focal length lens and same distance in order to get the centre square 100 pixels wide.
That's not how we photographers use cameras in the real world for everyday shooting, is it?
ps. I just amended the above. Even I sometimes get confused .
-
For images of equal FoV at equal exposure, the same amount of light would be distributed amongst the four pixels of the larger format
1. Equal FoV means equal angle of view.
2. Four times the sensor size means twice the width and length.
3. The same angle requires the lens to be twice the distance from the larger sensor, i.e. the focal length has to be twice as long.
4. Equal exposure means equal product of the shutter time and aperture size (the latter relatively to the focal length).
5. The same aperture number with the longer focal length means larger aperture diameter, more light.
That was it.
-
1. Equal FoV means equal angle of view.
2. Four times the sensor size means twice the width and length.
3. The same angle requires the lens to be twice the distance from the larger sensor, i.e. the focal length has to be twice as long.
4. Equal exposure means equal product of the shutter time and aperture size (the latter relatively to the focal length).
5. The same aperture number with the longer focal length means larger aperture diameter, more light.
That was it.
[a href=\"index.php?act=findpost&pid=160975\"][{POST_SNAPBACK}][/a]
That was what? You've just given the reasons why larger formats require a greater absolute exposure. In my example, you have 4 pixels in the larger format in place of one pixel in the smaller format. You therefore need 4x the amount of light for a proper exposure of the larger format, with equal FoV.
Using Jonathan's method is like using a Canon 20D in exactly the same way as a 1Ds3, same lens, same f stop, same exposure, same distance to subject.
What's the point of that? All you demonstrate is that a 1Ds3 sensor cropped to the same size as a 20D sensor is essentially a 20D camera.
Editing again: 'What's the point of that?' is of course rhetorical. There may be a point. Since the 1Ds3 and 20D have approximately the same pixel density, one might be curious as to whether Canon had manged to increase the DR of the individual pixels in the 1Ds3, compared with invidual pixels in the 20D. Jonathan's method would be suitable for that purpose.
-
Editing again: 'What's the point of that?' is of course rhetorical. There may be a point. Since the 1Ds3 and 20D have approximately the same pixel density, one might be curious as to whether Canon had manged to increase the DR of the individual pixels in the 1Ds3, compared with invidual pixels in the 20D. Jonathan's method would be suitable for that purpose.
And my method is also suitable for discovering how well the 1Ds-MkIII will perform without dumbing it down to the resolution of the 20D. When you do the test on two cameras with unequal resolution with the test chart full-frame, the results from the higher-resolution are only valid if you reduce the resolution of the high-res camera to match the low-res camera. If I'm going to pay the price difference between a 20D and 1Ds-MkIII to get the extra resolution, then if I test the DR, I want to measure DR at 1Ds-MkIII resolution, not 20D resolution. Otherwise, there's no reason not to bin the resolution down to 1 pixel and claim a DR of 25 stops and lose the whole real-world-ness of the measurement.
-
And my method is also suitable for discovering how well the 1Ds-MkIII will perform without dumbing it down to the resolution of the 20D. When you do the test on two cameras with unequal resolution with the test chart full-frame, the results from the higher-resolution are only valid if you reduce the resolution of the high-res camera to match the low-res camera. If I'm going to pay the price difference between a 20D and 1Ds-MkIII to get the extra resolution, then if I test the DR, I want to measure DR at 1Ds-MkIII resolution, not 20D resolution. Otherwise, there's no reason not to bin the resolution down to 1 pixel and claim a DR of 25 stops and lose the whole real-world-ness of the measurement.
[a href=\"index.php?act=findpost&pid=160981\"][{POST_SNAPBACK}][/a]
And my method is also suitable for discovering how well the 1Ds-MkIII will perform without dumbing it down to the resolution of the 20D.
Jonathan,
Here's where I have difficulty in following you. When we make prints, or just view images on the monitor, we're usually rezzing up or rezzing down the image to a suitable size for our immediate purpose. If the issue is noise, which affects DR to some extent, then rezzing down the 1Ds3 image to the 20D file size should produce a slightly cleaner result, if we assume that noise at the pixel level is the same for both cameras.
On the other hand, if we rezz up the 20D file to the same size as the 1Ds3 file, then we are also amplifying the noise in the 20D image. The 1Ds3 image will then not only look cleaner, have slightly more DR (perhaps?) but also higher resolution.
Either way, dumming down or smartening up, the 1Ds3 should retain some advantage regarding noise and DR without sacrificing resolution to the 20D.
However, I'm prepared to accept that the differences regarding DR and noise might be very marginal. This would be an interesting experiment for someone who owns both a 1Ds3 and a 20D or 30D to carry out. Perhaps Mark Segal would like to try this, if he has a 20D.
He could use your method, first to determine if the DR of the 1Ds3 pixel is greater than that of the 20D pixel. Who knows? It might have a 1/4 of a stop greater DR. The photodiodes on the 1Ds3 sensor might be slightly larger even though the pixel pitch is the same as that of the 20D.
Having determined any difference in DR between the two cameras at the pixel level, we could then do a subjective assessment of DR and noise using images of scenes of high brightness range and equal FoV, examining various print sizes, or image sizes on monitor, but always comparing equal sizes whether print or image.
Anyone who's not confident in carrying out this test could send their 1Ds3 to me. I'll do it .
-
Jonathan,
Here's where I have difficulty in following you. When we make prints, or just view images on the monitor, we're usually rezzing up or rezzing down the image to a suitable size for our immediate purpose.
That is entirely irrelevant. When you make a 600-pixel web JPEG from a 5D image, do you keep a copy of the original resolution fille? Of course! And if you need to upsize for a really large print, do you downsize to 2MP first? Of course not. Why? Because you want to keep as many original pixels intact as possible, especially when making large prints and actual original pixels are in short supply anyway. If you want to make a 40x60cm print, the fact that downsampling your image file 4:1 will significantly reduce noise is completely irrelevant to you, because doing so will reduce image detail unacceptably. So the DR you might achieve by doing so is also completely irrelevant.
The reason I chose the 100-pixel size for the center white square is because the small text at that size has some fairly fine detail that will get lost in the noise when the noise level starts getting high enough to be visually objectionable, like the points on the asterisks in particular. So framing to that pixel size makes the legibility test fairly sensitive to noise.
[attachment=4278:attachment][attachment=4279:attachment]
What you're looking at is the chart sized so the white square is 100 pixels wide. By adding only 2% Gaussian noise, the smallest text has become much harder to read, and the points on the asterisks are pretty much obliterated.
Now if we re-frame the target so that the white square is 200 pixels wide, we can add 4% noise before the small text becomes similarly unreadable.
[attachment=4281:attachment]
Congratulations, we just magically increased DR by an entire stop! But at what cost? Nothing major really, just tossing out 3/4 of the resolution of the camera, turning 12 megapixels into 3. Here's the 100-pixel version again, with 4% noise added:
[attachment=4282:attachment]
As you can see, the small text is pretty much obliterated. You can't even find some of the asterisks any more, let alone make out their points. Resolution has been severely compromised by the noise. Each stop you inflate the DR measurement comes at the cost of halving linear resolution, or throwing away 3/4 of your pixel count. And that's not an acceptable trade-off in the vast majority of real-world photography.
-
I guess the short version of all of this is this: how much dynamic range can a camera capture at its full resolution? Below a certain threshold, noise does not significantly affect resolution; the limiting factors are things like AA filter strength, lens quality, and pixel count. But once noise reaches that threshold, it begins to become the primary limiter of resolution, and every stop noise increases above that threshold halves linear resolution. What I'm trying to determine with my testing methodology is usable DR at the threshold where the noise level is just barely starting to limit overall resolution.
-
I guess the short version of all of this is this: how much dynamic range can a camera capture at its full resolution? Below a certain threshold, noise does not significantly affect resolution; the limiting factors are things like AA filter strength, lens quality, and pixel count. But once noise reaches that threshold, it begins to become the primary limiter of resolution, and every stop noise increases above that threshold halves linear resolution. What I'm trying to determine with my testing methodology is usable DR at the threshold where the noise level is just barely starting to limit overall resolution.
[a href=\"index.php?act=findpost&pid=161026\"][{POST_SNAPBACK}][/a]
I understand all your arguments, Jonathan. I understand that downsampling results in less resolution and detail.
The essense of this discussion for me can be encapsulated by just one question (of two parts).
Is the dynamic range of a camera determined only by the DR of a single pixel of average specification, or does the total number of pixels on the sensor, of that same specification, have some bearing on the matter?
If it is true that there is no dynamic range advantage in having a greater number of pixels, as you maintain, then it should not be necessary to specify the pixel width of the inner square of your target, which does after all make the testing procedures more complicated. We have to measure the width of that square and do a calculation based on the total number of pixels on our sensor (in one dimension) in order to determine how far we should be from the target.
Simply framing the entire target to cover the entire width of the sensor, as I have done, is much easier. If anyone wants to compare my results with their own results using a different camera with a different pixel count, a different pixel density and perhaps a different format, what's the problem?
They can simply downsample my file to the same size as their own, or downsample their file to the same size as mine if their's is larger, provided they have also used the simpler method of fitting the whole target to the sensor width.
Since pixel count has no bearing on the matter, according to you, where's the flaw in that methodology?
If on the other hand, pixel count does have a bearing on the DR results, then this method that I am recommending will reveal such differences.
Your method will not.
-
I can think of some models of digicam that were replaced by a higher pixel count model and the new model performed worse than the old one overall because noise levels went up so much...
[a href=\"index.php?act=findpost&pid=160951\"][{POST_SNAPBACK}][/a]
I've never seen that. Not in the RAW data, anyway, which is all that really matters if we're talking about capture potential. I'm sure there may be cases where overall noise increased significanty, but it wasn't because the pixels were more and smaller, per se. It was probably because some quick and dirty readout speed was designed, or no one bothered with good microlenses on that model, or the NR just got more obnoxious.
There are many illusions that suggest deterioration of image quality with more and smaller pixels:
There is a noise-reduction race paralleling the megapixel race.
100% views get noisier.
Most downsizing algorithms used in viewers carry over individual original-pixel noise, by using nearest neighbor or a hybrid thereof.
Expectations of IQ rise. How many great photos (IQ-wise) do you see in the 3, 4 megapixel P&S cameras of yesteryear? Everyone talks about how much less noisy they were, yet when you go back and look at them, they look relatively sickly, compared to today's P&S cameras that do RAW, with all kinds of bizarre detail-killing artifacts.
-
Is the dynamic range of a camera determined only by the DR of a single pixel of average specification, or does the total number of pixels on the sensor, of that same specification, have some bearing on the matter?
If it is true that there is no dynamic range advantage in having a greater number of pixels, as you maintain, then it should not be necessary to specify the pixel width of the inner square of your target, which does after all make the testing procedures more complicated. We have to measure the width of that square and do a calculation based on the total number of pixels on our sensor (in one dimension) in order to determine how far we should be from the target.
Simply framing the entire target to cover the entire width of the sensor, as I have done, is much easier. If anyone wants to compare my results with their own results using a different camera with a different pixel count, a different pixel density and perhaps a different format, what's the problem?
They can simply downsample my file to the same size as their own, or downsample their file to the same size as mine if their's is larger, provided they have also used the simpler method of fitting the whole target to the sensor width.
Since pixel count has no bearing on the matter, according to you, where's the flaw in that methodology?
The problem with your method is that you're measuring DR with resolution effectively reduced to <1/2 megapixel when you fill the frame with the chart, regardless of how you upsize or downsize it after the fact. Download this ZIP file (http://www.visual-vacations.com/images/DRTest.zip) and look at the images in it and you'll see what I mean. The ZIP contains 4 image files: the the center square sized 50 pixels with 0% noise added, the center square sized 100 pixels with 2% noise, the center square sized 200 pixels with 4% noise, and the center square sized 400 pixels with 8% noise. All four images have the same amount of real resolution and detail, and are pretty much identical if you size them to equal pixel dimensions.
If you do a DR test your way with a Phase One P45+ MFDB, you're measuring the DR you can get when you've thrown away approximately 79 in 80 of your original pixels. And unless you only ever use your back to make web JPEGS, the DR measurement you'll get by doing so will be at least 3 stops greater than the DR you'll get in actual full-resolution images. Conducting the test my way will measure the DR you can achieve without throwing away most of your resolution. If you're ever going to print large enough that you send a full-resolution or upsampled file to the printer, then my method (framing to size the center square 100 pixels) is the only one that is going to deliver realistic results.
-
If you do a DR test your way with a Phase One P45+ MFDB, you're measuring the DR you can get when you've thrown away approximately 79 in 80 of your original pixels. And unless you only ever use your back to make web JPEGS, the DR measurement you'll get by doing so will be at least 3 stops greater than the DR you'll get in actual full-resolution images. Conducting the test my way will measure the DR you can achieve without throwing away most of your resolution. If you're ever going to print large enough that you send a full-resolution or upsampled file to the printer, then my method (framing to size the center square 100 pixels) is the only one that is going to deliver realistic results.
[a href=\"index.php?act=findpost&pid=161058\"][{POST_SNAPBACK}][/a]
I don't have to throw away resolution. If you think this is not a good idea, we could uprez the smaller file to the same size as the larger file before comparing DR.
In fact, this is probably the more realistic method. I find I am often uprezzing 5D or 20D files beyond their native size/resolution at 240ppi, in order to make larger prints.
Since larger prints are usually viewed from a greater distance, I'm quite relaxed about this approach.
-
I don't have to throw away resolution. If you think this is not a good idea, we could uprez the smaller file to the same size as the larger file before comparing DR.
Bullshit. Look at the ZIP file I posted (http://www.visual-vacations.com/images/DRTest.zip). Up or downsampling the image files after the shot is taken is meaningless. When you let noise rise to the point where the smallest text is poorly legible (as shown in the example images in the ZIP), the effective resolution of the four quadrants is about 400x400 pixels total, no matter how many original pixels are in the image. All 4 images in the ZIP are equally legible and all 4 images have the same resolution. The increased noise level cancels out any resolution advantage of the images with greater pixel counts.
Go back to the full-frame test shots you've already taken, and find the one that most closely matches the legibility of my 4 example images. Now crop to the edges of the chart as closely as you can, and downsize that crop using Bicubic to match each of my test images. You'll find that you can go all the way down to 400x500 pixels without decreasing the overall detail of the image or the legibility of the text any further. Why? The crop never had more than 400x500 pixels worth of detail in the first place, because the noise level had already obliterated most of the detail in the original image.
If you shoot my target full-frame with any 3:2 aspect-ratio camera and measure DR based on the legibility of the smallest text, the only way you can get that DR in real-life images is to downsize them to 400x600 pixels before doing any cropping.
In fact, this is probably the more realistic method. I find I am often uprezzing 5D or 20D files beyond their native size/resolution at 240ppi, in order to make larger prints.
In fact it makes your method totally invalid. Your upsized prints will have at least 2-3 stops less DR than your test will predict.
-
Here's the fallacy in your arguments: any technique you use to give the higher-resolution camera a DR advantage over the other camera (or the larger format of film) will negate the resolution advantage by a corresponding amount, and require additional exposure to boot. Yes, you can bin 4 pixels together to reduce noise levels, but then you lose the resolution gain, and still have to increase exposure by a factor of 4.[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=160964\")
But that's another fallacy. You don't have to bin. I can't speak for others, but when I use binning as a reference, it is only to show that the fine, high-res capture contains what a lower-res would, IQ-wise. That's just a half-victory to my full argument, which is that the image is better without binning at all. The only time binning is better, IQ-wise (storage/transmission concerns aside) is when the highest subject frequency recorded is well below the nyquist, in which case no resolution losses are concerned. Did you do the experiment I suggested? Here's an composite image to help you along. This is a 100% crop of the green channel of Ray's under-exposure of your chart, and then 2x2, 3x3, and 4x4 binnings. Look at them from up close, and at a distance. There is no practical noise or DR benefit to the binning. Cut and paste the image into photoshop, and make a little rectangle outside the numbers, and look at the standard deviation in the histogram. Then, move the rectangle to the other quarters of the image. You can see that the standard deviation changes drastically from quarter to quarter, but the practical noise and DR doesn't change. The level of noise here obscures fine detail, but in a well-exposed area, again, the noise would be *practically* the same throughout the image, but resolution would be lost to the big pixels.
[a href=\"http://www.pbase.com/jps_photo/image/90404705/original]http://www.pbase.com/jps_photo/image/90404705/original[/url]
I've said it before, and I'll say it again; big pixel IQ is a big lie.
Big pixels only increase IQ when you increase the sensor size, too, and collect a proportionate amount of photons. Unfortunately, read noise does not decrease at a per-pixel level as well as shot noise does, with current technology. Real world cameras doing what my bins did would actually have more read noise.
-
You have some noise patterns, but no detail. What resizing did you use, nearest neighbor? Look at my images in the ZIP file, and explain to me which one has better resolution of the text in the quadrants. Show your work.
-
You have some noise patterns, but no detail. What resizing did you use, nearest neighbor? Look at my images in the ZIP file, and explain to me which one has better resolution of the text in the quadrants. Show your work.
[a href=\"index.php?act=findpost&pid=161083\"][{POST_SNAPBACK}][/a]
You don't see the numbers 0 through 6, when standing 10 feet back from the monitor? Strange; I can see them fairly clear. The vertical banding messes up the 3 quite a bit (I removed the horizontal banding to some degree; if the target had been shot against a black background with black borders on all sides, I could have removed more banding, both vertical and horizontal).
I pixelated, which is the same thing as binning except that it does not extend the bit depth. There are only 8 meaningful levels in the image, though, from the original RAW, so that should not be an issue here. I've done stuff like this promoted to 16-bit, and didn't see any difference. The only reason that there are more than 8 values in the histogram of the non-pixelated version is that I subtracted the mean of each horizontal line from it after stretching the 8 RAW levels to 255, to reduce the banding, something that only works when the area's detail is small compared to the dynamics of the noise.
I just downloaded your Zip, and can't make any sense out of it. You have one JPEG 400*500 with no noise, and 3 at 800*1000 with different amounts of noise, with the same FOV in each. Yes, the ones you added more noise to have more noise, as expected, but what is your point?
Your text saying that the three images with noise added have the center square at different pixel sizes contradicts the actual pixel dimensions.
-
I just downloaded your Zip, and can't make any sense out of it. You have one JPEG 400*500 with no noise, and 3 at 800*1000 with different amounts of noise, with the same FOV in each. Yes, the ones you added more noise to have more noise, as expected, but what is your point?
Your text saying that the three images with noise added have the center square at different pixel sizes contradicts the actual pixel dimensions.
My error, I accidentally uploaded a version of the ZIP with the wrong files in it. Try downloading again, please.
My point is that no matter what the pixel dimensions of the original capture, that if you use the illegibility of the smallest text as your baseline for defining the noise floor when running the DR test I propose, the noise level will be so high that the effective resolution of the capture area occupied by the chart will be reduced to about 400x500 pixels (high contrast black/white edges excepted). So if you fill the frame with the chart as Ray suggests, then the DR measurement you get from the procedure will only be achievable in practice for low-contrast image detail if you downsize the full frame to approximately 400 pixels in the narrow dimension. If you do not do so, then the measured DR will be 2-3 stops greater than that achievable in real-world images.
OTOH, if one conducts the test with the center white square 100 pixels wide in the original capture as I propose, then the measured DR will more accurately reflect what is achievable during the course of normal and customary photographic practice. If you measure 7 stops of usable DR using my method, then you can be confident that your camera will capture detail in a window curtain and a cabinet door that meter 7 stops apart if exposure is set properly. Ray's method might measure 9-10 stops of DR with the same camera, and cause disappointment when the camera fails to hold detail in two objects that meter 9 or 10 stops apart. The point of my procedure is to measure DR in a way that matches real-world experience, so that if you measure 8 stops of DR, that you can simultaneously hold detail in two parts of a single exposure thet meter 8 stops apart from each other.
-
My error, I accidentally uploaded a version of the ZIP with the wrong files in it. Try downloading again, please.
My point is that no matter what the pixel dimensions of the original capture, that if you use the illegibility of the smallest text as your baseline for defining the noise floor when running the DR test I propose, the noise level will be so high that the effective resolution of the capture area occupied by the chart will be reduced to about 400x500 pixels (high contrast black/white edges excepted). So if you fill the frame with the chart as Ray suggests, then the DR measurement you get from the procedure will only be achievable in practice for low-contrast image detail if you downsize the full frame to approximately 400 pixels in the narrow dimension. If you do not do so, then the measured DR will be 2-3 stops greater than that achievable in real-world images.
No matter how you slice it, you are concerning yourself with the DR of the pixels, and ignoring the DR of the image. They are both interesting, and the the latter is the most important, I think, for most photography, especially when you consider the fact that more pixels is always better, pixel DR being equal.
OTOH, if one conducts the test with the center white square 100 pixels wide in the original capture as I propose, then the measured DR will more accurately reflect what is achievable during the course of normal and customary photographic practice. If you measure 7 stops of usable DR using my method, then you can be confident that your camera will capture detail in a window curtain and a cabinet door that meter 7 stops apart if exposure is set properly. Ray's method might measure 9-10 stops of DR with the same camera, and cause disappointment when the camera fails to hold detail in two objects that meter 9 or 10 stops apart. The point of my procedure is to measure DR in a way that matches real-world experience, so that if you measure 8 stops of DR, that you can simultaneously hold detail in two parts of a single exposure thet meter 8 stops apart from each other.
[a href=\"index.php?act=findpost&pid=161103\"][{POST_SNAPBACK}][/a]
But looking at the DR of the pixels is not how to do that. Carry your thinking over to film, and you would be testing the DR of two different sizes of the same film with the same area of film covering the chart; only the optics would differ, and none of the benefits of the larger film would be realized. Also, if you were comparing two films of the same format, but of different grain sizes, your method would require using different distancews or lenses to make the grain size proportional to the chart size in both cases.
The real world of photography is about the image, not individual pixels. Individual pixel quality is only one factor in the image quality.
-
No matter how you slice it, you are concerning yourself with the DR of the pixels, and ignoring the DR of the image. They are both interesting, and the the latter is the most important, I think, for most photography, especially when you consider the fact that more pixels is always better, pixel DR being equal.
The real world of photography is about the image, not individual pixels. Individual pixel quality is only one factor in the image quality.
But individual pixel quality is being completely ignored with your methodology.
(Image quality) = (pixel quality) * (pixel quantity).
When noise levels become high enough that an individual pixel no longer contains meaningful image detail and you have to start averaging or binning them together to see the actual image data under the noise, pixel quality has been flushed down the toilet. Ignoring the effect of noise on individual pixels and looking only at the entire image is exactly the current flawed methodology that predicts grossly optimistic DR figures that are not achievable in real-world photography. In the real world, noise levels are deemed unacceptable when noise levels start causing a deterioration of resolution in low contrast subject matter such as the texture of cloth, hair, and skin, or the woodgrain pattern in furniture, etc.
The exposure range in which such subtleties can still be successfully resolved is of far more practical value to the photographer than the exposure range in which a few large, high-contrast objects can still be picked out from the background noise. My approach measures the former, yours only the latter. My approach has far more real-world relevance than yours.
If you want to dispute this, try the following experiment: Conduct the test twice, the first time with the chart filling the frame, and the second time with the chart farther away so that the white square is 100 pixels across. During both tests, have something with fine detail in the frame other than the chart, such as a doll, potted plant, or whatever, so that you have some kind of quasi-real-life subject matter in-frame besides the chart. Post the frames from both tests where the smallest chart text legibility most closely matches the sample images in the ZIP file (http://www.visual-vacations.com/images/DRTest.zip). Looking at the doll or whatever you put in the frame besides the chart, which image has a noise level you would consider acceptable to deliver to a paying client? Post 100% crops of both the chart and the doll or whatever, or better yet, DNGs.
I look forward to seeing the results with great interest.
-
But individual pixel quality is being completely ignored with your methodology.
Just to get things straight, I do not believe that it's not useful to measure pixel performance. I am merely stating that the DR of the pixel is not the DR of the camera. It is only the DR of the pixel. Both should be measured. However, in your comments there is an implication that individual pixel noise limits the DR of the camera. It clearly does not. If it did, a camera with 12 pixels would have the same DR as a camera with 12 billion pixels, all with the same shot noise and read noise.
(Image quality) = (pixel quality) * (pixel quantity).
The square root of pixel quantity.
When noise levels become high enough that an individual pixel no longer contains meaningful image detail and you have to start averaging or binning them together to see the actual image data under the noise, pixel quality has been flushed down the toilet.
Nope. Your dismissive terminology does not render the pixels useless. There is no threshold upon which a pixel becomes totally useless. You simply need more of them, and you do get sufficiently more of them if you've been subdividing into smaller pixels.
Ignoring the effect of noise on individual pixels and looking only at the entire image is exactly the current flawed methodology that predicts grossly optimistic DR figures that are not achievable in real-world photography.
No. That's another problem, entirely. The bigger problem is that no one has bothered to come up with a standard way of dealing with resolution vs noise, because most of the more scientifically-oriented people are missing the forest for the trees and measuring pixels, while the people looking at "images" aren't even accounting for RAW conversion issues. Both are barking up the wrong tree.
In the real world, noise levels are deemed unacceptable when noise levels start causing a deterioration of resolution in low contrast subject matter such as the texture of cloth, hair, and skin, or the woodgrain pattern in furniture, etc.
People are not very clever in the real world. They get fooled very easily.
The real fact of the real world is that noise does not limit resolution any worse than having big pixels does, so the effect is moot. Smaller pixels do, however, allow higher resolution for high contrast or tonal ranges where noise is not a big problem.
The exposure range in which such subtleties can still be successfully resolved is of far more practical value to the photographer than the exposure range in which a few large, high-contrast objects can still be picked out from the background noise.
Most of the time, yes, but the principles are similar. You do not get better resolution by trading for bigger pixels with less pixel noise. It does not happen. In less (image-)noisy tonal ranges, however, the higher resolution of the smaller pixels will resolve more detail.
My approach measures the former, yours only the latter. My approach has far more real-world relevance than yours.
Really? Let's take your approach to the extreme. Let's say you use a camera that only has enough pixels to make the white square 100 pixels, when the target fills the entire frame. You also have a camera with pixels the same size, pitch, QE, capacity, and read noise, but 100x as many of them, and shoot the target with the center crop. What have you just tested? You have just tested the pixels, which are exactly the same. To say that both have the same dynamic range, as cameras, is ridiculous, because with the proper lenses for equal FOV, the camera with 100x as many pixels will show details much farther down into the shadows.
If you want to dispute this,
I'm not sure your "this" refers to anything I actually believe, but I'll try to follow along.
try the following experiment: Conduct the test twice, the first time with the chart filling the frame, and the second time with the chart farther away so that the white square is 100 pixels across. During both tests, have something with fine detail in the frame other than the chart, such as a doll, potted plant, or whatever, so that you have some kind of quasi-real-life subject matter in-frame besides the chart. Post the frames from both tests where the smallest chart text legibility most closely matches the sample images in the ZIP file (http://www.visual-vacations.com/images/DRTest.zip). Looking at the doll or whatever you put in the frame besides the chart, which image has a noise level you would consider acceptable to deliver to a paying client? Post 100% crops of both the chart and the doll or whatever, or better yet, DNGs.
I look forward to seeing the results with great interest.
[a href=\"index.php?act=findpost&pid=161453\"][{POST_SNAPBACK}][/a]
What does this have to do with anything I've said? Obviously, the subject detail is going to be less when the subject fills less of the frame, exposure remaining equal. When the target fills the frame, you can get the same resolution of the smallest text with a lower exposure.
Let me refresh what I am arguing (and Ray, and some others); measuring the DR range of the pixel is only about the pixel. It only means something to the DR of the image when the number of them are also taken into account.
I don't have a working printer right now, so I can't print your target. If you made it clear what you think you are proving, I could comment, though.
-
I think the heart of the dispute lies in the subjective assumption of what the final deliverable product will be. A lot of the MFDB guys are having to downsize for final delivery, and I'm guessing it's an issue for 1Ds Mk II/III shooters as well.
Are you making a 16x20 fine art print? Then Jonathan's assumption that pixel quality reigns supreme carries some merit.
Are you doing a 1/2 page spread for magazine reproduction? You certainly don't need 16 or 22 megapixels to do that and you can't ignore the DR/noise advantage that a high resolution camera gives when downsizing the final file.
Perhaps there needs to be a third metric in Jonathan's test that accounts for the difference. What about shooting full frame, then resizing to a standardized 8x10 at 300dpi (or any other standard print size)?
-
I think the heart of the dispute lies in the subjective assumption of what the final deliverable product will be. A lot of the MFDB guys are having to downsize for final delivery, and I'm guessing it's an issue for 1Ds Mk II/III shooters as well.
Are you talking about downsampling just to reduce noise? It is utterly ridiculous if anyone has to do that. It shows a visual naivite on the part of the recipient. If you're talking about a limited resolution medium for display, then of course downsampling is necessary at some stage, and you might want to excercise quality control by doing it yourself. It should ideally be done in the conversion process, especially with Canons, as Canons have RAW data that is ideal for downsampling near-blacks with a minimum of (chromatic) noise.
Are you making a 16x20 fine art print? Then Jonathan's assumption that pixel quality reigns supreme carries some merit.
The point I have been trying to make is that unless you have your face so deep into the pixels that each is clearly distinct, pixel quality is irrelevant, except insofar as it affects image quality as a single factor (and bigger, cleaner pixels would look like a mosaic with similar overall noise). Otherwise, I have never seen a single demonstration that shows lower noise for bigger but fewer pixels or binnings or downsamples. It's just a mantra that many people repeat, without ever demonstrating its validity.
Are you doing a 1/2 page spread for magazine reproduction? You certainly don't need 16 or 22 megapixels to do that and you can't ignore the DR/noise advantage that a high resolution camera gives when downsizing the final file.
Perhaps there needs to be a third metric in Jonathan's test that accounts for the difference. What about shooting full frame, then resizing to a standardized 8x10 at 300dpi (or any other standard print size)?
[a href=\"index.php?act=findpost&pid=162006\"][{POST_SNAPBACK}][/a]
That would standardize the "image noise" issue well, but is unfair to the camera(s) that lose more resolution in the downsampling. That's why upsampling is preferable, IMO; then all cameras can show their full resolution and noise.
-
The metric I'm using is that I don't want to base my assessment of DR performance on an assumption of reducing the effective resolution of my camera. Yes, I know that one can sacrifice resolution for DR, but in most cases, I don't want to. So for me, knowing how much DR I can get without sacrificing resolution is if far more interest than knowing what I can get if I reduce resolution to web-JPEG levels. I expect a DR specification to hold true for the largest prints I expect to make, not the smallest. If I choose to trade away resolution for additional DR, I can still do so, but assuming that I will find that trade-off acceptable in every case is stupid.
Printing more native-resolution pixels will increase print quality until you pass 300-360 PPI, depending in the specific printer brand and print process. For a 12x18" print (which is NOT an unreasonably large size; well within the capabilities of the hobby-level photographer) that translates to ~19.5-28MP of native camera resolution before further increases cease to offer additional visible improvement. For prints that size, there's no reason for someone shooting with a 1Ds-MkIII or below to downsize before printing, and noise levels high enough to compromise pixel quality will degrade the image quality of such prints.
-
(Image quality) = (pixel quality) * (pixel quantity).
The square root of pixel quantity.
Better double-check your math there, your statement is only true if you're talking in terms of linear measurements in a single dimension, which I'm not. I'm talking about the quality of the image as a whole.
-
Better double-check your math there, your statement is only true if you're talking in terms of linear measurements in a single dimension, which I'm not. I'm talking about the quality of the image as a whole.
[a href=\"index.php?act=findpost&pid=162105\"][{POST_SNAPBACK}][/a]
No, I meant what I said. That is exactly how noise resamples/bins.
-
No, I meant what I said. That is exactly how noise resamples/bins.
And I'm talking about image quality when you DON'T resample or bin. Given pixels of a certain quality, if you double the pixel count, you can double the image area (note I'm NOT saying linear dimensions!) while maintaining a constant image quality per unit of area.
-
And I'm talking about image quality when you DON'T resample or bin.
Me too. When did I ever imply that binning or resampling is necessary to increase image DR?
Given pixels of a certain quality, if you double the pixel count, you can double the image area (note I'm NOT saying linear dimensions!) while maintaining a constant image quality per unit of area.
[a href=\"index.php?act=findpost&pid=162132\"][{POST_SNAPBACK}][/a]
Yes, and viewed at the same size, it can have 1.414x the DR (a half stop more), without any resampling to a resolution lower than the original.
-
Me too. When did I ever imply that binning or resampling is necessary to increase image DR?
You've only talked about adding pixels to increase DR in every single one of your posts. Here's a novel thought: perhaps when I increase pixel count, I wish to increase "image resolution" and not "image DR". To do that, I need to set a consistent per-pixel noise/quality standard.
Yes, and viewed at the same size, it can have 1.414x the DR (a half stop more), without any resampling to a resolution lower than the original.
But only at the cost of NOT increasing resolution. You can have one or the other, but you don't get to have both. There is no free lunch here. Once noise starts negatively affecting resolution, you can add pixels to increase resolution or increase DR, but to the extent that you do one, you lose the ability to do the other. Is that so hard to understand?
-
It is great that you are around to answer these questions. But I have to say Jonathan, you must have stamina because it would drive me crazy. What's more, no one seems to appreciate your wit.
-
I'm currently at Walter Reed Army Medical Center getting medical treatment for some neurological problems, so I have more free time than I normally do, and not a lot of stuff to occupy it. The blog in my signature has more details if you feel so inclined...
-
I'm currently at Walter Reed Army Medical Center getting medical treatment for some neurological problems, so I have more free time than I normally do, and not a lot of stuff to occupy it. The blog in my signature has more details if you feel so inclined...
[a href=\"index.php?act=findpost&pid=162170\"][{POST_SNAPBACK}][/a]
I read it and PM'd you (fyi).
-
Having read the recent postings in this thread, I still can't grasp Jonathan's concept here.
I can see a reason for carrying out the test in the manner he suggests in order to obtain specific information about individual pixel performance. I've always found it curious that the larger pixels of the 1Ds seemed to exhibit more noise than the smaller pixels in the D60. However, because the 1Ds was a later model, high ISO performance was better than the D60, just as the next model, the 10D, had even better high ISO performance, a trend which was continued with the 20D.
But at ISO 100, the D60 apparently has less noise than the 1Ds. Does this mean that at ISO 100 and at the pixel level this 6mp, cropped format D60 has slightly more dynamic range than the 1Ds?
I don't know, but one way to find out would be to shoot Jonathan's target in the manner he suggests, with both cameras. But my reasoning tells me that, just as the total image noise in a 1Ds image is less than in a D60 image, the dynamic range of the 1Ds image might also be slightly greater.
We should not forget that an artificial target like the one designed by Jonathan is still essentially a real world image. We like to make a distinction between test charts and real world scenes. But they are of course both real world scenes.
The large B&W text in Jonathan's 'Dynamic Range Test Chart' would be quite analagous to some large B&W text on a bill board in any street scene in a city or on the highway. It would be the last detail to become unrecognisable with extreme underexposure. The smaller and fainter numbers in the centre of the chart are analagous to any fine and low contrast detail in any so-called real scene and such detail would be the first to become unrecognisable with extreme underexposure.
-
Having read the recent postings in this thread, I still can't grasp Jonathan's concept here.
Let me start from the beginning with the foundational equation:
(Image quality) = (pixel quality) * (pixel quantity).
Lets start with pixel quality, since that is the hardest to define. Pixel quality can range from 0 to 1. Quality 0 is visually meaningless image data such as pure noise, gross blur, etc, with a signal/noise ratio of 0. No matter how many quality 0 pixels you have, you cannot discern a meaingful image from them.
Quality 1 is perfect; each pixel is as good as it can possibly be with regards to detail, resolution, etc. If you take a MFDB capure made at base ISO with optimal exposure, focus, lighting, and post-processing, and size that image down to an 800-pixel TIFF, the resulting pixels will have a quality level very close to 1, perhaps 0.997 or something like that.
If you look at the images in this ZIP file (http://www.visual-vacations.com/images/DRTest.zip), you can see a demonstration of this. The first image (0) has a pixel quality very close to 1; it was downsized from an image 8x larger in linear dimensions, so each of its pixels are derived from 64 source image pixels. The number of pixels is the only thing limiting the quality of this image.
Image 2, in contrast, has 4x the pixel count of image 0, but overall resolution and detail are practically identical. The noise level is such that pixel quality is ~0.25, or 1/4. Since the pixel quality of this image is 1/4 that of image 0, it needs 4x as many pixels to match the overall image quality.
Image 4 continues the progression. It has 16x the pixel count of image 0, but double the noise level of image 2, so it has the same overall image quality as both 0 and 2. So it has a pixel quality of ~0.0625, or 1/16.
And then there's image 8. It has 64x the pixel count of image 0, and twice the noise of image 4, so again it has the same overall image quality. So pixel quality is ~0.015625, or 1/64.
Different individuals' tastes will come into play at this point, but once pixel quality gets down to about 0.25 or so, the noise or whatever other image artifacts are causing that quality loss starts being noticeable in decent-sized prints ("decent-sized" meaning one sends the full-resolution file to the printer and the resulting PPI is <300-360). So if noise/grain is not part of your creative vision, and you want to be able to make decent-sized prints of your work, then you need to maintain a pixel quality of ~0.25 or greater.
If the above conditions apply to you, then measuring DR with any methodology that nets a pixel quality <0.25 is invalid for you. Measuring DR using the B&W text as your reference, or shooting the chart full-frame (or worse, both!) will measure DR at a pixel quality level far below 1/64 (0.015625) and the noise levels will be intolerable for real-world images. If you were to shoot a portrait for a paying client with the noise level present in the underexposed shot you posted at the beginning of this thread, do you think the client would be pleased? Would you even be able to recognize the client's face? I doubt it, on both counts. So does that test correlate meaningfully to real-world shooting conditions? No. In contrast, if you were to reshoot the test as I specified, composed with the center square 100 pixels wide, if you choose the resulting frame that most closely matches image 2 in the ZIP file, the noise level you'll find in the image overall would be acceptable to most people, even when printed "decent-sized".
I recognize that not everyone has the same noise tolerance, which is why I put various sizes of text in the chart. Each successively larger line of text corresponds to an additional 1/2-stop of noise, so if you like lots of noise in your images, use the third-smallest line of text instead of the smallest for your legibility threshold, and you'll get a DR reading that is 1 stop greater than if you were to use the smallest line. If that methodology works for you and your style, great, but at least there is a common baseline for comparison for someone with a more stringent noise tolerance than you, and both of you have something to work with that accurately reflects the results you get shooting in real-world situations.
Using the legibility test of the lines of text in the quadrants as a guide (center square 100 pixels wide), the approximate corresponding pixel quality values start at 1/4 for the smallest line of text, and for each successively larger line, pixel quality is divided by 2. When conducting the test, have other elements on the composition so you can evaluate what pixel quality level is the minimum you or your clients will accept. If you decide that your minimum acceptable pixel quality level is 1/8 instead of 1/4 based on pixel peeping at 100% or printing the test images "decent sized", there is nothing wrong with that. As long as you include that in your result, then I can easily correlate that to my work and tastes, and have a good idea if your camera is suitable for my purposes. At least there is a standardized methodology involved so that the results achieved by different testers with different preferences can be meaningfully compared.