Luminous Landscape Forum
Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: marcmccalmont on December 27, 2007, 03:55:50 pm
-
The following are links to both 5D and P30 RAW files
5D 60mm f9 iso 100
p30 80mm f12 iso 100
5D;
http://download.yousendit.com/648514375875EFC8 (http://download.yousendit.com/648514375875EFC8)
http://download.yousendit.com/4CE5D23F2F045A91 (http://download.yousendit.com/4CE5D23F2F045A91)
http://download.yousendit.com/0494FB603F426A30 (http://download.yousendit.com/0494FB603F426A30)
P30;
http://download.yousendit.com/DB179C8A62750E5E (http://download.yousendit.com/DB179C8A62750E5E)
http://download.yousendit.com/3A99538D7688B545 (http://download.yousendit.com/3A99538D7688B545)
http://download.yousendit.com/788812D41467C4AD (http://download.yousendit.com/788812D41467C4AD)
I think (using Breezebrowsers histogram) 2035 and 19371 are the best exposures for comparison
Marc
P30 [attachment=4404:attachment]
5D [attachment=4405:attachment]
-
The following are links to both 5D and P30 RAW files
[a href=\"index.php?act=findpost&pid=163433\"][{POST_SNAPBACK}][/a]
Well, I've been looking at the first P30 RAW file. This is what I see:
1) Low-ISO pixel-level read noise (relative to maximum RAW value) is almost exactly the same as the Canon 1Dsmk2, 1Dsmk3, 1Dmk2, 1Dmk3 and Nikon D3 (all are about 0.6 stops lower than the 5D).
2) As I would expect with this level of read noise, the 4 least significant bits are pretty much noise only, and a waste of time and space. I can't distinguish a version interpolated to RGB with an extreme push in the shadows in 16-bits and quantized to 12 bits (before interpolation).
I have yet to try to measure shot noise, but I need to restart my computer now because its acting strange (Firefox froze up and then disappeared), and want to get this thread moving a little with these comments.
-
Well, I've been looking at the first P30 RAW file. This is what I see:
1) Low-ISO pixel-level read noise (relative to maximum RAW value) is almost exactly the same as the Canon 1Dsmk2, 1Dsmk3, 1Dmk2, 1Dmk3 and Nikon D3 (all are about 0.6 stops lower than the 5D).
2) As I would expect with this level of read noise, the 4 least significant bits are pretty much noise only, and a waste of time and space. I can't distinguish a version interpolated to RGB with an extreme push in the shadows in 16-bits and quantized to 12 bits (before interpolation).
I have yet to try to measure shot noise, but I need to restart my computer now because its acting strange (Firefox froze up and then disappeared), and want to get this thread moving a little with these comments.
[a href=\"index.php?act=findpost&pid=163479\"][{POST_SNAPBACK}][/a]
Since the least significant four bits are mostly noise (as one would expect) how can you (easily) turn them into zeros in your file (nothing should be better than noise?)
Marc
-
I have yet to try to measure shot noise, but I need to restart my computer now because its acting strange (Firefox froze up and then disappeared), and want to get this thread moving a little with these comments.
[a href=\"index.php?act=findpost&pid=163479\"][{POST_SNAPBACK}][/a]
Well, I rebooted, and Firefox remembered where it was. I could have probably just started firefox again, but I haven't rebooted this PC in months, so I figured I'd get a fresh start.
Well, I just searched the P30 RAW for areas of OOF flat areas, and they consistently point to about 27K photons at RAW saturation at ISO 100. Let's look at some of the photon collections of current cameras at their lowest "full DR" ISOs:
1Dsmk3 - 60K at ISO 100 * 22.1MP = 1326 GPh (gigaphotons)
P30 - 27K at ISO 100 * 30MP = 810 GPh
D3 - 53.5K at ISO 200 * 12.1MP = 647 GPh
5D - 52.3K at ISO 100 * 12.8MP = 669 GPh
I don't know what the maximums are for the D3 and 1Dsmk3 at their "low"
ISOs (100 and 50, respectively, AFAIK), or anything the P30 has below ISO 100, but they may collect more photons in them. The 5D's maximum is 79.9K * 12.8MP = 1023 GPh.
Like most MFDBs whose RAW data I've looked at, the P30 collects a lot less photons than are possible with the pixel density. MFDBs in general seem to collect less photons per square mm than DSLRs and compact P&S cameras, at a given exposure rate.
-
Since the least significant four bits are mostly noise (as one would expect) how can you (easily) turn them into zeros in your file (nothing should be better than noise?)
Marc
[a href=\"index.php?act=findpost&pid=163483\"][{POST_SNAPBACK}][/a]
No; turning them to zeros technically makes them noisier. The point is, however, that the amount that zeroing them makes them noisier is relatively insignificant, compared to the analog noise present in the image, so there is little point in storing them, unless you're going to do something like add a whole bunch of images together, in which case the tiny amount of signal present in each added image will sum into slightly more usable signal.
-
I wrote:
No; turning them to zeros technically makes them noisier. The point is, however, that the amount that zeroing them makes them noisier is relatively insignificant, compared to the analog noise present in the image, so there is little point in storing them, unless you're going to do something like add a whole bunch of images together, in which case the tiny amount of signal present in each added image will sum into slightly more usable signal.
[a href=\"index.php?act=findpost&pid=163487\"][{POST_SNAPBACK}][/a]
Here's RAW levels 0 through 360 out of about 0 - 65535; an effective push from ISO 100 to about ISO 4500, including some white balancing:
(http://www.pbase.com/jps_photo/image/90879928/original.jpg)
One half of this image is 16-bit; the other was quantized to 12-bit first before color interpolation and WB (the one on the right was flipped horizontally, to make comparison easier). Can you tell which half is which?
You can see that the noise in the image is what limits IQ, not 12 bits. When you get to higher ISOs, you don't even need 12 bits.
-
Can you tell which half is which?
I can't, but then you really can't tell accurate bit level on an 8bit video system which we all are stuck with.
It's still better than what you'ld get on film under similar exposure parameters IMO.
-
Can you tell which half is which?/quote]
I can't, but then you really can't tell accurate bit level on an 8bit video system which we all are stuck with.
It's still better than what you'ld get on film under similar exposure parameters IMO.
[a href=\"index.php?act=findpost&pid=163521\"][{POST_SNAPBACK}][/a]
Why would you expect 8-bit displays to get in the way? This is only about 8.33 bits (361 original RAW levels) in the 16-bit image, and 4.33 bits (22.6 quantized levels) in the 12-bit version. The 8-bit display is equivalent to about 15.5 bits in the original. Remember, this is a push to about ISO 4500 from ISO 100.
-
John,
I can't see any difference between the two halves. I don't even know what to look for.
I see evenly distributed chroma noise throughout and definition of detail looks the same as well.
-
John,
I can't see any difference between the two halves. I don't even know what to look for.
I see evenly distributed chroma noise throughout and definition of detail looks the same as well.
[a href=\"index.php?act=findpost&pid=163622\"][{POST_SNAPBACK}][/a]
Well, that's my point. You can clearly see the noise, and the 16 bits of one half (15.5 visible; I should have pushed a little more and used 0 - 255 of the original 16-bit levels to keep things simpler). Therefore, any subtle benefit of 16 bit is lost in the capture, due to the noise.
Companies aren't friends. I'm sure that they usually wait until they are improving the IQ at the analog level, if possible, and then introduce a higher bit depth, so that the public attributes the higher IQ to the higher bit depth.
The reality is that the analog aspects of reading out sensors are not ready yet for more than 12 bits. More bits may be useful to the astrophotographer who is stacking multiple images (but they are probably using Canons, which are the only cameras whose RAWs, AFAIK, are unclipped and linear near black), but for the rest of us taking one shot at a time, it's just a waste of storage space and time. The most ridiculous RAW I've seen is the Panasonic; my FZ50 compact ZLR has 12 bits of RAW data, only worthy of 11 bits at ISO 100, but stores the RAWs as 16-bit RAWs with 4 trailing zeros for each pixel. They weigh in at 20MB each (and take over 5 seconds to write), but I have compressed them down to 3MB with advanced ZIP options.
-
I think (using Breezebrowsers histogram) 2035 and 19371 are the best exposures for comparison
CF019372 and _MG_2033 are the best shots. They are as far to the right as possible without clipping, and they are equal in exposure as far as possible.
CF019372 is a tiny bit less bright than _MG_2033, but increased already by 1/3 stop it becomes brighter than _MG_2033. Note, that one needs +1 EV in ACR because of a "pre-mediated" adjustment of -1 EV, which has been carried out by ACR without asking you and without telling you about it.
Although the highlights can't be analyzed (clouds are not very suitable to comparison between two shots), that does not pose any problem, as the stress is not on the details and noise in the highest range.
Unfortunately, the salt shaker is not in the shots, but the funny animals on the yellow pot with the snow flakes can be used as "fine detail".
I added 5 EV to both images and selected some originally dark areas. Based on the visual appearance, IMO the 5D's DR over 8 stops is a myth, while the P30 goes somewhat over 10 stops.
It is important to note, that the 5D has only 8 levels to spare for the 9th stop (and seven for anything beyound that). This explains the "good noise behaviour" of the 5D: the noise is low, because there are no levels to distinguish between.
A layered TIFF with some related screen shots can be downloaded from here, only 1.8 MB (http://www.panopeeper.com/Demo/DRcomparison_P30-5D_2.tif)
Finally, a small test for the value of the bit depth over 12. One of the layers show the P30 image with the four low ordeer bits shifted out, +5 EV. The same area selected as in the normal processing, the noise (in terms of standard deviation) is higher without the low order bits.
-
1Dsmk3 - 60K at ISO 100 * 22.1MP = 1326 GPh (gigaphotons)
P30 - 27K at ISO 100 * 30MP = 810 GPh
D3 - 53.5K at ISO 200 * 12.1MP = 647 GPh
5D - 52.3K at ISO 100 * 12.8MP = 669 GPh
John, where did you get this figures from?
-
John, where did you get this figures from?
[a href=\"index.php?act=findpost&pid=163708\"][{POST_SNAPBACK}][/a]
The first 3 I measured myself, the best I could with the files available, but the results were consistent enough to make me feel they are very close. The 1Dsmk3 was a little weird, though. At ISO 100, it seemed to have some kind of added noise, that limited SNRs in the highlights (my XTi has this type of effect, too; it never goes higher than about 100:1). I had to get the 1Dsmk3 values from ISO 100 shadow areas, and higher ISOs' highlight areas.
The 5D I got from Roger Clark, IIRC. Actually, that's the 1Dmk2 but they are supposed to be the same in almost every way, electrically at the pixel level.
It would be nice to have real test shots geared towards proper measurements, though. The many in-focus and textured RAW samples made available are not the most useful for measuring photons collected.
-
The first 3 I measured myself, the best I could with the files available
I wonder how you "measure" that. FYI: the well capacity of the P30 is 60 Ke, according to the manufacturer's specification.
-
I wonder how you "measure" that. FYI: the well capacity of the P30 is 60 Ke, according to the manufacturer's specification.
[a href=\"index.php?act=findpost&pid=163719\"][{POST_SNAPBACK}][/a]
In a completely out-of-focus blank surface with flat illumination, the SNR is supposed to be the square root of the number of photons.
I said that the P30 was about 27K at ISO 100 (not full-well capacity if that's what you thought I wrote). I made a distinction between at the lowest ISO, and at certain ISOs, in my post. Full well might be even different (higher) than the lowest ISO.
If it's much more than 27K at ISO 100, then it may have some kind of other noise limiting highlight SNR.
-
If it's much more than 27K at ISO 100, then it may have some kind of other noise limiting highlight SNR.
[a href=\"index.php?act=findpost&pid=163742\"][{POST_SNAPBACK}][/a]
I poked around a bit more through the RAW, and can find areas that suggest 30.5K photons at RAW level 65535. Nothing higher. The fact that I'm getting 27-30.5 from various levels suggest to me that this indeed mostly shot noise, with no other major influence. If there were some other SNR-limiting factor, the brighter areas would suggest much lower photon counts than the medium areas, and that is not the case. Perhaps full-well is 60K, but for anti-blooming purposes, they don't approach saturation at ISO 100? Or, perhaps the camera has an extended highlight mode which, unlike those in the new Canons, actually works at the lowest ISO? I see on the Phase One site that ISO 100 is the lowest that they list.
-
Here is one that should test the antiblooming theory shot into the sun
http://download.yousendit.com/31ED44D359677D72 (http://download.yousendit.com/31ED44D359677D72)
All Rights Reserved on this one, it made it to a final print
Feel free to print one for your own personal use though.
Marc
[attachment=4462:attachment]
-
What surprises me is the lack of latitude regarding highlight recovery in ACR and the extreme sensitivity of the P30 files to the slightest adjustment of the EC slider.
Whenever the brightest part of an image shows uniform values in all 3 channels, such as a cloud or any whitish surface, I assume that's a reconstruction by ACR and that technically the image is clipped.
The P30 shot with the greatest exposure (1/15th) in default mode looks like a true ETTR. But to get the histogram looking right (I mean correct) requires no more than a 1/3rd stop negative EC adjustment. After such a minimal adjustment, the brightest part of the cloud shows these uniform values in all 3 channels, indicating there's at least some clipping. A more drastic adjustment of the EC slider to -1 EV pushes the entire histogram into the left half, yet the cloud still shows that tell-tale sign of clipping.
Is this significant?
Below are screen captures of the ACR window at 0 EC, -0.33 EC and -1 EC.
[attachment=4464:attachment] [attachment=4465:attachment] [attachment=4466:attachment]
-
It is important to note, that the 5D has only 8 levels to spare for the 9th stop (and seven for anything beyound that). This explains the "good noise behaviour" of the 5D: the noise is low, because there are no levels to distinguish between.
That does not make any sense. "Too few levels" can only increase the intensity of noise; it does not hide it.
Open your mind, keep looking at simulations, and eventually you will see that "the number of levels" is something, which, at this point in time, is only relevant for computer-generated graphics with no built-in dithering noise. 12 bits is all that is needed to convey all the signal and analog noise in current cameras, and that is needed only at the lowest ISO(s).
A lot of the problem with the 5D is line (1-dimensional) noise - 5Ds are one of the more banding-prone Canons. Banding, however, is addressable to a good degree. The black borders on the left and top of 5D RAWs can remove it roughly, and a good algorithm could probably correct it even further. It's too bad Canon isn't smart enough to put thicker black borders on all 4 sides of a sensor - that would make banding a non-issue, even if the banding intensity ramps throughout the length of lines.
Finally, a small test for the value of the bit depth over 12. One of the layers show the P30 image with the four low ordeer bits shifted out, +5 EV. The same area selected as in the normal processing, the noise (in terms of standard deviation) is higher without the low order bits.
[a href=\"index.php?act=findpost&pid=163707\"][{POST_SNAPBACK}][/a]
You can't zero out the LSBs and expect to get good results (and you *did* do it at the grey CFA stage of the RAW data, I hope). You are clipping shadow detail by doing that, and darkening the overall intensity. You must add 1/2 ADU of the resulting bit depth in the original one first. In this case, you must add 8 before you integer-divide by 16.
The lack of black in your non-quantized +5EV P30 makes me think that there is some quirk in the processing. The image looks too pale green.
-
The P30 shot with the greatest exposure (1/15th) in default mode looks like a true ETTR
This shot is clearly overexposed (the magenta indicates the lack of green, because I substituted the clipped pixels by null for this display).
And again: you have to add 1 EV in ACR to see the truth. I don't know, what caused this effect, but ACR treats every shot with the P30, what Marc posted, as in need for a -1 EV adjustment (thus you have to compensate with +1 EV in order to land at 0).
-
That does not make any sense. "Too few levels" can only increase the intensity of noise; it does not hide it.
When you have only three levels in a stop, you see "smooth" areas.
12 bits is all that is needed to convey all the signal and analog noise in current cameras, and that is needed only at the lowest ISO(s)
I just showed above, that the lack of reduction of the 16bit values to 12bit caused noise.
Note, that this does not contradict my other statement, that the lack of levels causes the 5D images appear less noisy; in the area I used in the demonstration there were still 8-12 levels per channel after the reduction.
You can't zero out the LSBs and expect to get good results. (and you *did* do it at the grey CFA stage of the RAW data, I hope). You are clipping shadow detail by doing that, and darkening the overall intensity. You must add 1/2 ADU of the resulting bit depth in the original one first. In this case, you must add 8 before you integer-divide by 16
The effect of *maximum ONE* in the pixel value range of 1-4095 is negligable. Btw, who told you, that the value would be rounded if there were only 12 bits, instead of truncated?
The lack of black in your non-quantized +5EV P30 makes me think that there is some quirk in the processing. The image looks too pale green.
After +5 EV only the "true blacks" appear black, and those are in other parts of the image (only some edges of the wooden doors on the kitchen island show massive blacks).
Re the color: 1. I did not make any WB, 2. there is no proper de-mosaicing, only an averaging of the colors. Sometimes I turn off even that, and then the image becomes very green, because half of all pixels are green (the true sharpness of the image can be seen that way).
-
This shot is clearly overexposed (the magenta indicates the lack of green, because I substituted the clipped pixels by null for this display).
And again: you have to add 1 EV in ACR to see the truth. I don't know, what caused this effect, but ACR treats every shot with the P30, what Marc posted, as in need for a -1 EV adjustment (thus you have to compensate with +1 EV in order to land at 0).
[a href=\"index.php?act=findpost&pid=163977\"][{POST_SNAPBACK}][/a]
However, if the only clipping of channels is in greyish clouds, then does it matter? Variation of grey scale values exist as one moves the eyedropper around the brightest areas. It's not as though the clouds appear as uniform white sheets of paper.
In a situation like this where one is striving to get detail in the shadows, insignificant technical clipping of grey clouds should not be an issue. The 1/15 sec exposure is the one I'd use for processing of this scene.
-
In a situation like this where one is striving to get detail in the shadows, insignificant technical clipping of grey clouds should not be an issue. The 1/15 sec exposure is the one I'd use for processing of this scene.
It's a trade-off; if not everything fits, you got to decide, what to sacrifice. I often shoot with exposure bracketing and decide later, what is better.
Clipping is often limited to small or irrelevant areas, while underexposure can reduce the quality of large parts of the image.
The problem is, that the raw processors don't show the truth (that's what Rawnalyze is for). The clipping indication reflects the result after WB application and after exposure, brightness, contrast, saturation, curves adjustments. You can be led to believe, that there was clipping, while in fact no data is lost and by reducing the exposure (and adjusting the curves) you can get "back" everything (that's what the "recovery" function is doing). On the other hand, factual clipping can become "eliminated" by changing the adjustments.
-
Gabor, I'd love to argue these things with you, but it is very difficult parsing your English.
When you have only three levels in a stop, you see "smooth" areas.
Not from a digital camera you don't. None of them have low enough noise for that to ever happen to the RAW data with 12 bits.
I just showed above, that the lack of reduction of the 16bit values to 12bit caused noise.
That sentence does not make any sense to me. I don't mean that I disagree; I mean that I have no idea what you're trying to say there. What is "lack of reduction"?
Note, that this does not contradict my other statement, that the lack of levels causes the 5D images appear less noisy; in the area I used in the demonstration there were still 8-12 levels per channel after the reduction.
The effect of *maximum ONE* in the pixel value range of 1-4095 is negligable. Btw, who told you, that the value would be rounded if there were only 12 bits, instead of truncated?
Quantization always lowers the mean, if the bits being dropped have an equally distributed usage.
What is "maximum ONE"?
After +5 EV only the "true blacks" appear black, and those are in other parts of the image (only some edges of the wooden doors on the kitchen island show massive blacks).
Re the color: 1. I did not make any WB, 2. there is no proper de-mosaicing, only an averaging of the colors.[a href=\"index.php?act=findpost&pid=163981\"][{POST_SNAPBACK}][/a]
The main issue is *when* the quantization occurs; not the quality of the spatial color interpolation.
When do you do it? I had the same problem with Roger Clark; he quantized a linear TIFF, already interpolated, and not properly offset, to argue against my claim.
If two people (or three people) quantize, and one loses nothing visible, that person can not possibly be wrong. If there was anything to lose in the process, they would have lost it. The people who lose something going to the same number of levels are obviously doing something wrong.