Pages: [1] 2   Go Down

Author Topic: P30 and 5D Dynamic Range  (Read 7664 times)

marcmccalmont

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1780
P30 and 5D Dynamic Range
« on: December 27, 2007, 03:55:50 pm »

The following are links to both 5D and P30 RAW files
5D 60mm f9 iso 100
p30 80mm f12 iso 100

5D;
http://download.yousendit.com/648514375875EFC8
http://download.yousendit.com/4CE5D23F2F045A91
http://download.yousendit.com/0494FB603F426A30

P30;
http://download.yousendit.com/DB179C8A62750E5E
http://download.yousendit.com/3A99538D7688B545
http://download.yousendit.com/788812D41467C4AD


I think (using Breezebrowsers histogram) 2035 and 19371 are the best exposures for comparison
Marc
P30 [attachment=4404:attachment]
5D [attachment=4405:attachment]
Logged
Marc McCalmont

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
P30 and 5D Dynamic Range
« Reply #1 on: December 27, 2007, 08:07:50 pm »

Quote
The following are links to both 5D and P30 RAW files
[a href=\"index.php?act=findpost&pid=163433\"][{POST_SNAPBACK}][/a]

Well, I've been looking at the first P30 RAW file.  This is what I see:

1) Low-ISO pixel-level read noise (relative to maximum RAW value) is almost exactly the same as the Canon 1Dsmk2, 1Dsmk3, 1Dmk2, 1Dmk3 and Nikon D3 (all are about 0.6 stops lower than the 5D).

2) As I would expect with this level of read noise, the 4 least significant bits are pretty much noise only, and a waste of time and space.  I can't distinguish a version interpolated to RGB with an extreme push in the shadows in 16-bits and quantized to 12 bits (before interpolation).

I have yet to try to measure shot noise, but I need to restart my computer now because its acting strange (Firefox froze up and then disappeared), and want to get this thread moving a little with these comments.
Logged

marcmccalmont

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1780
P30 and 5D Dynamic Range
« Reply #2 on: December 27, 2007, 08:49:07 pm »

Quote
Well, I've been looking at the first P30 RAW file.  This is what I see:

1) Low-ISO pixel-level read noise (relative to maximum RAW value) is almost exactly the same as the Canon 1Dsmk2, 1Dsmk3, 1Dmk2, 1Dmk3 and Nikon D3 (all are about 0.6 stops lower than the 5D).

2) As I would expect with this level of read noise, the 4 least significant bits are pretty much noise only, and a waste of time and space.  I can't distinguish a version interpolated to RGB with an extreme push in the shadows in 16-bits and quantized to 12 bits (before interpolation).

I have yet to try to measure shot noise, but I need to restart my computer now because its acting strange (Firefox froze up and then disappeared), and want to get this thread moving a little with these comments.
[a href=\"index.php?act=findpost&pid=163479\"][{POST_SNAPBACK}][/a]


Since the least significant four bits are mostly noise (as one would expect)  how can you (easily) turn them into zeros in your file (nothing should be better than noise?)
Marc
Logged
Marc McCalmont

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
P30 and 5D Dynamic Range
« Reply #3 on: December 27, 2007, 09:02:01 pm »

Quote
I have yet to try to measure shot noise, but I need to restart my computer now because its acting strange (Firefox froze up and then disappeared), and want to get this thread moving a little with these comments.
[a href=\"index.php?act=findpost&pid=163479\"][{POST_SNAPBACK}][/a]

Well, I rebooted, and Firefox remembered where it was.  I could have probably just started firefox again, but I haven't rebooted this PC in months, so I figured I'd get a fresh start.

Well, I just searched the P30 RAW for areas of OOF flat areas, and they consistently point to about 27K photons at RAW saturation at ISO 100.  Let's look at some of the photon collections of current cameras at their lowest "full DR" ISOs:

1Dsmk3 - 60K at ISO 100 * 22.1MP = 1326 GPh (gigaphotons)
P30 - 27K at ISO 100 * 30MP = 810 GPh
D3 - 53.5K at ISO 200 * 12.1MP = 647 GPh
5D - 52.3K at ISO 100 * 12.8MP = 669 GPh

I don't know what the maximums are for the D3 and 1Dsmk3 at their "low"
ISOs (100 and 50, respectively, AFAIK), or anything the P30 has below ISO 100, but they may collect more photons in them.  The 5D's maximum is 79.9K * 12.8MP = 1023 GPh.

Like most MFDBs whose RAW data I've looked at, the P30 collects a lot less photons than are possible with the pixel density.  MFDBs in general seem to collect less photons per square mm than DSLRs and compact P&S cameras, at a given exposure rate.
« Last Edit: December 27, 2007, 09:19:35 pm by John Sheehy »
Logged

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
P30 and 5D Dynamic Range
« Reply #4 on: December 27, 2007, 09:05:48 pm »

Quote
Since the least significant four bits are mostly noise (as one would expect)  how can you (easily) turn them into zeros in your file (nothing should be better than noise?)
Marc
[a href=\"index.php?act=findpost&pid=163483\"][{POST_SNAPBACK}][/a]

No; turning them to zeros technically makes them noisier.  The point is, however, that the amount that zeroing them makes them noisier is relatively insignificant, compared to the analog noise present in the image, so there is little point in storing them, unless you're going to do something like add a whole bunch of images together, in which case the tiny amount of signal present in each added image will sum into slightly more usable signal.
Logged

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
P30 and 5D Dynamic Range
« Reply #5 on: December 27, 2007, 10:00:50 pm »

I wrote:

Quote
No; turning them to zeros technically makes them noisier.  The point is, however, that the amount that zeroing them makes them noisier is relatively insignificant, compared to the analog noise present in the image, so there is little point in storing them, unless you're going to do something like add a whole bunch of images together, in which case the tiny amount of signal present in each added image will sum into slightly more usable signal.
[a href=\"index.php?act=findpost&pid=163487\"][{POST_SNAPBACK}][/a]

Here's RAW levels 0 through 360 out of about 0 - 65535; an effective push from ISO 100 to about ISO 4500, including some white balancing:



One half of this image is 16-bit; the other was quantized to 12-bit first before color interpolation and WB (the one on the right was flipped horizontally, to make comparison easier).  Can you tell which half is which?

You can see that the noise in the image is what limits IQ, not 12 bits.  When you get to higher ISOs, you don't even need 12 bits.
Logged

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
P30 and 5D Dynamic Range
« Reply #6 on: December 28, 2007, 12:24:27 am »

Can you tell which half is which?
Quote

I can't, but then you really can't tell accurate bit level on an 8bit video system which we all are stuck with.

It's still better than what you'ld get on film under similar exposure parameters IMO.
« Last Edit: December 28, 2007, 12:26:50 am by tlooknbill »
Logged

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
P30 and 5D Dynamic Range
« Reply #7 on: December 28, 2007, 08:31:32 am »

Quote
Can you tell which half is which?/quote]

I can't, but then you really can't tell accurate bit level on an 8bit video system which we all are stuck with.

It's still better than what you'ld get on film under similar exposure parameters IMO.
[a href=\"index.php?act=findpost&pid=163521\"][{POST_SNAPBACK}][/a]

Why would you expect 8-bit displays to get in the way?  This is only about 8.33 bits (361 original RAW levels) in the 16-bit image, and 4.33 bits (22.6 quantized levels) in the 12-bit version.  The 8-bit display is equivalent to about 15.5 bits in the original.  Remember, this is a push to about ISO 4500 from ISO 100.
« Last Edit: December 28, 2007, 08:33:09 am by John Sheehy »
Logged

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
P30 and 5D Dynamic Range
« Reply #8 on: December 28, 2007, 01:10:26 pm »

John,

I can't see any difference between the two halves. I don't even know what to look for.

I see evenly distributed chroma noise throughout and definition of detail looks the same as well.
Logged

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
P30 and 5D Dynamic Range
« Reply #9 on: December 28, 2007, 05:15:53 pm »

Quote
John,

I can't see any difference between the two halves. I don't even know what to look for.

I see evenly distributed chroma noise throughout and definition of detail looks the same as well.
[a href=\"index.php?act=findpost&pid=163622\"][{POST_SNAPBACK}][/a]

Well, that's my point.  You can clearly see the noise, and the 16 bits of one half (15.5 visible; I should have pushed a little more and used 0 - 255 of the original 16-bit levels to keep things simpler).  Therefore, any subtle benefit of 16 bit is lost in the capture, due to the noise.

Companies aren't friends.  I'm sure that they usually wait until they are improving the IQ at the analog level, if possible, and then introduce a higher bit depth, so that the public attributes the higher IQ to the higher bit depth.

The reality is that the analog aspects of reading out sensors are not ready yet for more than 12 bits.  More bits may be useful to the astrophotographer who is stacking multiple images (but they are probably using Canons, which are the only cameras whose RAWs, AFAIK, are unclipped and linear near black), but for the rest of us taking one shot at a time, it's just a waste of storage space and time.  The most ridiculous RAW I've seen is the Panasonic; my FZ50 compact ZLR has 12 bits of RAW data, only worthy of 11 bits at ISO 100, but stores the RAWs as 16-bit RAWs with 4 trailing zeros for each pixel.  They weigh in at 20MB each (and take over 5 seconds to write), but I have compressed them down to 3MB with advanced ZIP options.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
P30 and 5D Dynamic Range
« Reply #10 on: December 28, 2007, 09:12:37 pm »

Quote
I think (using Breezebrowsers histogram) 2035 and 19371 are the best exposures for comparison

CF019372 and _MG_2033 are the best shots. They are as far to the right as possible without clipping, and they are equal in exposure as far as possible.

CF019372 is a tiny bit less bright than _MG_2033, but increased already by 1/3 stop it becomes brighter than _MG_2033. Note, that one needs +1 EV in ACR because of a "pre-mediated" adjustment of -1 EV, which has been carried out by ACR without asking you and without telling you about it.

Although the highlights can't be analyzed (clouds are not very suitable to comparison between two shots), that does not pose any problem, as the stress is not on the details and noise in the highest range.

Unfortunately, the salt shaker is not in the shots, but the funny animals on the yellow pot with the snow flakes can be used as "fine detail".

I added 5 EV to both images and selected some originally dark areas. Based on the visual appearance, IMO the 5D's DR over 8 stops is a myth, while the P30 goes somewhat over 10 stops.

It is important to note, that the 5D has only 8 levels to spare for the 9th stop (and seven for anything beyound that). This explains the "good noise behaviour" of the 5D: the noise is low, because there are no levels to distinguish between.

A layered TIFF with some related screen shots can be downloaded from here, only 1.8 MB

Finally, a small test for the value of the bit depth over 12. One of the layers show the P30 image with the four low ordeer bits shifted out, +5 EV. The same area selected as in the normal processing, the noise (in terms of standard deviation) is higher without the low order bits.
Logged
Gabor

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
P30 and 5D Dynamic Range
« Reply #11 on: December 28, 2007, 09:15:49 pm »

Quote
1Dsmk3 - 60K at ISO 100 * 22.1MP = 1326 GPh (gigaphotons)
P30 - 27K at ISO 100 * 30MP = 810 GPh
D3 - 53.5K at ISO 200 * 12.1MP = 647 GPh
5D - 52.3K at ISO 100 * 12.8MP = 669 GPh

John, where did you get this figures from?
Logged
Gabor

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
P30 and 5D Dynamic Range
« Reply #12 on: December 28, 2007, 10:25:17 pm »

Quote
John, where did you get this figures from?
[a href=\"index.php?act=findpost&pid=163708\"][{POST_SNAPBACK}][/a]

The first 3 I measured myself, the best I could with the files available, but the results were consistent enough to make me feel they are very close.  The 1Dsmk3 was a little weird, though.  At ISO 100, it seemed to have some kind of added noise, that limited SNRs in the highlights (my XTi has this type of effect, too; it never goes higher than about 100:1).  I had to get the 1Dsmk3 values from ISO 100 shadow areas, and higher ISOs' highlight areas.

The 5D I got from Roger Clark, IIRC.  Actually, that's the 1Dmk2 but they are supposed to be the same in almost every way, electrically at the pixel level.

It would be nice to have real test shots geared towards proper measurements, though.  The many in-focus and textured RAW samples made available are not the most useful for measuring photons collected.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
P30 and 5D Dynamic Range
« Reply #13 on: December 28, 2007, 10:29:19 pm »

Quote
The first 3 I measured myself, the best I could with the files available

I wonder how you "measure" that. FYI: the well capacity of the P30 is 60 Ke, according to the manufacturer's specification.
Logged
Gabor

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
P30 and 5D Dynamic Range
« Reply #14 on: December 29, 2007, 12:40:55 am »

Quote
I wonder how you "measure" that. FYI: the well capacity of the P30 is 60 Ke, according to the manufacturer's specification.
[a href=\"index.php?act=findpost&pid=163719\"][{POST_SNAPBACK}][/a]

In a completely out-of-focus blank surface with flat illumination, the SNR is supposed to be the square root of the number of photons.

I said that the P30 was about 27K at ISO 100 (not full-well capacity if that's what you thought I wrote).  I made a distinction between at the lowest ISO, and at certain ISOs, in my post.  Full well might be even different (higher) than the lowest ISO.

If it's much more than 27K at ISO 100, then it may have some kind of other noise limiting highlight SNR.
Logged

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
P30 and 5D Dynamic Range
« Reply #15 on: December 29, 2007, 10:47:28 am »

Quote
If it's much more than 27K at ISO 100, then it may have some kind of other noise limiting highlight SNR.
[a href=\"index.php?act=findpost&pid=163742\"][{POST_SNAPBACK}][/a]

I poked around a bit more through the RAW, and can find areas that suggest 30.5K photons at RAW level 65535.  Nothing higher.  The fact that I'm getting 27-30.5 from various levels suggest to me that this indeed mostly shot noise, with no other major influence.  If there were some other SNR-limiting factor, the brighter areas would suggest much lower photon counts than the medium areas, and that is not the case.  Perhaps full-well is 60K, but for anti-blooming purposes, they don't approach saturation at ISO 100?  Or, perhaps the camera has an extended highlight mode which, unlike those in the new Canons, actually works at the lowest ISO?  I see on the Phase One site that ISO 100 is the lowest that they list.
Logged

marcmccalmont

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1780
P30 and 5D Dynamic Range
« Reply #16 on: December 29, 2007, 12:20:33 pm »

Here is one that should test the antiblooming theory shot into the sun

http://download.yousendit.com/31ED44D359677D72

All Rights Reserved on this one, it made it to a final print
Feel free to print one for your own personal use though.

Marc

[attachment=4462:attachment]
« Last Edit: December 29, 2007, 12:23:42 pm by marcmccalmont »
Logged
Marc McCalmont

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
P30 and 5D Dynamic Range
« Reply #17 on: December 29, 2007, 12:40:06 pm »

What surprises me is the lack of latitude regarding highlight recovery in ACR and the extreme sensitivity of the P30 files to the slightest adjustment of the EC slider.

Whenever the brightest part of an image shows uniform values in all 3 channels, such as a cloud or any whitish surface, I assume that's a reconstruction by ACR and that technically the image is clipped.

The P30 shot with the greatest exposure (1/15th) in default mode looks like a true ETTR. But to get the histogram looking right (I mean correct) requires no more than a 1/3rd stop negative EC adjustment. After such a minimal adjustment, the brightest part of the cloud shows these uniform values in all 3 channels, indicating there's at least some clipping. A more drastic adjustment of the EC slider to -1 EV pushes the entire histogram into the left half, yet the cloud still shows that tell-tale sign of clipping.

Is this significant?

Below are screen captures of the ACR window at 0 EC, -0.33 EC and -1 EC.

[attachment=4464:attachment]  [attachment=4465:attachment]  [attachment=4466:attachment]
Logged

John Sheehy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 838
P30 and 5D Dynamic Range
« Reply #18 on: December 29, 2007, 04:44:40 pm »

Quote
It is important to note, that the 5D has only 8 levels to spare for the 9th stop (and seven for anything beyound that). This explains the "good noise behaviour" of the 5D: the noise is low, because there are no levels to distinguish between.

That does not make any sense.  "Too few levels" can only increase the intensity of noise; it does not hide it.

Open your mind, keep looking at simulations, and eventually you will see that "the number of levels" is something, which, at this point in time, is only relevant for computer-generated graphics with no built-in dithering noise.  12 bits is all that is needed to convey all the signal and analog noise in current cameras, and that is needed only at the lowest ISO(s).

A lot of the problem with the 5D is line (1-dimensional) noise - 5Ds are one of the more banding-prone Canons.  Banding, however, is addressable to a good degree.  The black borders on the left and top of 5D RAWs can remove it roughly, and a good algorithm could probably correct it even further.  It's too bad Canon isn't smart enough to put thicker black borders on all 4 sides of a sensor - that would make banding a non-issue, even if the banding intensity ramps throughout the length of lines.

Quote
Finally, a small test for the value of the bit depth over 12. One of the layers show the P30 image with the four low ordeer bits shifted out, +5 EV. The same area selected as in the normal processing, the noise (in terms of standard deviation) is higher without the low order bits.
[a href=\"index.php?act=findpost&pid=163707\"][{POST_SNAPBACK}][/a]

You can't zero out the LSBs and expect to get good results (and you *did* do it at the grey CFA stage of the RAW data, I hope).  You are clipping shadow detail by doing that, and darkening the overall intensity.  You must add 1/2 ADU of the resulting bit depth in the original one first.  In this case, you must add 8 before you integer-divide by 16.

The lack of black in your non-quantized +5EV P30 makes me think that there is some quirk in the processing.  The image looks too pale green.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
P30 and 5D Dynamic Range
« Reply #19 on: December 29, 2007, 11:26:30 pm »

Quote
The P30 shot with the greatest exposure (1/15th) in default mode looks like a true ETTR

This shot is clearly overexposed (the magenta indicates the lack of green, because I substituted the clipped pixels by null for this display).

And again: you have to add 1 EV in ACR to see the truth. I don't know, what caused this effect, but ACR treats every shot with the P30, what Marc posted, as in need for a -1 EV adjustment (thus you have to compensate with +1 EV in order to land at 0).
Logged
Gabor
Pages: [1] 2   Go Up