An example of the shadow limitations of a MF CCD

[Paul2660]

There has been so much talk recently on this forum and others about CMOS and CCD and the difference in color and dynamic range between the two technologies.  I wanted to post this crop and I hope it shows up large enough for you folks to see.   This crop was taken right from C1 and shows the massive difference that a shutter speed of 1/4 of second and 1 second make in shadows.  Both of these crops are from the far right of of 15mm shift from a 40mm Rodie at F11 on a IQ260.  The areas in the shot that received normal light look about the same, you can see this in the dogwood leaves (pale red) and some of the green leaves above the dogwood.  However look at the shadows right above the dogwood.  Here you can see some grasses and foliage typical of a Ozark mountain, moss, twigs etc.  It's just not really there on the 1/4 shot, yes, you can make it out, but it's vague and not well formed.  Looking at the 1 second shot, you can make out individual grass blades and look well back into the shadows.  This is using Capture One, 8 which is by far the best version I have used.

I see this over and over in my work.  I love the color that such chips are capable of creating, but when you lose all the available details in the shadows it really makes for a tasking shot.  You won't get them back, as the CCD did not have enough light to figure out what these structures were so it's just muddy.  The 1 second shot has plenty of details (all I need to do is add a bit of focus magic if I really want to pull them out).  You can also see the same issue in some of the smaller green leaves.  On the 1/4 second shot you can see the leaves but they show noise and no details.  

This was taken in mixed light, which is most typical of my working environment.  

No having a CMOS back to compare, I don't know what it would done color wise, but I can assure you that it would have given much more detail than even my 1 second exposure.

Now consider the shooting situation I was in.  This is 3 part shift looking down a row of Sweet Gum's that line a river bluff and each year there is a brief period where they are all yellow and red.  This is a classic mixed light shot as the distant bluff is brightly lit up, but the foreground is in shade.  So you have to bracket anyway.  If there was wind blowing, then I would have needed iso 200 (a real stretch on a IQ260) or iso 400 (not worth the trouble in low light) to stop the motion of the leaves.  As it was this afternoon there was just a light intermittent breeze and I was able to get the shot I wanted.  

This image is a bit soft and I contribute that to fact that it's the far edge of a shift 15mm and at 1 second I may had just a tad of motion blurr, but I don't see it as enough to effect the image.  

The Phase One backs are amazing in what they can do, if the conditions will allow it.  

Paul

[ErikKaffehr]

Hi Paul,

Could you "lift" the shadows in the 1/4s exposure? From the sample it is hard to see if the lack of tonal separation in the darks is due to darkness and possibly a toe characteristics on the tone curve or by the lack of tonal separation in the raw data.

Another point that Anders Torger made on a thread at GetDPI, is that there may be to few photons captured. The way it seems, modern CMOS is very good at keeping down readout noise, but they cannot do anything about low photons counts.

To examplify:

Let's assume that a group of pixels collect 64 photons in average. In this case photon noise would be 8 photons, so we would have a quite decent signal/noise ration of 64/8 = 8.

Now, a typical CCD may have 12 electron charges of readout noise. Each sample photon corresponds one electron (by and large). Noise adds in quadrature, so we would have a total noise of sqrt( 12 * 12 + 8 * 8 ) = 14.4 electron charges. So SNR would be 64/14.4 = 4.4, not so impressive.

With a Nikon D810 readout noise may be more like 2 electrons, so SNR may be like: 64 / sqrt( 2*2 + 8*8 ) = 7.76 that is almost pure shot (photon) noise. So it would give cleaner shadows. But, the photon count may be not enough to offer good separation in colour.

Best regards
Erik

There has been so much talk recently on this forum and others about CMOS and CCD and the difference in color and dynamic range between the two technologies.  I wanted to post this crop and I hope it shows up large enough for you folks to see.   This crop was taken right from C1 and shows the massive difference that a shutter speed of 1/4 of second and 1 second make in shadows.  Both of these crops are from the far right of of 15mm shift from a 40mm Rodie at F11 on a IQ260.  The areas in the shot that received normal light look about the same, you can see this in the dogwood leaves (pale red) and some of the green leaves above the dogwood.  However look at the shadows right above the dogwood.  Here you can see some grasses and foliage typical of a Ozark mountain, moss, twigs etc.  It's just not really there on the 1/4 shot, yes, you can make it out, but it's vague and not well formed.  Looking at the 1 second shot, you can make out individual grass blades and look well back into the shadows.  This is using Capture One, 8 which is by far the best version I have used.

I see this over and over in my work.  I love the color that such chips are capable of creating, but when you lose all the available details in the shadows it really makes for a tasking shot.  You won't get them back, as the CCD did not have enough light to figure out what these structures were so it's just muddy.  The 1 second shot has plenty of details (all I need to do is add a bit of focus magic if I really want to pull them out).  You can also see the same issue in some of the smaller green leaves.  On the 1/4 second shot you can see the leaves but they show noise and no details.  

This was taken in mixed light, which is most typical of my working environment.  

No having a CMOS back to compare, I don't know what it would done color wise, but I can assure you that it would have given much more detail than even my 1 second exposure.

Now consider the shooting situation I was in.  This is 3 part shift looking down a row of Sweet Gum's that line a river bluff and each year there is a brief period where they are all yellow and red.  This is a classic mixed light shot as the distant bluff is brightly lit up, but the foreground is in shade.  So you have to bracket anyway.  If there was wind blowing, then I would have needed iso 200 (a real stretch on a IQ260) or iso 400 (not worth the trouble in low light) to stop the motion of the leaves.  As it was this afternoon there was just a light intermittent breeze and I was able to get the shot I wanted.  

This image is a bit soft and I contribute that to fact that it's the far edge of a shift 15mm and at 1 second I may had just a tad of motion blurr, but I don't see it as enough to effect the image.  

The Phase One backs are amazing in what they can do, if the conditions will allow it.  

Paul

[Paul2660]

Hi Paul,

Could you "lift" the shadows in the 1/4s exposure? From the sample it is hard to see if the lack of tonal separation in the darks is due to darkness and possibly a toe characteristics on the tone curve or by the lack of tonal separation in the raw data.

Another point that Anders Torger made on a thread at GetDPI, is that there may be to few photons captured. The way it seems, modern CMOS is very good at keeping down readout noise, but they cannot do anything about low photons counts.

To examplify:

Let's assume that a group of pixels collect 64 photons in average. In this case photon noise would be 8 photons, so we would have a quite decent signal/noise ration of 64/8 = 8.

Now, a typical CCD may have 12 electron charges of readout noise. Each sample photon corresponds one electron (by and large). Noise adds in quadrature, so we would have a total noise of sqrt( 12 * 12 + 8 * = 14.4 electron charges. So SNR would be 64/14.4 = 4.4, not so impressive.

With a Nikon D810 readout noise may be more like 2 electrons, so SNR may be like: sqrt( 2*2 + 8* = 7.76 that is almost pure shot (photon) noise. So it would give cleaner shadows. But, the photon count may be not enough to offer good separation in colour.

Best regards
Erik

Hi Eric,   

They were lifted, I forgot to mention this.  I pulled shadows from both the main tool set, and in an adjustment layer.  I tried to get the info bar to show but the image was too large.
This is why I bracket pretty much every shot, when I have extreme shadows, as I know I will get a shutter speed that will get there.  However with a tech camera this is tedious and can bring blur into play just due to moving too fast between setting/cocking and firing the shutter, which I have been known to do   I forgot to mention that these are both iso 50 image, i.e the supposed base of the IQ260, but that may be a false as a lot of evidence now shows it's really iso100 not 50. 

Paul

[voidshatter]

Another point that Anders Torger made on a thread at GetDPI, is that there may be to few photons captured. The way it seems, modern CMOS is very good at keeping down readout noise, but they cannot do anything about low photons counts.

With a Nikon D810 readout noise may be more like 2 electrons, so SNR may be like: sqrt( 2*2 + 8* = 7.76 that is almost pure shot (photon) noise. So it would give cleaner shadows. But, the photon count may be not enough to offer good separation in colour.

Best regards
Erik

Again I need to see hard evidence for this, not just speculations. I post again here:

It would be good to know whether your claimed lack of color / dynamism / realism is true from pushed shadows. I don't see such difference imo. Below shows the identical color / look / saturation etc from two bracketed exposures 2ev apart. (link to download raw)

[BJL]

With a Nikon D810 readout noise may be more like 2 electrons, so SNR may be like: sqrt( 2*2 + 8*8) = 7.76 ...

Eric, you need to check "Don't use smileys" when you post, to avoid misinterpretation of "8)" as "Cool"!

(Also, sqrt( 2*2 + 8*8) = 8.25.)

[ErikKaffehr]

Thanks,

I mixed up noise and NSR, now fixed!

Best regards
Erik

Eric, you need to check "Don't use smileys" when you post, to avoid misinterpretation of "" as "Cool"!

(Also, sqrt( 2*2 + 8* = 8.25.)

[ErikKaffehr]

Hi,

I have looked at your images a bit. Now, these are from the IQ-250, which doesn't have the CCD-sensors MFD used to have.

One impression I may have that Capture One is overdoing noise reduction.

Below are same samples from:

- C1 using no NR and LR using no NR
- C1 using default NR and LR using no NR
- C1 using no NR and AccuRaw using default NR

Best regards
Erik

Again I need to see hard evidence for this, not just speculations. I post again here:

It would be good to know whether your claimed lack of color / dynamism / realism is true from pushed shadows. I don't see such difference imo. Below shows the identical color / look / saturation etc from two bracketed exposures 2ev apart. (link to download raw)

[torger]

I think most raw converters overdo noise reduction. I think it's because they want to make a clean pixel peep impression rather than thinking about how global color look in a print. Some grain is no problem for a print, but dead color is.

Concerning my observation that there might be not enough photons to give good colors in pushed shadows it's indeed a speculation, trying to explain what I see in many shots. I haven't yet looked at the above shots, but I guess 2 pushed stops may not be enough to show a significant effect, I'm thinking more like 4 stops.

What I often see in backlit landscape scenes with pushed shadows is that there's often a brown(ish)-centered palette in the pushed areas, color does not look as good in the well-exposed areas. I have not investigated in detail why this is, but have speculated that it's too few photons captured to make good color.

[ErikKaffehr]

Sila mygg och svälja elefanter...

/Erik

I think most raw converters overdo noise reduction. I think it's because they want to make a clean pixel peep impression rather than thinking about how global color look in a print. Some grain is no problem for a print, but dead color is.

Concerning my observation that there might be not enough photons to give good colors in pushed shadows it's indeed a speculation, trying to explain what I see in many shots. I haven't yet looked at the above shots, but I guess 2 pushed stops may not be enough to show a significant effect, I'm thinking more like 4 stops.

What I often see in backlit landscape scenes with pushed shadows is that there's often a brown(ish)-centered palette in the pushed areas, color does not look as good in the well-exposed areas. I have not investigated in detail why this is, but have speculated that it's too few photons captured to make good color.

[torger]

Here's a demo of what I see. Both files are processed in RawTherapee and have exactly the same processing parameters, except that the darker (the first) has been pushed 2 more stops. In theory they should look identical.

Watch both images side by side or layer them on top of eachother (it can be hard to see differences when above and below like they become here, unfortunately) and see that the colors are not the same in the shadows, the benches are a good thing to look at, but you can see differences in brighter colors too. The underexposed file has taken on a greenish cast (I've noted that it varies if sensors take on a greenish or brownish cast when underexposed, this sensor seems to be of the green type, while D800 is brown if I remember correctly). I'm sure color separation is noticeably worse too, but as it's mostly only brown wood there we don't see it in this picture.

Note: in this case I've used Capture One's IQ250 tungsten icc profile, which might have hue twists which could affect the result. It would have been better if I've just used plain matrix color (guaranteed linear without hue twists), however I just did a comparison with matrix color and the difference in look is the same, so if there are any hue twists in the IQ250 icc it's negligible for this comparison.

This reduced color fidelity I think is a problem, and therefore I don't think it's a good idea to underexpose and push hard, even if the sensor has low read noise, and therefore I think that the advantage of Sony Exmor clean shadows is exaggerated when it comes to high fidelity photography, it doesn't free you from making good exposures if you want good color.

I haven't really investigated this phenomenon in detail, it would be interesting to do. I suspect that if you are concerned about color fidelity the extra 1-2 stop you gain in noise with the Sony sensor is not very usable as color fidelity is poor anyway. (If you're into long exposure then there's a lot to gain though, the examples of IQ260 I've seen has been quite disappointing, it depends so heavily on noise reduction that color fidelity and detail must suffer a lot, my guess is that the P45+ actually produces better results for long exposure, both in terms of detail and color, and of course the IQ250 is best.)

[voidshatter]

Here's a demo of what I see. Both files are processed in RawTherapee and have exactly the same processing parameters, except that the darker (the first) has been pushed 2 more stops. In theory they should look identical.

Watch both images side by side or layer them on top of eachother (it can be hard to see differences when above and below like they become here, unfortunately) and see that the colors are not the same in the shadows, the benches are a good thing to look at, but you can see differences in brighter colors too. The underexposed file has taken on a greenish cast (I've noted that it varies if sensors take on a greenish or brownish cast when underexposed, this sensor seems to be of the green type, while D800 is brown if I remember correctly). I'm sure color separation is noticeably worse too, but as it's mostly only brown wood there we don't see it in this picture.

Note: in this case I've used Capture One's IQ250 tungsten icc profile, which might have hue twists which could affect the result. It would have been better if I've just used plain matrix color (guaranteed linear without hue twists), however I just did a comparison with matrix color and the difference in look is the same, so if there are any hue twists in the IQ250 icc it's negligible for this comparison.

This reduced color fidelity I think is a problem, and therefore I don't think it's a good idea to underexpose and push hard, even if the sensor has low read noise, and therefore I think that the advantage of Sony Exmor clean shadows is exaggerated when it comes to high fidelity photography, it doesn't free you from making good exposures if you want good color.

I haven't really investigated this phenomenon in detail, it would be interesting to do. I suspect that if you are concerned about color fidelity the extra 1-2 stop you gain in noise with the Sony sensor is not very usable as color fidelity is poor anyway. (If you're into long exposure then there's a lot to gain though, the examples of IQ260 I've seen has been quite disappointing, it depends so heavily on noise reduction that color fidelity and detail must suffer a lot, my guess is that the P45+ actually produces better results for long exposure, both in terms of detail and color, and of course the IQ250 is best.)

What you see here is just color cast of shadow noise. The shadow noise of a Sony Exmor can sometimes have a green cast and sometimes have a magenta cast. It has nothing to do with color fidelity. A while ago I did an article about how to achieve great dynamic range with temporal noise reduction. The method was to use a Nikon D800 to continuously shoot 60 severely underexposed images, then push them in ACR and stack them in Photoshop to reduce shadow noise. It's the same way as how Red Epic Dragon scores top 1 dynamic range at dxomark.com.



















If you pay attention to the shadow noise you would see that even throughout the same session of continuous shots (i.e. same temperature, humidity, light etc) you can't expect the color of shadow noise to be neutral for each frame. However the good part is that if you shoot many enough, then the color cast of noise of each frame eventually cancels each other and you get a neutral one (consider that the number of green cast frames are equal to the number of magenta cast frames).

Actually looking back to other shadow push examples, you would see that the IQ250 had magenta shadow noise while the D800E had green shadow noise. If you shoot more frames under the same condition, you could get the opposite results.



I would say that if you stay away from pushing the shadow too far producing shadow noise, then the extra 3 (yes, three) stops you gain in noise with the Sony sensor is very usable.

[Ken R]

There has been so much talk recently on this forum and others about CMOS and CCD and the difference in color and dynamic range between the two technologies.  I wanted to post this crop and I hope it shows up large enough for you folks to see.   This crop was taken right from C1 and shows the massive difference that a shutter speed of 1/4 of second and 1 second make in shadows.  Both of these crops are from the far right of of 15mm shift from a 40mm Rodie at F11 on a IQ260.  The areas in the shot that received normal light look about the same, you can see this in the dogwood leaves (pale red) and some of the green leaves above the dogwood.  However look at the shadows right above the dogwood.  Here you can see some grasses and foliage typical of a Ozark mountain, moss, twigs etc.  It's just not really there on the 1/4 shot, yes, you can make it out, but it's vague and not well formed.  Looking at the 1 second shot, you can make out individual grass blades and look well back into the shadows.  This is using Capture One, 8 which is by far the best version I have used.

I see this over and over in my work.  I love the color that such chips are capable of creating, but when you lose all the available details in the shadows it really makes for a tasking shot.  You won't get them back, as the CCD did not have enough light to figure out what these structures were so it's just muddy.  The 1 second shot has plenty of details (all I need to do is add a bit of focus magic if I really want to pull them out).  You can also see the same issue in some of the smaller green leaves.  On the 1/4 second shot you can see the leaves but they show noise and no details.  

This was taken in mixed light, which is most typical of my working environment.  

No having a CMOS back to compare, I don't know what it would done color wise, but I can assure you that it would have given much more detail than even my 1 second exposure.

Now consider the shooting situation I was in.  This is 3 part shift looking down a row of Sweet Gum's that line a river bluff and each year there is a brief period where they are all yellow and red.  This is a classic mixed light shot as the distant bluff is brightly lit up, but the foreground is in shade.  So you have to bracket anyway.  If there was wind blowing, then I would have needed iso 200 (a real stretch on a IQ260) or iso 400 (not worth the trouble in low light) to stop the motion of the leaves.  As it was this afternoon there was just a light intermittent breeze and I was able to get the shot I wanted.  

This image is a bit soft and I contribute that to fact that it's the far edge of a shift 15mm and at 1 second I may had just a tad of motion blurr, but I don't see it as enough to effect the image.  

The Phase One backs are amazing in what they can do, if the conditions will allow it.  

Paul

Try using the Roddie 40 at f8 instead of f11. I find that sharpness is reduced significantly at f11 vs f5.6~f8. Dof is not that much different in typical landscape situations.

Also, if the shadow areas are in the outer edges of the image circle then it's a double whammy since you are not only bringing up the shadows in post but correcting for the vignetting (and color cast) of the lens as well (which is not bad on the 40mm roddie but still there is some).

But, this extreme shadow recovery, specially in longer exposures (bad light) and of course higher iso / long exposure performance are the only weaknesses of the latest CCD medium format backs. They are superb in most situations.

So, if your typical shooting involves needing high iso, long exposures and/or EXTREME shadow recovery then the Sony CMOSs are the right choice for you. 

Right tool for the job.

[torger]

What you see here is just color cast of shadow noise.

Let us assume it's indeed a shot noise issue, then stacking will solve it as it's a way of gathering more photons.

Varying color of the cast from time to time sounds interesting indeed, I would guess it may be due to narrow band light sources, but if it indeeds varies under the same conditions over time I do wonder what the reason is, and then it's a larger color fidelity issue as you can't simply apply a static correction, you would need to hand-tune it from time to time, or just ignore it as most would do.

That shadow color cast has nothing to do with color fidelity I disagree with, there's a cast to start with. I guess you mean that if you just correct the cast there's no issue with color separation (and hue/saturation), and well that might be true but I seriously doubt that and would like to test sometime, shooting a color checker with various degrees of underexposure would be a good lab condition test.

[torger]

Note: in this case I've used Capture One's IQ250 tungsten icc profile, which might have hue twists which could affect the result. It would have been better if I've just used plain matrix color (guaranteed linear without hue twists), however I just did a comparison with matrix color and the difference in look is the same, so if there are any hue twists in the IQ250 icc it's negligible for this comparison.

Looked a bit closer, and yes there are significant hue twists in the profile, the different color on the quite bright organ in the center is due to the profile. The shadow color cast is still there with linear color conversion though.

I haven't really checked recently how C1 applies their profiles, do they do it before or after exposure correction? If they do it before (like RT does) the advantage is that you can make a profile that corrects for nonlinearities in color response (this is what you would want to do in reproduction), but the disadvantage is that a profile with hue twists will yield different color depending on exposure correction. Adobe's DCP has two sets of color correction tables, one that is applied before exposure correction (the purpose is to correct color) and the other is after exposure correction/tonemapping etc (the purpose is to apply a subjective look). That's an advantage of DCP over ICC... with ICC you need to choose if to apply it before or after...

posting a linear conversion soon...

[voidshatter]

That shadow color cast has nothing to do with color fidelity I disagree with, there's a cast to start with. I guess you mean that if you just correct the cast there's no issue with color separation (and hue/saturation), and well that might be true but I seriously doubt that and would like to test sometime, shooting a color checker with various degrees of underexposure would be a good lab condition test.

No. I mean if you don't push too hard to have visible shadow noise then you don't have color cast. Compared against the CCD sensors and the current Canon sensors (excluding the upcoming 5DSR) You have 3 (three) stops of more room to push on the Sony CMOS sensor.

[torger]

Here's a linear comparison, ie linear matrix color no icc or dcp profile applied. Still have the issue, but it does look less bad thanks to that there are no hue twists. The pushed image is the one to the left.

It would be worthwhile to look into how C1 applies its ICC profiles, if it's before exposure correction there's an issue and one should probably roll one's own profile without hue twists. I don't have C1 on this computer so I can't test myself right now.

[torger]

No. I mean if you don't push too hard to have visible shadow noise then you don't have color cast. Compared against the CCD sensors and the current Canon sensors (excluding the upcoming 5DSR) You have 3 (three) stops of more room to push on the Sony CMOS sensor.

Ok. Can't argue against that as I haven't done any tests. Sounds quite reasonable though, less noise in general means better color fidelity.

[Bart_van_der_Wolf]

Sounds quite reasonable though, less noise in general means better color fidelity.

Yes, more signal, a higher S/N ratio, will allow a more stable demosaicing of color. Topaz Denoise even has a specific control for automatic Black point adjustment when lots of noise reduction is required in shadows.

Cheers,
Bart

[Paul2660]

However, if you take a look at a sensor plus example, the results are very impressive indeed.  This is a 1/125 shutter speed, and I needed it due to wind.  I had a CL-PL on only, no CF.  At iso 50, the recovery of shadows would have next to impossible without a bracketing, and with wind combining brackets becomes a task.

On this side by side, you have a IQ260 image, at ISO 200 in LE (long exposure mode).  On the right is image with only the LCC applied and on the left I have worked up the shot.  Notice the area under the main trees.  In the right hand shot, it's basically almost black, and shows very little details.  Remember this is iso 200.  If you took a full resolution shot at 200, in normal or LE mode, these areas would be mushy and have little to no recoverable details.  Where as the sensor plus image holds up to an almost 1.5 stop push.  I over did it just a bit to see if I would see noise, but you don't.  So, the sensor plus story is a good one and pretty impressive technology.  However for my work, it only makes sense in the IQ180 as here you are using a 20MP image, which really has a lot more room for printing, especially when considering that I could stitch it. 

The IQ260 only gives 15MP, which unless I do a nodal pan or a lot of stitching is going to stretch the resolution for a large print. 

Looking back, I should have moved into the 180 instead of the 260 as the sensor plus images I have seen from the 180 are just as impressive. 

With sensor plus, you don't see in loss in saturation at 400 up to 800, where as in full resolution mode, by iso 200 you are really going to see a large increase in noise in the shadows and considerable loss in both and saturation.  If you push to iso 400 even in good light, you will see a loss in details and saturation enough that it may make more sense to use Sensor plus and have the better details and color/saturation, then try to uprez the image with one of the many software tools out there. 

Paul

[voidshatter]

Paul, do you find real advantage of the sensor+ tech, when compared against downsampled pictures at the same high ISO without using sensor+?