An example of the shadow limitations of a MF CCD

[ErikKaffehr]

Hi,

I just compared a normal exposure with an -4 EV exposure on on my P45+, the -4 EV exposure was corrected by a 4EV push in development. The two TIFFs were compared in Patchtool.

In the attached screen dump comparison images are shown with C1 v8 on the left and LR 5.6 on the right.

I see very little color shift on the P45+ at -4EV exposure.

Best regards
Erik

[voidshatter]

This is nice. What kind of color passport did you shoot? I'd like to test it as well! 

[Paul2660]

Paul, do you find real advantage of the sensor+ tech, when compared against downsampled pictures at the same high ISO without using sensor+?

Yes, it's pretty impressive, and something I really did not pay much attention to until I read some other posts here and elsewhere. 

With the IQ260, you can shoot at iso 50 or 100 and expect pretty clean results unless the back has gotten hot which in the summer in my area does happen.  But if you push the back to 200 you start to see noise in most of the shadows.  You can bracket exposure at 200 and many times this will get you by on non shifted files, but when you start to move 15mm to 20mm, the shifted files really can suffer, enough that you really can't use them in a large print as the noise just really starts to degrade the image.  If you go to iso 400 then things really get ugly, which non useable shadows, but you also see the falloff of finer details and the saturation will seem off a bit.  It was this reason I had hoped that the IQ260, in LE mode for iso 200 and 400 would produce a much more useable file in full resolution but that has turned out NOT to be the case at all, if anything the images appear worse in normal exposure ranges.  LE mode really only seems to help in shots longer than 2 minutes or more. 

In sensor plus, at 200 and 400, the images appear to have much more normal color saturation and you do not lose the finer details.  The shadows will pull up quite well, better in fact than at iso 50 in many cases.  Colors look excellent and the files are extremely clean.  Looking back, if I had moved to the 180, and had 20MP sensor plus, it would have been a much better decision for me.

I am shooting only with tech lenses, which provide for amazing details so the 15MP sensor plus files will uprez about 100% to a 30MP file before they start to show the usual uprez issues.  If a better engine come out, (of which development seems to have stopped as more companies are putting out high res sensors), then these 15MP files could go a long way.  They are very clean to start with, so you don't have a lot of trash pixels being uprez'd, (noise most often).  I feel that iso 800 is the limit and that may be a push as often the files start to look like the 60MP full resolution files taken at iso 400, same issues.

Sensor plus is a good technology for sure.  I had been in the MP race for so long, I never really worked with it until several shoots last year where I just could not operate at iso50. 

Paul

[ErikKaffehr]

Hi,

It is an IT8 test target, the developed TIFF read by "scanin" program from ArgyllCMS and the data sets compared by Patchtool from BabelColor.

http://echophoto.dnsalias.net/ekr/Articles//darkrepro/20150225-CF046081.iiq

http://echophoto.dnsalias.net/ekr/Articles//darkrepro/20150225-CF046083.iiq

More details later today, when I have some time.

Best regards
Erik

This is nice. What kind of color passport did you shoot? I'd like to test it as well! 

[torger]

I see very little color shift on the P45+ at -4EV exposure.

I think you need to lower it considerably more. Look at those church benches of voidshatter's example shot, it's that kind of light level, ie almost black that is raised. One experiment could be to lower step by step until you start see color shifts to see where the limit is.

Also note that color profile may have hue shifts and might affect the result, so it would be better if you used a linear profile (ie only a matrix) when doing these experiments.

[Bart_van_der_Wolf]

I think you need to lower it considerably more. Look at those church benches of voidshatter's example shot, it's that kind of light level, ie almost black that is raised. One experiment could be to lower step by step until you start see color shifts to see where the limit is.

Also note that color profile may have hue shifts and might affect the result, so it would be better if you used a linear profile (ie only a matrix) when doing these experiments.

I agree, the least biased would be e.g. RawDigger data of the same very dark patch of a series of increasingly lower exposures. That would give the baseline data. On top of that one can stack any kind of demosaicing+profiling to see how the noise gets demosaiced into a non-neutral black point, or if something else is happening. My hypothesis is that it's the demosaicing of read-noise that can throw the blackpoint off neutral, and what the profiling does with that is anyone's guess.

At least RawTherapee allows to directly influence the RGGB BlackPoints, or one could resort to the demonstrated benefit of averaging multiple exposures, or even use a masked combination of regular ETTR exposure + Shadow exposure bracket, where we can align the exposures in Raw processing and replace the clipped regions of the shadow exposure with the ETTR data with a mask (the method that Hans Kruse recently mentioned in a thread here, and Guillermo Luijk before that with his ZeroNoise program).

Cheers,
Bart

[ErikKaffehr]

Hi,

I will make som more exposures with an ND-filter, and check with RawDigger. This was just a first attempt.

Best regards
Erik

I think you need to lower it considerably more. Look at those church benches of voidshatter's example shot, it's that kind of light level, ie almost black that is raised. One experiment could be to lower step by step until you start see color shifts to see where the limit is.

Also note that color profile may have hue shifts and might affect the result, so it would be better if you used a linear profile (ie only a matrix) when doing these experiments.

[Hans Kruse]

I think most raw converters overdo noise reduction. I think it's because they want to make a clean pixel peep impression rather than thinking about how global color look in a print. Some grain is no problem for a print, but dead color is.

Concerning my observation that there might be not enough photons to give good colors in pushed shadows it's indeed a speculation, trying to explain what I see in many shots. I haven't yet looked at the above shots, but I guess 2 pushed stops may not be enough to show a significant effect, I'm thinking more like 4 stops.

What I often see in backlit landscape scenes with pushed shadows is that there's often a brown(ish)-centered palette in the pushed areas, color does not look as good in the well-exposed areas. I have not investigated in detail why this is, but have speculated that it's too few photons captured to make good color.

I sometimes see quite different color in pushed shadows. Here is one example from the Nikon D810 in a Scottish landscape and shot at ISO 64.

The first picture is a screen shot from Lightroom. The left hand is a -3EV exposure and the right hand is -1EV. Unfortunately I don't have more brackets. The edits on both exposures are exactly the same and same WB. The only difference is the exposure setting in Lightroom.

The second is the entire scene. This blended from the -3EV and -1EV and the bridge and landscape around it is all from the -1EV and has the right colors.

The third is the -3EV with no edits. So due to the very strong light the shadows are very much underexposed.

[voidshatter]

Great example Hans! This is exactly the randomness of shadow noise. For me this is nothing really worth to worry about as I would never push it that hard. The Sony CMOS sensor (e.g. D810, IQ250 etc) has 3 stops of more room to abuse than any CCD digital backs. If I really have to push it that hard, I would just fix it using something like "Split Toning" in Lightroom or ACR. Landscape is subjective after all, just like the CCD fans would insist the "CCD-look" or "CCD-color" to be special, so even with some color cast it is totally fine and can be regarded as romantic.

[Ken R]

I sometimes see quite different color in pushed shadows. Here is one example from the Nikon D810 in a Scottish landscape and shot at ISO 64.

The first picture is a screen shot from Lightroom. The left hand is a -3EV exposure and the right hand is -1EV. Unfortunately I don't have more brackets. The edits on both exposures are exactly the same and same WB. The only difference is the exposure setting in Lightroom.

The second is the entire scene. This blended from the -3EV and -1EV and the bridge and landscape around it is all from the -1EV and has the right colors.

The third is the -3EV with no edits. So due to the very strong light the shadows are very much underexposed.

Hans nice shot!

Regarding the exposure don't you think -3 was a little too extreme and not required? I mean, from looking at the final result didn't -2 or even -1.5 work? or did it blow the highlights too much.

I think its tough to compare a CCD sensor and a Sony CMOS sensor since both need to be exposed differently (and even between CCDs) IMHO everything else being equal (base ISO, scene, lighting etc). I found that the CCD in my IQ160 held more highlight information with a smoother curve into blow highlights than the Nikon D800E (both at base iso). I could be more "aggressive" with the IQ160 exposure (brighter) compared to the Nikon (in which I had to protect the highlights with exposure more). With my Canon's it was somewhere in between those two. This is obviously before making any curve (or any) adjustments to the raw file, just a linear conversion. FWIW, the 645D was worse than all three. It blew the highlights pretty abruptly and earlier than all the other cameras.  

Again, I don't know and really don't care why that happens (science / physics) but that is what happens when using the cameras for photography.

[Hans Kruse]

Hans nice shot!

Regarding the exposure don't you think -3 was a little too extreme and not required? I mean, from looking at the final result didn't -2 or even -1.5 work? or did it blow the highlights too much.

I think its tough to compare a CCD sensor and a Sony CMOS sensor since both need to be exposed differently (and even between CCDs) IMHO everything else being equal (base ISO, scene, lighting etc). I found that the CCD in my IQ160 held more highlight information with a smoother curve into blow highlights than the Nikon D800E (both at base iso). I could be more "aggressive" with the IQ160 exposure (brighter) compared to the Nikon (in which I had to protect the highlights with exposure more). With my Canon's it was somewhere in between those two. This is obviously before making any curve (or any) adjustments to the raw file, just a linear conversion. FWIW, the 645D was worse than all three. It blew the highlights pretty abruptly and earlier than all the other cameras.  

Again, I don't know and really don't care why that happens (science / physics) but that is what happens when using the cameras for photography.

Ken,

I have attached the full image of the -1EV that I merged with the -3EV. I bracketed the shots as I always do and then I choose in Lightroom which exposure is optimal (the most exposed with no essential highlights clipped). If the optimal exposure looks good after editing then I'll just go with that. If not I will blend the shot with another one from the bracketing sequence (if it exists). If I find the noise in the shadows a bit too much then I will copy the edits from the exposure I have edited to all other pictures in the bracketing sequence (including all parameters) and then I will match the exposure (shift-alt-cmd-M). The I will check the highlights and shadows in the other exposures. If none can be used standalone and this does not happen that often, then I will do the blending. The blending of two exposures (one for then highlights and one for the shadows) will be done in Photoshop as layers. This is easy to do since the edits is exactly the same and the exposure difference has been eliminated. It's just a few clicks and shortcuts in Photoshop and it is done.

This is basically my shooting style no matter if I shoot Canon or Nikon. For the Phase One bracketing was slow and tedious but I did it when needed. On the Canons and Nikons I can fire off the bracket sequence in continous mode in live view for at least focus lengths less than 100mm. For more than that I need to be more careful.

I like to know what happens and what is the explanation for things, but my shooting style is optimized towards not having to deal with histograms and exposure compensation, filters etc. I want a completely mechanical style where the only thing I concentrate on is to find the best viewpoints, the best compositions and sometimes this is within minutes and time is precious.

[Hans Kruse]

Great example Hans! This is exactly the randomness of shadow noise. For me this is nothing really worth to worry about as I would never push it that hard. The Sony CMOS sensor (e.g. D810, IQ250 etc) has 3 stops of more room to abuse than any CCD digital backs. If I really have to push it that hard, I would just fix it using something like "Split Toning" in Lightroom or ACR. Landscape is subjective after all, just like the CCD fans would insist the "CCD-look" or "CCD-color" to be special, so even with some color cast it is totally fine and can be regarded as romantic.

Thanks and I agree very much. I did not consider the -3EV shot as ruined even though I pushed it hard, but found that the blended result was better and what I wanted to preserve. In fact I'm not sure anybody would notice the flaws in the -3EVF shot

[voidshatter]

Thanks and I agree very much. I did not consider the -3EV shot as ruined even though I pushed it hard, but found that the blended result was better and what I wanted to preserve. In fact I'm not sure anybody would notice the flaws in the -3EVF shot

Wow I just noticed that you are the famous mighy Hans at 500px! I don't follow many there but I started following your shots a long while ago! Damn your telephoto landscape shots are just too great which really makes it hard to justify heavy investments into MFDB! I would perhaps go back to Canikon again 

[Hans Kruse]

Wow I just noticed that you are the famous mighy Hans at 500px! I don't follow many there but I started following your shots a long while ago! Damn your telephoto landscape shots are just too great which really makes it hard to justify heavy investments into MFDB! I would perhaps go back to Canikon again 

Thanks I also have a few there from the Phase One. E.g. this one https://500px.com/photo/58213766/lenticular-clouds-forming-over-sassolungo-by-hans-kruse

[ErikKaffehr]

Hi,

I had the pleasure to take part in one of Hans's workshops.

As a small observation, I just happen to have a small exhibition hanging on the theme Mountains, Valleys and Water. Albeit I have been shooting MFD for something like 20 months, none of the MFD images that made it to that exhibition. I don't know why, honestly. One reason was probably that I don't carry the MFD kit for long walks.

The images are here: http://echophoto.dnsalias.net/ekr/Articles/Exhibitions/BergDalOchVatten_1/

I have shot some very nices images with the MFD stuff at home.

Best regards
Erik

Wow I just noticed that you are the famous mighy Hans at 500px! I don't follow many there but I started following your shots a long while ago! Damn your telephoto landscape shots are just too great which really makes it hard to justify heavy investments into MFDB! I would perhaps go back to Canikon again 

[ErikKaffehr]

Hi,

A longer answer…

Just to say, I have not really seen colour shifts in the darks on my P45+, but decided to look into that a bit. My background is in engineering, so I have a tendency to look fora feasible explanations for any observation. With MFD there are many myths, little observations and very few explanations.

I have a couple of tools that I originally acquired to do "sanity checks" on my printer profiles. One of those tools is PatchTool from BabelColor. That tool can read a pair of CGATS files and do a graphical comparison between the colours. Very useful.

A while ago I got interested in looking into the accuracy of colour reproduction on my P45+ compared to my Sony Alpha 99, and I was looking into getting a test chart with more colours than the classic Colour Checker. The Colour Checker SG would be a natural alternative, but it is a bit expensive and reference values are not easy to obtain.

Than I realised I could use an IT8 colour chart. It has a lot of fields and covers most parts of Adobe RGB and they always come with reference data:

http://www.targets.coloraid.de

I used the C1 target, the reference data is here: http://echophoto.dnsalias.net/ekr/Articles/darkrepro/Ref/R050301_remap.txt (note the fields are remapped for easier use)

Now, I made 5 different shots with 0, -2EV, -4EV, -7EV and -6EV exposures (the last two using a 3X ND filter)

Those images are here: http://echophoto.dnsalias.net/ekr/Articles/darkrepro/Originals

I made raw conversions from the 0EV and -4EV images using both LR5 (using process version 2010) and Capture One compensating for underexposure, saving the results as 16 bit TIFFS.

Next step was to use the "scanin" program from Argyll CMS to read the IT8 field values from TIFF files into CGATS files.

The last step was to read the reference file and the CGATS files from scanin into PatchToo and evaluate the differences.

I didn't go beyond -4EV  as I wanted to compensate for the underexposure in LR, and LR with PV 2010 only goes to +4EV compensation

The include screen dumps from RawDigger show the three darkest grey patches in RawDigger from the -6EV exposure.

I would think that once exposure is so small that readout noise "contaminates" shot noise there would be a colour shift as the different channels would be differently affected, and that colour shift would vary by white balance and CFA design.

Best regards
Erik

This is nice. What kind of color passport did you shoot? I'd like to test it as well!  

[voidshatter]

Thanks I also have a few there from the Phase One. E.g. this one https://500px.com/photo/58213766/lenticular-clouds-forming-over-sassolungo-by-hans-kruse

Obviously you have got more keepers from Canikon gear! Now Canon offers the 5DSR with choices like 11-24mm f/4 L and we also had the light-weighted 70-200mm f/4 L. If I don't shoot long exposures (hence no alignment issues with filters), then with your bracketing technique to deal with high dynamic range I see no problem with the predicted inferior dynamic range of the 5DSR. It is even harder to justify MFDB now.

[voidshatter]

This is great Erik. I have the same color passport as yours. When I have my lens back I will shoot it with an IQ250 and ask you to run the color check program

[alatreille]

All subjective, but my 1 month with my Credo 60 sees me feeling as follows:
Credo - Protect the shadows
Canon - Protect the highlights.

ie as Ken mentioned, I feel I can use longer exposures with the credo in comparison to the Canon (given equivalent aperture/iso combo)
I haven't used a sony cmos, but would like to soon

[ErikKaffehr]

Hi,

What Hans is advocating is to "repair" highlights using a lower exposure rather than repairing the "shadows" using a higher exposure. It is pretty smart, as the highlights are often sky/sunlight phenomena which mix well. He was kind enough to demonstrate it on my own images and he got better results than I was able to achieve.

I have been shooting Sony's since 2006, and "never" had issues with DR, it was always there when needed. This is of course a moving target.

Another side of the coin is that tone mapping is needed to properly render a high DR exposure.

I started using HDR-mapping on single exposures, but I gave up on the technique when LR4 with PV 2012 arrived. I felt that LR 4 did a much more subtle tone mapping than what I could achieve with Photoshop tools albeit I feel I would like to have a bit more of control.

Best regards
Erik

At least RawTherapee allows to directly influence the RGGB BlackPoints, or one could resort to the demonstrated benefit of averaging multiple exposures, or even use a masked combination of regular ETTR exposure + Shadow exposure bracket, where we can align the exposures in Raw processing and replace the clipped regions of the shadow exposure with the ETTR data with a mask (the method that Hans Kruse recently mentioned in a thread here, and Guillermo Luijk before that with his ZeroNoise program).

Cheers,
Bart