An example of the shadow limitations of a MF CCD

[ErikKaffehr]

Hi,

In my experience MFD uses a pair of tricks to protect highlights

- Overrating ISO
- Biasing in processing

If you use Capture One, try out film curve versus linear response. With "film curve" the images will be far to bright so you learn to underexpose to protect highlights.

The only difference between the enclosed images is that the brighter one uses "Film Curve" while the darker one uses "Linear Response".

The last image shows the "true" histogram of that shot, using RawDigger which shows the actual data in the raw file. Very clearly, that image could take one more EV of exposure without clipping, but I am pretty sure that camera histogram was on the brink of overexposure, as I always expose ETTR

Best regards
Erik

All subjective, but my 1 month with my Credo 60 sees me feeling as follows:
Credo - Protect the shadows
Canon - Protect the highlights.

ie as Ken mentioned, I feel I can use longer exposures with the credo in comparison to the Canon (given equivalent aperture/iso combo)
I haven't used a sony cmos, but would like to soon

[voidshatter]

Erik, could you test color shift of shadow pushed by 4ev on your Sony camera (preferably a sensor similar to the IMX094)? Thanks!

[ErikKaffehr]

Hi,

I have already done that, but I was not happy with exposure so I will remake that experiment. Coming in a couple days. The Sony I intend to test is the Alpha 99.

Best regards
Erik

Erik, could you test color shift of shadow pushed by 4ev on your Sony camera (preferably a sensor similar to the IMX094)? Thanks!

[Bart_van_der_Wolf]

Hi,

What Hans is advocating is to "repair" highlights using a lower exposure rather than repairing the "shadows" using a higher exposure.

Indeed, but he uses the shadows exposure (with better shot noise) for the majority of the image, except for the clipped highlights which get replaced by the ETTR exposure. Both Raw conversions were synchronized for exposure setting, so the blend between them is virtually seamless. t works quite well, also because one doesn't have to do a huge amount of additional tonemapping compared to a single exposure. It's just that the shadows and mid-tones have been better exposed, have lower noise, and thus allow more manipulation.

It is pretty smart, as the highlights are often sky/sunlight phenomena which mix well.

Yes, it does work well. The only issue can be e.g. moving clouds that could have poorer registration between images.

Another side of the coin is that tone mapping is needed to properly render a high DR exposure.

Yes, but then all images require some sort of tonemapping, because a print has a much lower DR than e.g. a backlit panel or a display can have.

Cheers,
Bart

[torger]

"Linear curve" in most commercial raw converters are usually still not linear, a common thing they do is to clip away the last bit of highlights, and from the Capture One example above it seems like it does exactly that. It's prbably recoverable by just lowering exposure though.

I did a test yesterday with C1 to check if the ICC profile is applied before or after exposure adjustment (and curve), and it's applied after. I kind of new this already as you can export a TIF with the camera ICC, meaning that the ICC must be applied in the end. Phase One's ICC profiles has hue twists by the way, just as Adobe's DCPs typically have. I think Phocus own profile format is wihtout hue twists, but I haven't verified it for sure.

Anyway, this is a weakness of the ICC format, with Adobe DCP you can apply one correction at the raw before exposure adjustments tonemapping etc, and then apply one after. The first is intented to correct color (and they recommend against hue twists there), the other is intented to apply a look (and generally contains hue twists, eg saturation increase in shadows as many like that).

If there is a static color shift in the darkest shadows, that could be corrected with a DCP profile (using hue twists in the first HueSatDelta correction table), but it's not possible to correct with C1's ICC. In any case there's noone that does it today, but it could be an interesting new development in camera profiles. If it's like voidshatter says that the cast varies over time it's not possible to correct anyway though.

RawTherapee supports C1 ICC files, but it's color pipeline is designed such that profiles should only correct for accuracy and look is up to the user using the raw converter tools (which is a flow I prefer for my own photography), this means there is only one profile application step and that is before any adjustments, so if you push a heavily underexposed file the hue twists in the ICC will make it all look wrong, so to use C1 ICCs and indeed Adobe DCPs with a "look" in RT with desired result you need to have a file which has the right exposure from the start. RT's own DCPs are without hue twists and designed for accuracy at D50.

[ErikKaffehr]

Hi,

I shot some decent samples with my P45+ and my Sony Alpha 99 SLT.

Exposures were 0, -2, -4, -6 and -7 EV. The two lowest exposure were achieved using ND filter. The 0, -4EV was using studio flash and varying intensity.

Raw images are here: http://echophoto.dnsalias.net/ekr/Articles/darkrepro/RawFiles/

This image shows tonal shifts caused by underexposing 4EV. P45+ shows less tonal shift and SLT99 more. All processing I tried gives this result.
CGATS files are here: http://echophoto.dnsalias.net/ekr/Articles/darkrepro/CGATS


This is 0EV and -4EV comparison on P45+ in Lightroom

Original: http://echophoto.dnsalias.net/ekr/Articles/darkrepro/ScreenDumps/SSLT99_4EV.png
And this is 0 and -4EV comparison on SLT 99 in Lightroom

Original: http://echophoto.dnsalias.net/ekr/Articles/darkrepro/ScreenDumps/SSLT99_4EV.png

For some reason, the SLT99 images are in 12 bit, the Sony cameras choose between 14-bit and 12-bit a bit arbitrarily, single exposure used to be 14-bits. I did some raw digging, but I have not found anything sensational.

Best regards
Erik

[voidshatter]

Great tests! The IMX128 seems to hold strong. Would be curious to see how IMX094 performs (preferably on a lossless one such like the D800E).

If you ever turn on any of the following settings on a Sony EVF camera you would get 12-bit RAW (actually around 1300 levels for each channel, 10.4-bit equivalent) instead of 14-bit RAW (actually around 1700 levels for each channel, 10.7-bit equivalent):

Long exposure noise reduction (darkframe NR)
B mode (bulb mode)
continuous shooting
speed priority continuous shooting
silent shutter (for the A7S)

A while ago when I did an article about how the lossy compresison of the Sony EVF camera impacts image quality, I did a stress test on shadow push of long exposure shots. Below shows a comparison between the A7 (left) and the D610 (right):



As can be seen, even if the same neutral area was selected for the "Pick White Balance Tool", the shadow still had different degree of color casts. Note that the above was both multi-sampling (temporal noise reduction) from 30 frames each. Our first speculation was that the lossy compression of the Sony RAW had too many 0s (zero's) in the shadow, resulting in demosaicing failures regarding color fidelity.

[ErikKaffehr]

Hi,

If the IMX094 is the one used in the 36MP cameras it seems to be an (even) better performer, although it has smaller pixels. I have no access to any Nikon FF, unfortunately. I know a lady who has a D810 (or some other of the 800 models), but I am pretty sure she is using it for shooting and earning money and is less than enthusiastic about pixel peeping.

Once Sony has next generation high pixel camera I will probably buy. I don't want to switch systems, and I feel that Sony finally got it right with the A7 models. Hopefully they will make an A9.

Best regards
Erik

Great tests! The IMX128 seems to hold strong. Would be curious to see how IMX094 performs (preferably how a lossless one such like the D800E).

If you ever turn on any of these on a current Sony camera you would get 12-bit RAW (actually around 1300 levels for each channel, 10.4-bit equivalent) instead of 14-bit RAW (actually around 1700 levels for each channel, 10.7-bit equivalent):

Long exposure noise reduction (darkframe NR)
B mode (bulb mode)
continuous shooting
speed priority continuous shooting
silent shutter (for the A7S)

[voidshatter]

Hi,

If the IMX094 is the one used in the 36MP cameras it seems to be an (even) better performer, although it has smaller pixels. I have no access to any Nikon FF, unfortunately. I know a lady who has a D810 (or some other of the 800 models), but I am pretty sure she is using it for shooting and earning money and is less than enthusiastic about pixel peeping.

Once Sony has next generation high pixel camera I will probably buy. I don't want to switch systems, and I feel that Sony finally got it right with the A7 models. Hopefully they will make an A9.

Best regards
Erik

I have added more info in my reply above (a comparison between an A7 and a D610).

I agree with you that the Sony A series is really a success. I will also most likely buy an A9. The portability is hard to resist!

I have a D800E by my hand. Would it work if I shoot the X-rite color check passport and upload the RAW files to you?

[ErikKaffehr]

Hi,

yes, of course, but my colour shift tests are based on the C1 target on this page: http://www.targets.coloraid.de

If shooting the Color Checker Passport, please try to expose ETTR so the white field is near saturation.

Thanks for the info Sony exposure modes. My images used to be "14-bit".

Best regards
Erik

I have added more info in my reply above (a comparison between an A7 and a D610).

I agree with you that the Sony A series is really a success. I will also most likely buy an A9. The portability is hard to resist!

I have a D800E by my hand. Would it work if I shoot the X-rite color check passport and upload the RAW files to you?

[voidshatter]

IQ250 RAW files: https://www.dropbox.com/sh/fgircd45mbjf62r/AAC2evO6UQMAOxmQpLccyzmfa?dl=0

[ErikKaffehr]

Hi "Void",

Downloading them right now. Will take a look the coming few days and report back.

BTW, thanks for sharing your comparison between Sony and Nikon. Personally, I don't think it is the compression that is causing the difference, but it is interesting to see how much cleaner the Nikon data is.

Best regards
Erik

IQ250 RAW files: https://www.dropbox.com/sh/fgircd45mbjf62r/AAC2evO6UQMAOxmQpLccyzmfa?dl=0

[AlterEgo]

Anyway, this is a weakness of the ICC format, with Adobe DCP you can apply one correction at the raw before exposure adjustments tonemapping etc, and then apply one after. The first is intented to correct color (and they recommend against hue twists there), the other is intented to apply a look (and generally contains hue twists, eg saturation increase in shadows as many like that).

format or converter... nobody prevents your converter to be written in such manner so that it will use ICC container to store pre and post adjustment corrections... icc container can store multiple sets of color transforms - you might argue that injection of converter adjustments (done by converters code, reflecting user action in UI and not guided by purely the data in icc container) is not exactly what ICC (organization) prescribed, but then who cares really - just document the use... no ?? what do you say

[torger]

format or converter... nobody prevents your converter to be written in such manner so that it will use ICC container to store pre and post adjustment corrections... icc container can store multiple sets of color transforms - you might argue that injection of converter adjustments (done by converters code, reflecting user action in UI and not guided by purely the data in icc container) is not exactly what ICC (organization) prescribed, but then who cares really - just document the use... no ?? what do you say

Good point, that can work. ICC profiles are already used today in "non-standard" ways when used for photography. The ICC profile expects some sort of pre-processing, the Capture One native ICC profiles require a 1.8 gamma to be applied to the input for example if I remember correctly.

[Bart_van_der_Wolf]

Good point, that can work. ICC profiles are already used today in "non-standard" ways when used for photography. The ICC profile expects some sort of pre-processing, the Capture One native ICC profiles require a 1.8 gamma to be applied to the input for example if I remember correctly.

Hi Anders,

That has been mentioned by some, but that's only a very rough approximation.
For example, the profile for my 1Ds3 is reasonably well (an R^2 of 1.0 would be perfect)
fitted by something close to a gamma of 1/1.94 (the c-parameter in the attached curve fit).

Canon 1Ds Mark III, Green TRC from generic profile by Phase One

Formula: y = b*(x-a)^c*exp(-(x-a)/d)

Parameters:
   a = 2.89348
   b = 3816.21694
   c = 0.51478
   d = 1.11691E13
R^2: 0.99808

But indeed, there are assumptions involved which the Raw converter should convert the data to.

Cheers,
Bart

[torger]

Thanks for the IQ250 examples voidshatter. There's impressive performance in there. I assume the 1726 is the +0? (the 1727 is otherwise almost perfect ETTR on the raw level).

Using linear color conversion in RT shadow neutral color shift towards green is evident at 1720 (+6) but also slightly visible at 1722 (+4), then becoming invisible.

I looked at some Aptus 75 shots I did for this test: http://www.ludd.ltu.se/~torger/photography/noise-test.html back in 2012, and I'd say that although the +4 shot is noisier and the +6 shot is unusable (tiling visible), the visual color stability is about the same, ie a cast starts showing (also green on the Aptus) but not really much more evident on the Aptus 75 than on the IQ250.

It does re-inforce my view that what you gain with the Sony sensor compared to old-school CCDs is about 2 stops, but the value of those extra two stops is a bit limited due to the weaker color fidelity down there. I guess my conclusion is that the CCDs have quite good signal, they just overlay it with lots of read noise which takes away some shadow push capability.

[AlterEgo]

The ICC profile expects some sort of pre-processing

why ? you can bake the curve in ICC trcs/luts and feed totally linear (g1) demosaicked data in the pipeline - if your converter allows/does that

[ErikKaffehr]

Hi Anders,

That is what I would expect from spec sheets and DxO data. But, darks are darks and I don't think we look for good colour in barely visible parts of the image. What we want to see is a good and nice poisson distributed noise without colourful salt and pepper noise.

Best regards
Erik

…
It does re-inforce my view that what you gain with the Sony sensor compared to old-school CCDs is about 2 stops, but the value of those extra two stops is a bit limited due to the weaker color fidelity down there. I guess my conclusion is that the CCDs have quite good signal, they just overlay it with lots of read noise which takes away some shadow push capability.

[torger]

That has been mentioned by some, but that's only a very rough approximation.

I'm a bit rusty on this aspect of ICC profiles but I think it works like this in C1, the TRC curves are not used (you can strip away them and it will make no difference), instead conversion is made with the A2B0 table which converts from RGB to Lab with 1.8 gamma, so if you work in linear gamma floating point you need to apply gamma 1/1.8 (plus an expected film curve) before conversion, and then revert curve and gamma if you want to continue in linear space.

[voidshatter]

Thanks for the IQ250 examples voidshatter. There's impressive performance in there. I assume the 1726 is the +0? (the 1727 is otherwise almost perfect ETTR on the raw level).

Using linear color conversion in RT shadow neutral color shift towards green is evident at 1720 (+6) but also slightly visible at 1722 (+4), then becoming invisible.

I looked at some Aptus 75 shots I did for this test: http://www.ludd.ltu.se/~torger/photography/noise-test.html back in 2012, and I'd say that although the +4 shot is noisier and the +6 shot is unusable (tiling visible), the visual color stability is about the same, ie a cast starts showing (also green on the Aptus) but not really much more evident on the Aptus 75 than on the IQ250.

It does re-inforce my view that what you gain with the Sony sensor compared to old-school CCDs is about 2 stops, but the value of those extra two stops is a bit limited due to the weaker color fidelity down there. I guess my conclusion is that the CCDs have quite good signal, they just overlay it with lots of read noise which takes away some shadow push capability.

In your ancient test the D7000 (a smaller version of the IQ250/D800E) already outperformed the 5D2 and the Leaf CCD. Far before you see color casts the SNR becomes an issue first. If you try long exposure you would see how limited the CCD backs are. It wouldn't be just 2 stops. It would be something like 3 stops, which is a big deal.