5D3 vs Nikon 800 @ 25600

[32BT]

Have been playing around with the files over at IR.

I am very impressed by the Nikon. I think they got the AA filter exactly right. Will show some samples later.

I thought that this comparison was interesting as well. The ISO 25600 images compared.

NOTES:
- I don't have color profiles for either camera. They were both converted with a canon profile. You would be surprised by the way how well that works under normal circumstances…

- The Nikon file has been scaled down by 80% to create comparable images. In my personal opinion that is the more useful comparison for high iso images. YMMV

- The canon file has been adjusted slightly for creating similar contrast. Problem with the Nikon RAW files is that they already have the blackpoint scaled down to zero.

[32BT]

The Nikon version of the resolution/ColorChecker file at normal iso.

1) the RAW data with equalized channels,
2) a possible conversion.

I presume the camera tested at IR was the one WITH AA filter, but looking at the file, one could easily be mistaken…

One disconcerting issue is the blooming-like blue sheen around the high-contrast letters "x-rite". The lens used doesn't seem to be optically optimal, nor does it seem focussed properly and/or wrong aperture setting, so it could be a lens issue.

[32BT]

Following is a real world image available from www.rawsamples.ch

It contains a fence around a bend which nicely goes to Nyquist. Because it is a fence, we have a certain expectation of seeing more verticals towards infinity. Interestingly enough, for our eye/brain combination, this just has to be a hint. This is why paintings work so well, if the brush strokes are placed in the same direction as the expected edges, the corresponding structure in the paint creates the correct illusion of extended detail.

And I personally believe that it works exactly similar for RAW conversion. Clearly we didn't sample the exact detail, but the remaining detail hints are enough to produce useful edge information for real world image reproductions. This is why I believe that filterless should work fine in most cases, assuming that the sensel-wells are close enough via microlenses or something similar. And IF a filter is chosen, then at least it shouldn't blur ALL hints of "incorrectly" sampled data.

From the file in post 2 I currently conclude that the AA filter is doing exactly the right thing in the Nikon, and it could be very well possible that the filterless version does not bring any useful gain. Given that both the filtered version as well as the filterless version use the same amount of layers, I do not believe that the a-priori microcontrast should in any way be better from either version. But this conclusion should probably wait for real world experience with the camera…


EDIT: the fence samples are from a D3, not from the 800...

[marcmccalmont]

How about a comparison of DR at base ISO?
Marc

[32BT]

How about a comparison of DR at base ISO?
Marc

Valid request. A bit problematic though, because:
1) i first need to know what the base iso values are,
2) the definition of the blacklevel in the Canon is not strictly defined,
3) we need controlled images which actually exhibit useful dynamic range…
 
But even so, I think the following pictures would suggest that at lower iso values the camera's dynamic range response is quite similar:

pict 1 & 2 show the dynamic range of the RAW source data, Canon's black level is fixed at 2047, which is the most likely value.
The red line depicts the absolute average which maps to an Fstop of about -4 for both camera's, and the default image reproduction is included to show that the reproduction at least is comparable, so that the dynamic range response values are also most likely comparable (as in not trying to compare apples and oranges).

pict 3 & 4 show the histograms for total-captured-data-equalized-for-gray with a perceptual gamma curve applied.
Minute differences in the Fstop histogram can make significant differences in normal viewing space. The perceptual histograms however also suggest no specific or extreme differences…

[32BT]

Sometimes it's also a matter of just doing the right thing. Obviously, it is not very useful to compare raw output to in-camera jpg, but whatever the F it is that Canon is doing here for sharpening, it is clearly doing the exact opposite of what it is supposed to do… There is heaps of microcontrast available, they are just not extracting it properly from their imagedata.

This also serves as a great example of how "overprocessing" can be detrimental to an image, and actually create an uneasy viewing experience. Plus the image is totally unsuitable for any further processing. (additional sharpening for different output requirements for example).

Left is raw conversion, right is in-camera jpg:

[Bart_van_der_Wolf]

Valid request. A bit problematic though, because:
1) i first need to know what the base iso values are,
2) the definition of the blacklevel in the Canon is not strictly defined,
3) we need controlled images which actually exhibit useful dynamic range…

Hi Oscar,

The Base ISO for the 5D3 is ISO 100 because that maximizes the Dynamic range.
The black level for Canon is usually 1024. The file data can fluctuate a bit, it's noise afterall.
Controlled images will have to wait for the release of actual production models, unfortunately.

Cheers,
Bart

[32BT]

The black level for Canon is usually 1024. The file data can fluctuate a bit, it's noise afterall.

Yes, that used to be the case, at least for the 40D and 5D2, but I consistently find 2047 for the 5D3 as well as the 7D.
In the mean time I have adapted the DCRaw code to understand the margins of the 5D3, and it too reports 2040-ish numbers based on the black margins...

Also, comparing ISO 100 and up gives very consistent results, where the average seems to map perfectly to -4, which seems a very plausible value depending on the metering mode of course.

The only exception is ISO 50 which seems to be overexposed by exactly 1 stop…
i.e. it is shifted by 1 stop; the average maps 1 stop higher; but it does exhibit highlight clipping.
This I presume is consistent with ISO 50 not being a true ISO setting, but merely a digital adjustment.

attached: iso 50 (on the left) iso 100 and 200

[BJL]

Bart and Oscar,

Sorry to interrupt this high level technical discussion with a beginner's question, but what do you mean by black level? I was guessing the level in digital output below which everything is intended to be displayed as black, but values like 1024 or 2047 seem way too high for that.

Thanks in advance.

[32BT]

Bart and Oscar,

Sorry to interrupt this high level technical discussion with a beginner's question, but what do you mean by black level? I was guessing the level in digital output below which everything is intended to be displayed as black, but values like 1024 or 2047 seem way too high for that.

Thanks in advance.

Short answer:
Your guess is right.

Longer answer:
I believe it is the minimum switching level of the sensor transistors above which they respond more or less linear to the amount of light. You will find lower values, but these are either non-linear and/or not consistent. Some transistors will switch, some won't, and therefore they will just produce noise. However, on a larger surface area, the number of transistors that respond will be consistent with the amount of light, so you will be able to discern darker black levels, as long as the area represents some flat tone, but detail in those areas will be completely lost to noise.

See attached example. The black garment contains silvery gray curls which are obviously completely invisible in the noise. However, you can still easily discern the darker edge. (This btw is the reason that it is not useful to preclip the data at the suggested noise-floor, like Nikon does).

[Bart_van_der_Wolf]

Sorry to interrupt this high level technical discussion with a beginner's question, but what do you mean by black level? I was guessing the level in digital output below which everything is intended to be displayed as black, but values like 1024 or 2047 seem way too high for that.

Hi,

It's the Black level, an offset used by the ADC, and is used to allow accurate Read Noise analysis.  On the pre-production Raws at Imaging Resource it indeed looks like the 5D3 uses an elevated Blackpoint in the order of 2048, presumably to accommodated unclipped ISO 102400 technical data with Read noise that can go very low on the lower tail of the Gaussian pattern.

Cheers,
Bart

[32BT]

Bart,

In the highest iso images I seem to measure additional gaussian bumps, especially in green. Any suggestions what causes these?

It's almost as if they try to stretch the incoming values over too little available bits in ADC…!?

[Bart_van_der_Wolf]

Bart,

In the highest iso images I seem to measure additional gaussian bumps, especially in green. Any suggestions what causes these?

Hi Oscar,

It depends on what you are measuring. When you inspect the technical sensels (especially visible on the highest ISO settings) with a utility like 'Rawdigger', there are several areas, it's not a uniform signal area at the top of the image.

It's almost as if they try to stretch the incoming values over too little available bits in ADC…!?

From an earlier analysis I reached a conclusion that the "unity gain" level (where 1 converted Photon changes the DN by 1 unit) is probably around ISO 800. That means that higher ISO settings will start to show histograms with combing, the gaps obscured a bit by amplified noise. So in a sense yes, the actual Digital Numbers (DN) or Analog to Digital Units (ADU) are spread thin above unity gain. What else is going on is hard to analyse without proper test files of boring uniform exposured featureless areas. We'll probably need to wait for production bodies to become available.
 
Cheers,
Bart

[jrsforums]

Hi,

It's the Black level, an offset used by the ADC, and is used to allow accurate Read Noise analysis.  On the pre-production Raws at Imaging Resource it indeed looks like the 5D3 uses an elevated Blackpoint in the order of 2048, presumably to accommodated unclipped ISO 102400 technical data with Read noise that can go very low on the lower tail of the Gaussian pattern.

Cheers,
Bart

Bart....please....translate for us commoners. :-)

....and is this higher black level good, bad, or depends..?

John

[Bart_van_der_Wolf]

Bart....please....translate for us commoners. :-)

....and is this higher black level good, bad, or depends..?

Hi John,

The analog voltage that results from exposure is read from each sensel, and converted to discrete (natural) numbers by the Analog to Digital Converter (ADC). The ADCs we are talking about produce Digital Numbers (DN), also called Analog to Digital Units (ADU), in the range of 0 to 16383 (for a 14 bit ADC). A few of the highest DNs are not utilized.

In that read-out and conversion process, noise is unfortunately added by the electronics. It would require super cooling to avoid that. So even if there was no exposure, zero exposure signal, there will still be a randomly fluctuating DN coming out of the ADC. That noise will have a Gaussian distribution around its zero signal average, and thus negative numbers which we cannot encode when zero is the lowest DN.

Some manufacturers (e.g. Nikon) have chosen to truncate that Gaussian distribution, and they output the zero input signal level as zero output level. That also means that they discard some information that would be smaller than zero, the lower half of the Gaussian noise distribution. So their Raws do not contain all the info (noise is also info) that comes out of the AD conversion.

Other manufacturers (e.g. Canon) have chosen to keep all that info, and therefore add a fixed offset to all of the output DNs for a given ISO. While that reduces the number of individual DNs that can be assigned to the different input signal levels, it also allows things like advanced DarkFrame subtraction. That can be of particular interest to people who need to do number crunching on the Raw data such as Astro photographers who combine many Raw data frames to reduce the lowest noise levels statistically before Demosaicing.

In this case it also can help us to easily and directly determine the Read Noise amount, a parameter used to calculate the Dynamic Range (DR) of our sensors. Therefore it is useful to be able and read that metric directly from the Raw data, and use it when we can benefit from having the info.

Canon used to use a fixed Black level offset of 1024, but in the 5D3 that level is apparently increased to 2048 for the lowest ISO settings (assuming the pre-production Raw data encoding is similar to that of the production models). It does reduce the remaining number of DNs that can be used for exposure signals, but there are still more than 14000 discrete levels (some 13.8 bits out of the 14 we started with) available which should be enough to smoothly render gradients, even after Gamma precompensation for display.

Here is the result from some measurements I did with 'Rawdigger' on the pre-production Raws at Imaging Resource. I chose a rectangular area of the technical sensels outside the imaging area. That area seemed rather uniform in all channels and at all ISOs, so I hope that it is representative of the read-noise we will get in the image area itself. The first three columns are ISO and exposure data, then 4 columns of average signal level (the Black level offset), and finally 4 columns with the Read-Noise Sigmas for the color channels:

Canon 5D Mark III - Read noise - 14-bit numbers

ISO     T       A    R       G1      B       G2       Sr     Sg1    Sb    Sg2
    50  1/6     8    2048.4  2048.0  2047.2  2048.3    6.4    6.4    6.3   6.3
   100  1/12    8    2048.6  2047.8  2047.0  2047.8    6.4    6.3    6.2   6.3
   200  1/25    8    2048.8  2048.2  2047.4  2047.7    6.5    6.5    6.5   6.5
   400  1/49    8    2048.2  2048.9  2047.9  2045.2    6.7    6.6    6.5   6.6
   800  1/99    8    2049.0  2048.7  2047.8  2048.0    6.7    6.6    6.5   6.7
  1600  1/197   8    2050.7  2050.2  2049.3  2049.7    6.9    6.7    6.7   6.8
  3200  1/395   8    2053.2  2052.7  2051.0  2051.4    7.4    7.2    7.2   7.4
  6400  1/790   8    2059.9  2058.6  2055.7  2057.0    8.1    8.0    7.8   8.1
 12800  1/1579  8    2069.8  2066.7  2059.6  2062.9   10.1    9.5    9.4  10.1
 25600  1/3158  8    2092.4  2085.8  2071.9  2079.0   20.2   18.9   19.0  20.2
 51200  1/6317  8    2135.1  2123.6  2097.0  2110.9   40.2   38.2   37.8  40.1
102400  1/6317 11    2227.4  2197.7  2149.4  2170.9   81.0   75.6   74.7  80.0

We can see an interesting deviation from how previous Raw data looked (read noise amplified at each higher ISO). The Read Noise is not significantly amplified until ISO 1600 to 3200, and the 3 highest ISO settings are bogus, just binary shifts to compensate for the under-exposure, but useful if you must shoot JPEGs. Raw shooters may as well underexpose (thus retaining highlights) and push in postprocessing.

BTW. A separate test shows that the exposed image areas of the Raw file are amplified, so that a given scene luminance will produce the same DN (with shot-noise accuracy), despite the lower exposure level.

So, the short answer is, it's helpful to have access to the data. All we need now is the highlight clipping DN levels to calculate the Dynamic Range (engineering definition) per ISO, before DxOMARK.com publishes their findings ...

Cheers,
Bart

[BJL]

Bart,
So is this done by adding a bias voltage somewhere along the way, so that a sensel devoid of electons produces a voltage of about 1/8 of that which gets converted to maximum level? Your numbers show that all the way about to about exposure index (aka sensitivity as defined by ISO Standard 12232:2006, but I have a little vendetta against the misnomer "ISO") of 6400, the noise in the ADU output is dominated by noise entering during or after gain is applied, so this bias could be working to maintain noise cancellation options with respect to noise from the gain amplification used to adjust Exposure Index.

By the way, there is nothing bogus about adjusting sensitivity partially with digital adjustment rather than entirely with analog gain, and given the evidence that the gain amplification mit itself be a significant source of noise, it might even be a good idea. Digital adjustment is not in violation of the ISO definitions of sensitivity, based on the ratio between sensor illumination levels and numerical output levels. It is just maybe not a great idea, compared to flagging the raw file for later bit shifting.

[jrsforums]

Thanks, Bart...

BTW....do you have similar numbers for the 5d2....just curious. :-)

John

[32BT]

Well, the attached image may be of additional interest. They are the graphs for the pictures in the original post.

Nikon on the left, Canon on the right.

The Nikon might look horrible, and..., well… it is, but what may be going on here is possibly something like this:

In the Canon, the individual sensel's switching behavior is less stable. This results in more noise over the entire capturing interval, which results in a much finer grained noise in the image overall. This is actually desirable for several reasons especially in high-iso captures. It acts much like dithering… Additionally, I am inclined to believe that the multi-gaussian-humps in Canon and blocked discontinuities in the Nikon also indicate non-linear sensor behavior. In other words, in order for the camera to abide the ISO setting, it has to move the capturing parameters to an increasingly non-linear part of the sensor.

But, all of this is really outside my area of expertise, so I may be totally goofballing here. I do pixels, I don't do electronics.

[32BT]

The attached picture shows I good example of the finer grained noise. The picts show the RAW data.

Note also that the coarser noise in the nikon is additionally emphasized by their a-priori blackpoint clipping.

And don't be fooled by the larger Nikon image, they are both at 1:1 magnification...