Another Stupid Camera Test: IQ 260 / A7r / 5d2 / Epic

[ErikKaffehr]

Hi Illiah,

I am somewhat surprised that A7r exif says 14 bit but RawDigger shows 12 bit data (going up to 1:4096) .

If we assume true 14 bit we would have 16384 values, but the significance of the LSBs (least significant bits) at high numbers would be very low. If we assume that FWC corresponds to 16384 the standard of deviation would be 128. It is quite obvious the last few bits only represent noise. On the low end more bits are used but the last two bits (or so) are still fairly stochastic, are they not?

Best regards
Erik

Hi Bernard,

Not sure a camera that records 1775 values per channel (less than 11 bits) can replace a digital back in all cases.

[eronald]

Hi Chris,

Thanks for posting!

Regarding scientific tests, DxO does that better than most of us.

Best regards
Erik

I feel like number 90 on the tennis player list when someone says "Federer plays better than most of us".

Edmund

[Iliah]

Hi Erik,

What Sony are doing is applying a lossy compression curve - you can dump it and inspect if you wish. Data before compression is not truely 14-bit too, it is 13 bit. Here is the histogram of raw data for Sony as it is recorded in the file, before applying the decompression curve.

Looking at the right side you can see that the value range is 0 to 3546 while the number of levels is 1773, meaning we have missing values.
Now if we zoom to the data we can see that each second level is indeed missing


If we look at the result after decompression (linearization) we can see that the gaps are wider now:

Inspecting different histogram region, it becomes obvious that for the shadow portion the width of the gaps is 1 level, and they become progressively wider towards highlights.

When EXIF says 14-bits, it does not mean all 14 bits are in use, only that the resulting data maximum corresponds to 14 bits. It is like a staircase, 1 meter staircase consisting of 7 steps each of 15 cm. It is true that the total height is 1 meter, but it does not mean the levels are each centimeter.

[Iliah]

Hi Bernard,

If D800 is in lossless compression mode, the gaps are nearly non-existent, only white balance pre-conditioning adds them, and those are very few.

 The results with digital backs strongly depend on exposure, which includes the colour of light. For a relatively well-exposed regions the histogram is totally gapless:

[Iliah]

If the histograms are counting EV's from maximum level down, the difference between 14-bit and 16-bit data would report the same numerical level with an EV difference of two stops.

The maximum is normalized to "1". The absolute maximum value is not of importance anymore after normalization. The value of 0.5 is 1 stop below maximum; in absolute terms it means 16116/2= 8008 for Sony is equivalent to 65535/2= 32762 for IQ260.

You may want to look at this. Let's select two relatively close midrange regions on both shots and compute the difference.
Sony:

Average values for the green channel for selected areas are 2269.36 and 1273.61 and the difference between 2 samples is log2(2269.36/1273.61) = 0.83EV
IQ260:

Average values are 4330.84 and 1685.09 and the difference is log2(4330.84/1685.09) = 1.36 EV.
That is pretty significant change in light.
Now, taking into account maximum value for Sony is 16116, and maximum value for IQ260 is 65535 we can compute the difference in absolute terms. Lets' consider the brighter sample, 2269.36 for Sony and 4330.84 for IQ260. log2(16116/2269.36) = 2.83EV from saturation; log2(65535/4330.84) = 3.92 from saturation. That is they are 1 stop apart, and the difference seems to increase towards shadows: 3.66 EV for the darker sample on Sony shot, 5.28EV on the IQ260 shot.
The cause may be flare in metabones adaptor.

[BJL]

The maximum is normalized to "1". The absolute maximum value is not of importance anymore after normalization. The value of 0.5 is 1 stop below maximum; in absolute terms it means 16116/2= 8008 for Sony is equivalent to 65535/2= 32762 for IQ260.

You may want to look at this. Let's select two relatively close midrange regions on both shots and compute the difference.
Average values are 4330.84 and 1685.09 and the difference is log2(4330.84/1685.09) = 1.36 EV.
That is pretty significant change in light.

Thanks, but I still do not fully understand. As far as I now, these numbers are ADC output levels after the different cameras have likely applied different levels of amplification to the signal and so different conversion factors, whether measured in "photons counted per raw level" or the ratio between "photon count as a fraction of full well capacity" and "output level as a fraction of maximum level".

So how do these differently amplified output values tell us anything about the relative amount of light received by the sensors? We seem to need information relating back to photo-electron counts and well capacities in order to compare the amount of light received, either in absolute terms (photons counted) or in relative terms (fraction of full well capacity). Maybe that information is encoded in the ratio of DxO's sensitivity measurement to the ISO speed setting used.

Thinking about lens flare is very interesting though: having seen some measurements of lens flare with real world images which suggested that it is very hard to keep the darkest parts of the recorded image as far as 12 stops below the brightest, I am skeptical about the practical relevance of some extremes of DR measured in unnatural lab situations. For example, a DR test target with very bright areas adjacent to very dark ones might give different results that the usual graduated strip with the brightest and darkest regions at opposite ends.

[Chris Barrett]

Oh my God, my head hurts.  These graphs make my IQ260 look really impressive.  All I really care about in the end, though, is how my photographs look and whether my clients are happy or not.  Over the next couple weeks I'll shoot the Sony along side my Arca and take real world samples all the way through post.  That'll be interesting.

[Iliah]

these numbers are ADC output levels after the different cameras have likely applied levels of amplification to the signal

Yes, but the level of amplification (gain) does not change the linear proportion between input count and linear output number. Say, well is 32,000 electrons, gain is 8, maximum output count is 4,000 (12 bits). We collected half well, 1 EV less than full, 16,000; so it is 2,000 output counts, once again 1 EV less than full. We can just consider the normalized count, dividing by 32,000 at input, or divided by 4,000 at output. Normalized counts at input and at output are identical (less noise introduced by the amplifier and ADC).
The ADC and amplifier are optimized so that "full well" closely corresponds to the maximum output. Photon count is proportional to electron count through quantum efficiency. To the effect, we have a very linear system; and where it is not linear (deep shadows and extreme highlights) we want to clip to avoid poor colour, blotches, other artifacts.

Flare reduces the acceptably linear portion of the characteristic curve to 11 stops with absolute best prime lenses. Adaptors, filters, any tiny amount of dust reduce it further.

[eronald]

Thanks, but I still do not fully understand. As far as I now, these numbers are ADC output levels after the different cameras have likely applied different levels of amplification to the signal and so different conversion factors, whether measured in "photons counted per raw level" or the ratio between "photon count as a fraction of full well capacity" and "output level as a fraction of maximum level".

So how do these differently amplified output values tell us anything about the relative amount of light received by the sensors? We seem to need information relating back to photo-electron counts and well capacities in order to compare the amount of light received, either in absolute terms (photons counted) or in relative terms (fraction of full well capacity). Maybe that information is encoded in the ratio of DxO's sensitivity measurement to the ISO speed setting used.

Thinking about lens flare is very interesting though: having seen some measurements of lens flare with real world images which suggested that it is very hard to keep the darkest parts of the recorded image as far as 12 stops below the brightest, I am skeptical about the practical relevance of some extremes of DR measured in unnatural lab situations. For example, a DR test target with very bright areas adjacent to very dark ones might give different results that the usual graduated strip with the brightest and darkest regions at opposite ends.

12 bits non-flared may be the max, but that's 12 bits per channel, going down from the exposure max which is never set perfectly.

Add a couple of stops to deal with mismatched light balances, and another couple of stops for exposure headroom and you can see that 14 bits lab DR is the least you really want -you cannot expose perfectly so the top is not at the top, and anyway there's a non-linear break near the top- in the case of a digital back you also have digital ISO setting.

I disagree with the fact that we have enough practical DR; in fact we have BARELY ENOUGH for realistic non-lab use. Of course people who work in studios with lightmeters and controlled lighting won't run out of DR as quickly as those in the field who deal with unmatched lighting, ISO set a bit higher and underexposed images who will be in purgatory quicker than they can shout "Beatrice!" .

If you wish we can try and do the numbers exactly.

Edmund  

[BJL]

The ADC and amplifier are optimized so that "full well" closely corresponds to the maximum output.

How do we know that? It is certainly not true at ISO speed settings higher than base-ISO speed in cameras that apply increased analog gain in that case, and so can convert a signal far less that full well charge to maximum ADC level. It probably should be true with those CCD MF backs that apply a fixed analog gain and leave output adjustments for exposure at higher EI to be done in conversion from raw. But with the Sony A7R offering an ISO speed setting lower than the one used in this comparison, and with its base ISO speed apparently less than 100, I would not be so sure in that case.

Flare reduces the acceptably linear portion of the characteristic curve to 11 stops with absolute best prime lenses. Adaptors, filters, any tiny amount of dust reduce it further.

Thanks, that fits with my recollection. So when and how is 13 stops of sensor DR better than 12? This question is to everyone who enjoys discussing DR, not Iliah in particular.

[Iliah]

Hi Edmund,

Problem, plainly, there is no way to accurately compute quality DR from lab DR. With current cameras it looks like -5 EV, more or less; empirically.

[BJL]

Hi Edmund,

Problem, plainly, there is no way to accurately compute quality DR from lab DR. With current cameras it looks like -5 EV, more or less; empirically.

Indeed: that is one reason that I am much more interested in measuring the range from maximum down to a point where the local SNR is something like 10:1 or 5:1 Anything much darker than that is probably only of interest to astronomy and surveillance, because lifting it enough to display as even dark shadows rather than pure black is going to look ugly.

Photon shot noise alone means that 5:1 SNR requires at least a 25 photon count, about three or four stops above the engineering DR floor of a good modern sensor with a dark/read noise level of about one or two electrons.
For example, with a well capacity of 50,000, that 5:1 floor is 11 stops down, and the traditional 10:1 guideline for "barely acceptable" requires a 100 count, so nine stops down from full well.

[Iliah]

How do we know that?

Because it is what designers state, also on the international conferences where the sensors and digital imaging are main topics. Because Chipworks made available some independent studies, though those are expensive. Finally, because this is how the system optimizations are performed, and not only for sensors.

It is certainly not true at ISO speed settings higher than base-ISO speed in cameras that apply increased analog gain in that case, and so can convert a signal far less that full well charge to maximum ADC level.

Yes, I was referring to "base ISO" mostly. However it is worth mentioning that the industry sometimes puts different meanings to familiar terms. Here is what happens with the "well" term. For each gain full well is defined accordingly. The most practically used definition of full well is the amount that can be converted in linear fashion. At "base" ISO it is usually a bit (literally) smaller than the holding capacity of well.

with the Sony A7R offering an ISO speed setting lower than the one used in this comparison, and with its base ISO speed apparently less than 100

You can do direct experimentation to see what actually the lowest "honest" ISO setting on each camera. Just expose a relatively uniformly lit featureless surface strongly out of focus according to spot meter at different ISO settings (including "intermediate", like 130 and 160) and examine raw histogram of a portion close to sensor centre.

[Iliah]

Any SNR on raw data can't be related directly to converted image, because demosaicking and colour conversion add significant noise and artifacts. 10:1 easily ages to 3:1 in the processed image, shadows are damaged much easier than midtones.

[BJL]

Yes, I was referring to "base ISO" mostly.

Agreed about base ISO speed, but that is my point: neither of these cameras is at base ISO speed in these examples, and they are above it by substantially different amounts: the ratio of their base ISO sensitivities seems to be about 3:1 (73:29), or about 1.3 stops. With the IQ180 probably amplified for that base ISO speed of 29, while the A7R is maybe amplified as if for ISO speed 73, if I am interpreting the DxO measurements correctly.


On 10:1 SNR degrading to 3:1 due to demosaicing and such: are you suggesting that the practical lower limit on the ratio of signal to shot noise might be even higher than 10:1 due to those effects, so needing more than a 100 photon count for  pixel to be "photographically useful"? That makes some sense, but I would have to think about the benefits in the case where one then down-samples, improving the "per pixel" SNR, or with print dithering, which has a similar visual effect.


P. S. Chris: sorry if this was not the discussion you were expecting; no good deed goes unpunished in this forum! Thanks again for the samples.

[ErikKaffehr]

Hi,

I have found very few images with more than 9 steps of DR. When I got my Alpha 99 I wanted to see if I could find a difference in DR to my Alpha 900. It took me something like three months, duping a high contrast Velvia slide in a "totally" dark room.

I would say that lens flare and other flare are a limiting factor.

On the other hand, I would suggest that we are quite tolerant on noise in the shadows. The shadows are compressed and hard see. The spectrum of the noise also matters. I had seldom issues with noise on my Sony's but the P45+ seems to be a more bit challenged. On the Sonys I can pull the shadows with confidence, on the P45+ the confidence is not there.

Best regards
Erik

Yes, but the level of amplification (gain) does not change the linear proportion between input count and linear output number. Say, well is 32,000 electrons, gain is 8, maximum output count is 4,000 (12 bits). We collected half well, 1 EV less than full, 16,000; so it is 2,000 output counts, once again 1 EV less than full. We can just consider the normalized count, dividing by 32,000 at input, or divided by 4,000 at output. Normalized counts at input and at output are identical (less noise introduced by the amplifier and ADC).
The ADC and amplifier are optimized so that "full well" closely corresponds to the maximum output. Photon count is proportional to electron count through quantum efficiency. To the effect, we have a very linear system; and where it is not linear (deep shadows and extreme highlights) we want to clip to avoid poor colour, blotches, other artifacts.

Flare reduces the acceptably linear portion of the characteristic curve to 11 stops with absolute best prime lenses. Adaptors, filters, any tiny amount of dust reduce it further.

[eronald]

P. S. Chris: sorry if this was not the discussion you were expecting; no good deed goes unpunished in this forum! Thanks again for the samples.

+1
The geeks in this forum have descended on those files like locusts


Edmund

[AlanG]

Thanks for these test samples. That is very impressive from the A7r. I shoot a lot of interiors using TSE lenses with the 5DIII tethered to C-1 but with DXO Optics for for my conversions. (I think I can control shadow fill light and look better in DXO than with any other program I have tried.)

My clients and I are happy with the Canon and I like my workflow. But I will serious consider going to the A7r for use with my Canon lenses once there is DXO raw conversion support for it. Hopefully C1 tethering will be supported soon too.  Until then, I think it might be too much hassle for me to use it even if it is ca[able of slightly better results.

[favalim]

Comparing pure channels tell us more than pure numbers

[Iliah]

Comparing pure channels tell us more than pure numbers

But you are not looking at pure channels as it seems, what you are looking at seems as processed channels?