DxO Overall Score - Basically No Change During 14 Years? (edited)

[biker]

Canon Overall Score
(or see the attached image)

Does it really mean there's no relevant IQ improvement except the increased resolution during the last 14 years?
Some other brands show a slight score increase but still nothing really impressive.

EDIT: I mean - if you cut a 6 MPix (or what was the top resolution at that time) piece from today's say 24 MPix sensor (and used a shorter focal distance, of course), would you get the same image quality (the same contrast or EV range, the same colour distinction and the same high ISO S/N ratio)? I'm afraid the DxO measurements say so.

[ErikKaffehr]

Hi,

Sensors are pretty good. Canon is lagging behind, as they still use a 500 nm process while the competition has moved on to 180 nm. That moves mean great investments in new technology.

Significant development is still done, but we are not that far from quantum limits, so progress is slow. Lot of hard work for small benefits.

Best regards
Erik

Canon Overall Score
(or see the attached image)

Does it really mean there's no relevant IQ improvement except the increased resolution during the last 14 years?
Some other brands show a slight score increase but still nothing really impressive.

EDIT: I mean - if you cut a 6 MPix (or what was the top resolution at that time) piece from today's say 24 MPix sensor (and used a shorter focal distance, of course), would you get the same image quality (the same contrast or EV range, the same colour distinction and the same high ISO S/N ratio)? I'm afraid the DxO measurements say so.

[Jim Kasson]

Significant development is still done, but we are not that far from quantum limits, so progress is slow. Lot of hard work for small benefits.

I don't see things slowing down. 1100 nm pitch, stacked sensors, CMOS BSI, and who knows what's on the horizon?

http://www.chipworks.com/sites/default/files/Fontaine_0.1_The%20State-of-the-Art%20of%20Mainstream%20CMOS%20Image%20Sensors.pdf

Jim

[ErikKaffehr]

Hi Jim,

Thanks for pointing me to the article. But I am not sure the developments will show up in DxO-mark which is mostly based on noise measurements.

Happy to hear about efforts on reducing cross talk, perhaps some hop for our MFD friends using technical cameras?

Best regards
Erik

I don't see things slowing down. 1100 nm pitch, stacked sensors, CMOS BSI, and who knows what's on the horizon?

http://www.chipworks.com/sites/default/files/Fontaine_0.1_The%20State-of-the-Art%20of%20Mainstream%20CMOS%20Image%20Sensors.pdf

Jim

[dwswager]

Does it really mean there's no relevant IQ improvement except the increased resolution during the last 14 years?
Some other brands show a slight score increase but still nothing really impressive.

EDIT: I mean - if you cut a 6 MPix (or what was the top resolution at that time) piece from today's say 24 MPix sensor (and used a shorter focal distance, of course), would you get the same image quality (the same contrast or EV range, the same colour distinction and the same high ISO S/N ratio)? I'm afraid the DxO measurements say so.

Looking at it from another perspective, great gains have been made, but have been, for the most part, utilized to buy more pixels.  Kodak says film had about 2400ppi...basically 8.6MP.  In addition, there are other parts of the equation not addressed by the DXOMark tests.

[Jim Kasson]

Happy to hear about efforts on reducing cross talk, perhaps some hop for our MFD friends using technical cameras?

The combination of BSI and a 1.5 um optical stack thickness would solve that problem completely, I think.

Jim

[biker]

Hi, thank for all your replies!

But I am not sure the developments will show up in DxO-mark which is mostly based on noise measurements.

If I understand right, the SNR is just one of the three main criteria in the DxO Overall Score. But yeah - for example sensor speed allowing to make fast sequences won't show up here.
I'm still a bit surprised that colour distinction hasn't improved at all. This IMHO shouldn't be so much dependent on the pixel size.

Btw. I would be interested to see low-light results from a 6 MPix full frame sensor featuring those cutting edge technologies (BSI...) if anybody made it.

[dwswager]

Btw. I would be interested to see low-light results from a 6 MPix full frame sensor featuring those cutting edge technologies (BSI...) if anybody made it.

I doubt you would ever see a 6MP sensor at this point, but you do see pixel density limitation.  Nikon will release the new D5 with a 20.8MP full frame sensor.  They could easily have gone with more pixels, but limit it for a number of reasons, not least of which is the direct relationship between pixel size and various aspects of sensor performance, especially SNR.

Cameras like the D810 and 5DS are specialty items which sacrifice performance in some areas do to smaller pixel size to gain more pixels overall.  While life is nothing but tradeoffs, our overall level of performance has increased significantly from the original Nikon D1 2.74MP DSLR!

[Jim Kasson]

I doubt you would ever see a 6MP sensor at this point, but you do see pixel density limitation.  Nikon will release the new D5 with a 20.8MP full frame sensor.  They could easily have gone with more pixels, but limit it for a number of reasons, not least of which is the direct relationship between pixel size and various aspects of sensor performance, especially SNR.

Cameras like the D810 and 5DS are specialty items which sacrifice performance in some areas do to smaller pixel size to gain more pixels overall.  While life is nothing but tradeoffs, our overall level of performance has increased significantly from the original Nikon D1 2.74MP DSLR!

Down sampling in post can provide most, if not all, of the noise reduction obtainable by increasing pixel pitch, and there are many other advantages of low pitches:

http://blog.kasson.com/?p=6078

I believe that the reasons that Nikon kept the resolution of the D5 down include:

20 MP is already overkill for most of the target market.

Frame rate is important to a significant segment of the target market.

Post processing speed is a concern to people working in the field on laptops.

Jim

[dwswager]

Down sampling in post can provide most, if not all, of the noise reduction obtainable by increasing pixel pitch, and there are many other advantages of low pitches:

http://blog.kasson.com/?p=6078

I believe that the reasons that Nikon kept the resolution of the D5 down include:

20 MP is already overkill for most of the target market.

Frame rate is important to a significant segment of the target market

Post processing speed is a concern to people working in the field on laptops.

Jim

I concur there are a whole host of reasons to limit the MP count as you indicated. 

I am unconvinced, as yet, that downsampling can overcome ALL the benefits a larger pixel size can provide.  It is impossible to test in the real world do to the differences in sensors AND the associated electronics packages.  Testing the D810 versus D5 would be interesting, but there are numerous reasons the D5 will cost $6500 versus $2700 for the D810.  Some will be functionality and durability, but some of it will be differences in the hardware and software signal processing. 

I will buy the D500, but I won't be fooling myself into thinking that at $2000 it will have the same level of electronics behavior over various environmental conditions that the $6500 D5 will sport. Being involved in military hardware development, maintaining acceptable performance at  the extremes is where the money get's spent.  Diminishing returns is expensive.

[Jim Kasson]

  It is impossible to test in the real world do to the differences in sensors AND the associated electronics packages. 

Agreed. That's why I wrote the simulator.

Jim

[ErikKaffehr]

Hi,

I would suggest that large pixels have small advantage in DR, compared to downsampling. So if we need a 100000 ISO camera, largish pixels make some sense. On the other hand, I don't really think 100000 ISO makes any sense.

The way I see it, a modern sensor has a base ISO around 100 - 200. If it has very large full well capacity  it may be closer to 100 ISO. So going to high ISOs it just means that we only use a small part of that well capacity, and no PR department can go around that fact.

With low utilisation of full well capacity we are going to get a lot of shot noise, due to photon statistics. If we reduce exposure enough we are going to see some readout noise. That readout noise can be reduced by using large pixels. If we combine four large pixels into one we will get four times the full well capacity., but readout noise would stay the same with all other factors constant.

Combining four pixels in software would still give us the same full well capacity, and therefore also the same shot noise. Readout noise would add in quadrature, AFAIK, so it would be doubled. So there would be a benefit in deep shadows but not much else.

Now, if we get back to DxO, all their data is based on noise, with SMI being the only exception but SMI is not included in the figure of merit.

So, I would say that DxO makes a lot of noise about noise. That said noise is an important parameter.

Another development is decrease of pixel pitch and increase of fill factor. Both are important for correct rendition, but correct rendition seems to be low on the priority list of photographers. Camera makers have a bit other priorities,but making customers happy is a high priority.

Best regards
Erik

Down sampling in post can provide most, if not all, of the noise reduction obtainable by increasing pixel pitch, and there are many other advantages of low pitches:

http://blog.kasson.com/?p=6078

I believe that the reasons that Nikon kept the resolution of the D5 down include:

20 MP is already overkill for most of the target market.

Frame rate is important to a significant segment of the target market.

Post processing speed is a concern to people working in the field on laptops.

Jim

[Guillermo Luijk]

Down sampling in post can provide most, if not all, of the noise reduction obtainable by increasing pixel pitch, and there are many other advantages of low pitches:

http://blog.kasson.com/?p=6078

Very interesting simulation Jim. I was a bit dissapointed when looking at the comparison A7S II (FF 12Mpx) vs A7R II (FF 42Mpx) in DxOMark measurements. I thougt the A7S II had more to offer at high ISOs vs a downsampled ("Print") A7R II, but this was not the case: same SNR at good exposure RAW levels (SNR18%), and no more than 1 stop of extra DR in very high ISOs.





My conclusion is that I would never choose an A7S II for still photography over the A7R II, even if low light applications are very important to me. For the same price the R can do everything the S can, but the opposite is not true.

Regards

[Jim Kasson]

Very interesting simulation Jim. I was a bit dissapointed when looking at the comparison A7S II (FF 12Mpx) vs A7R II (FF 42Mpx) in DxOMark measurements. I thougt the A7S II had more to offer at high ISOs vs a downsampled ("Print") A7R II, but this was not the case: same SNR at good exposure RAW levels (SNR18%), and no more than 1 stop of extra DR in very high ISOs.

My conclusion is that I would never choose an A7S II for still photography over the A7R II, even if low light applications are very important to me. For the same price the R can do everything the S can, but the opposite is not true.

Those cameras are interesting to compare. The a7S does the Aptina conversion gain change at the ISO 1600/2000 transition, so it can increase the conversion gain by a factor of 20. I don't know what the a7SII does, but I would expect it to be at lease as aggressive. The a7RII also changes conversion gain, but does it at the ISO 500/640 transition, which means it can only increase the gain by a factor of 6.4.

Thus, you'd expect the lower resolution camera to be better at ISOs of 2000 and above, about the same at ISOs of 640 and below, and worse between 640 and 2000, resolution corrected and sensor tech being similar.

But we know the sensor tech is different between the two cameras. BSI in the a7RII for one, so it's hard to get too precise about what we'd expect.

Thanks for showing us those curves.

Jim

[Jack Hogan]

Those cameras are interesting to compare. The a7S does the Aptina conversion gain change at the ISO 1600/2000 transition, so it can increase the conversion gain by a factor of 20. I don't know what the a7SII does, but I would expect it to be at lease as aggressive. The a7RII also changes conversion gain, but does it at the ISO 500/640 transition, which means it can only increase the gain by a factor of 6.4.

Thus, you'd expect the lower resolution camera to be better at ISOs of 2000 and above, about the same at ISOs of 640 and below, and worse between 640 and 2000, resolution corrected and sensor tech being similar.

Very nicely said, Jim.