Pages: [1] 2   Go Down

Author Topic: DxO Mark interpretation and application  (Read 9738 times)

marcmccalmont

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1780
DxO Mark interpretation and application
« on: April 04, 2012, 06:50:51 am »

It seems there is a lot of confusion on how to interpret (and apply) DxO Mark results both sensor and lens. Perhaps one of the smart guys in this forum could write a quick essay on how to properly interpret these metrics and how to apply them to real world photography and camera/lens purchases. Some people seem to misinterpret a score of 95 as saying the D800 is the best camera in the world and the other extreme is because a DSLR has a higher score than the MFDB the rating is hogwash. I think most of us are somewhere in between these 2 extremes. A clearly written essay would be invaluable for most of us.
Marc
Logged
Marc McCalmont

Tony Jay

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2965
Re: DxO Mark interpretation and application
« Reply #1 on: April 04, 2012, 07:04:03 am »

+1
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DxO Mark interpretation and application
« Reply #2 on: April 04, 2012, 07:21:14 am »

What one needs to know is what each individual measurement means, and then decide for yourself what aspects of performance that is important to you. (Information about what the measurements show can be found on the dxomark web site.) For example if 80 megapixels is of key importance than it does not matter how well the D800 36 megapixels perform.

The dxomark single score rating is not so useful because then dxo has itself chosen how important each aspect is, which may not match you own needs. They don't put any value in resolution for sensors for example, which is a great disadvantage for MF sensors. I'd say that today the MF advantage is 90% about resolution.

For lenses it may for some be important that it performs well at f/2.8, and for others f/8, so again their single number metric does not work.

But the individual measurements are valuable, one of the best available. Unfortunate that they have not tested much MF gear though, would love to see tests of Rodenstock and Schneider lenses... the MF world is much about "test for yourself at your dealer" though, so how MF actually performs as a system in controlled scientific test is not so easy to find out. It is good to test for yourself, but having well-made unbiased expert tests as a guide for which things to test and which products that are promising etc is truly helpful I think.
« Last Edit: April 04, 2012, 07:31:05 am by torger »
Logged

Christoph C. Feldhaim

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2509
  • There is no rule! No - wait ...
Re: DxO Mark interpretation and application
« Reply #3 on: April 04, 2012, 08:41:02 am »

I'm not in the market for a digital camera in the moment, but this may change in the future and I'm watching losely what comes up. For me it is a sort of finding out about the development of cost per technical performance level. But is totally clear for me, if I ever buy a high end digital system, I will rent at my favourite shop first (the give rent money back if you buy) and test for myself. Not to confirm the technical awesomeness I already know from DXO, but to experience the handling of the system and first to actually see the beauty or non beauty of the rendering of the world by the system as a whole.

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Re: DxO Mark interpretation and application
« Reply #4 on: April 04, 2012, 10:10:21 am »

For me, the great value of the DXOMark test results is not their over all scores but the side-by-side comparisons one is able to make amongst 3 different models of cameras, of specific sensor performance characteristics presented on graphs on the same page, at various measured ISO settings which are sometimes quite at variance with the manufacturers' claimed ISOs.

I imagine it might be offputting for some people looking at such graphs if they are not familiar with the units of measurement used with the graphs, so it might help to translate a few of the units, although I'm not necessarily the most qualified person to do this, but I'll try.  ;D

We all know the effects on noise of underexposing. It's why the ETTR concept has  been the subject of so many threads on the forum.  Underexpose just one stop (that is, give one stop less exposure than one needs to, without adverse consequences such as camera or subject movement) and one will usually notice an increase in noise somewhere in the image. Skin tones might not be as smooth as they otherwise would be and, if the image has deep shadows, there may be unacceptable noise there, causing one to blacken the shadows more than one might otherwise do during post processing, or use heavier noise reduction which tends to reduce resolution.

Under the DXOMark Measurements tab, there are a number of headings; ISO Sensitivity, SNR at 18% Grey, Dynamic Range, Tonal Range, and Color Sensitivity.

The unit of measurement used for SNR (Signal-to-noise ratio) is the decibel. As I understand, at the luminosity level of 18% grey, a difference of 3dB respresents a difference of one F\stop in exposure. In other words, if Camera A has 3dB better SNR at 18% grey than Camera B, it means one could underexpose an image from camera A by one full stop and still get skin tones (or other smooth surfaces) with about the same amount of noise as Camera B.

The unit of measurement for Dynamic Range is the Exposure Value, or EV. One EV is equivalent to one F/stop. As I understand, DR refers to noise in the deep shadows and should be distinguished from noise at 18% grey. If Camera A has 2 EV greater DR than Camera B, but only 3dB higher SNR at 18% grey, and one underexposes an image from Camera A by two stops, and compares with the image from Camera B which has been correctly exposed, then the noise in the deep shadows should be about the same in both images, but the noise at 18% grey, generally representative of skin tones, should actually be one stop better in the image from Camera B.

A difference on the graphs of less than 0.5EV in DR is of little consequence and probably not noticeable except at extreme pixel-peeping levels.

Tonal Range refers to the number of distinguishable grey levels the sensor can record, and the units used refer to the number of bits required to encode those levels. I believe a difference of less than 1 bit would not (or hardly) be noticeable.

Likewise, Color Sensitivity is also measured in bits, and the number of bits indicates what is required to encode all the subtle nuances of different shades of color the sensor is capable of capturing. Again, any differences in the test results smaller than 1 bit are of little consequence.

The above comments refer to comparisons of normalised or equal-size images. Whilst the 'print' mode on DXO graphs represents a particularly small size of 8"x12" at 300ppi, to avoid having to uprez images from the cameras with a low pixel count, which introduces other problems, the results for comparison purposes apply to whatever normalized size one may choose which doesn't involve interpolation. Whilst the absolute values may change with changing image size, the relative values won't.

In other words, if one compares the D800 with the Phase IQ180, the relative differences shown on the DXO graphs at 8"x12" and 300ppi would also apply if the D800 image were displayed or printed at 16"x24" and 300ppi, and the IQ180 image downsized to 16"x24" at 300ppi.

Now I can't guarantee that everything I've written here is absolutely correct, but it's as I understand it.


Logged

theguywitha645d

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 970
Re: DxO Mark interpretation and application
« Reply #5 on: April 04, 2012, 10:41:13 am »

The overall score does not mean much. The other data are interesting. I only use it to compare cameras I am interested in purchasing. And only compare sensors with the same format. And like Ray, slight differences in performance are not important. I use this as one data point in my selection. There are other factors that might be more important for me than simply sensor signal output. I like DxO scores, but I am not sure I would buy a camera based solely on the numbers--there is more to photography than that. I think the DxO scores will indicate a poor sensor, not validate a good one--and within a particular format.

But if I were choosing sides, then DxO scores would be either be the be-all and end-all evaluation or a bunch of rubbish depending if my favorite camera was winning or losing. Just imagine the embarrassment if another photographer saw me on the street with an inferior camera, especially if he had a camera with a better score. I would look such a fool.
Logged

marcmccalmont

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1780
Re: DxO Mark interpretation and application
« Reply #6 on: April 05, 2012, 06:18:08 pm »

Ray
Thanks
Marc
Logged
Marc McCalmont

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Re: DxO Mark interpretation and application
« Reply #7 on: April 05, 2012, 10:14:45 pm »

Expanding on what I've written above, I think it might be helpful to explain why the DXOMark over all scores can sometimes appear to be outrageously wrong, causing some people to dismiss the results and give them no credence.

The recent example of the Nikon D800 achieving a higher over all score than the Phase One IQ180 would be a good case to examine.

First we note that the over all score is based upon performance in just 3 categories of camera usage; Portraiture, Landscape, and Sports.

Now these categories may cover most shooting situations, but not all. What about Macro Photography? If this category were included, I think most P&S cameras would get a significantly higher score (in this category) than any MFDB due to the generally extremely close focussing capability of the P&S, sometimes as close as 1cm from the subject, and due to the good DoF at relatively wide apertures which allow the use of a fast shutter speed to freeze movement.

On the other hand, if the over all score were to include a 'Maximum Print Size' category, perhaps alternatively referred to as sensor resolution, then the IQ180 with over twice the number of pixels would undoubtedly get a much higher score than the D800, in this category, which would most certainly push the IQ180 ahead in the over all score.

Similarly, if one were to remove a particular category from the over all score, such as low-light performance for sports, which MFDBs in general are not suited for, then the over all score would likely favour the IQ180.

In the 3 categories shown, that comprise the over all score, the IQ180 has an advantage over the D800 in portraiture as a result of its better redition of subtle shades of color, and I presumes it's lower SNR at 18% grey which is over 1 stop better than that of the D800. However, this 'Portrait' advantage of the IQ180 is offset by the D800's better dynamic range for Landscape purposes, which is 0.8 stops better.

Neither advantage is particularly significant in my opinion, but should represent a noticeable difference. The point is, within the over all score these two factors cancel each other out, leaving the sole reason for the D800's higher over all score being its better low-light performance for sports purposes.

Now I'm sure that most owners of an IQ180 DB did not buy their back for sports coverage, so there should be no need to get upset at the DXOMark results.  ;D

http://www.dxomark.com/index.php/Cameras/Compare-Camera-Sensors/Compare-cameras-side-by-side/(appareil1)/792%7C0/(brand)/Nikon/(appareil2)/746%7C0/(brand2)/Phase%20One
Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: DxO Mark interpretation and application
« Reply #9 on: April 06, 2012, 03:06:25 am »

Hi,

My interpretation of DxO data is that the measurements are mostly about noise related data. So parameters like resolution and OLP filtering are not taken into account. The measurements are probably reliable. The DxO figure of merit mark is a compound of three parameters:

1) DR at minimum ISO  (Landscape)
2) High ISO performance  (Sports)
3) Color depth (Portrait)

So DxO mark leaves lens aside.

If we look at DR, it is defined essentially as log2(Full Well Capacity / readout noise). Full Well Capacity is the number of electrons each pixel can hold. FWC defines the saturation level in the sensor. Readout noise is measured in electron charges (most conveniently).

So what DR actually tells us is how noisy the readout channel is at minimum exposure. The best sensors today have readout noise of a couple of electrons.

The FWC is related to size of the pixels but also on process technology used. It seems that it is pretty constant function of the pixel area.

Most of the noise we see is what is called shot noise, simply a statistical variation of the number of photons captured. Shot noise is presumed to be Poisson distributed, which means (among other things) that noise = sqrt(signal).

It doesn't really matter for shot noise how many pixels are collecting the data if we split a big pixel into four smaller ones they will still collect the same amount of photons (if fill factor is constant). Each small pixel would be twice as noisy, but if we looked at the image at 50% size the four pixels would appear to be binned, so pixel noise would be the same as in the image with the bigger pixels seen at 100% size.

En extension of this is that shot noise is determined by sensor area, or more correctly by number of pixels times FWC (per pixel).

The "tonal range" measurement in DxO mark measures essentially shot noise. The curves are normally quite close when seen in print mode. If a curve is "higher" at base ISO it would really indicate that FWC was pushed in process technology. If the curves are shifted to the right it indicates that photon capture is more efficient. Better "quantum efficiency", better microlenses or a weaker (less orthogonal) color grid array.

On many older cameras DR is essentially flat up till say 800 ISO after which it starts to drop in the normal fashion. That would indicate a very clean sensor signal but noisy readout circuitry.

So, what does this mean in practice?

Having an extended DR essentially means that we can extract more shadow detail in an image that is correctly exposed to the right. This comes into play mostly at base ISO. On MFD or Sony Exmoor type CMOS sensors. We need to keep in mind that readout noise is ugly noise (salt and pepper type impulse noise) while shot noise is smoother noise. So if we want to extract dark detail we would certainly want to keep read noise down.

We need also keep in mind that DxO mark (sensor) does not measure lens MTF. A very good lens on an MF camera will probably transfer much more contrast on a single structure like a straw of grass than a possibly less good lens on a DSLR. The main reason is that we would use a longer lens on an MF camera that would make for a larger image on the sensor.

On most DSLRs the image is also somewhat diffused by an OLP-filter, mainly intended to suppress color moiré. Having more pixels is essentially almost always advantageous. Bigger sensor with longer lens for equivalent field of view will always yield more detail for the plane in focus. Except for diffraction. But, diffraction is benign to sharpening.

Another factor with DR is that in practice is always limited by lens flare. That is the reason that the Arriflex test target DxO uses has just a few small holes. Better lenses with better coatings, fewer glass air surface, blackened edges on lens elements and good internal baffling will give better shadow detail than complex lenses built with less care.

This may be interesting: http://www.arri.com/fileadmin/media/arri.com/downloads/Camera/Camera_Technologies/2011_05-13_Dynamic_Range_Test_Charts_Brochure.pdf

Best regards
Erik




It seems there is a lot of confusion on how to interpret (and apply) DxO Mark results both sensor and lens. Perhaps one of the smart guys in this forum could write a quick essay on how to properly interpret these metrics and how to apply them to real world photography and camera/lens purchases. Some people seem to misinterpret a score of 95 as saying the D800 is the best camera in the world and the other extreme is because a DSLR has a higher score than the MFDB the rating is hogwash. I think most of us are somewhere in between these 2 extremes. A clearly written essay would be invaluable for most of us.
Marc
Logged
Erik Kaffehr
 

marcmccalmont

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1780
Re: DxO Mark interpretation and application
« Reply #10 on: April 06, 2012, 06:23:36 pm »

Erik
Thanks
Marc
Logged
Marc McCalmont

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
DxO Mark interpretation: use of "true ISO" in graphs is dodgy
« Reply #11 on: April 13, 2012, 09:50:40 pm »

I have just one disagreement with Dx0, apart from the consensus view that the individual measurements are far more objective and useful than the aggregate score. This is the use of a sensitivity ("ISO") measurement based on highlight headroom to calibrate comparisons of noise levels and dynamic range. The only valid correction to cameras' stated ISO speed in such comparisons would be to compensate for different cameras giving different shutter speeds under the same lighting, aperture and ISO speed settings, so that the curves line up vertically for equal shutter speeds. What Dx0 instead does is report a lower ISO speed even if a camera gives the same or higher shutter speed, simply because it positions the midtones at a lower output levels, which leaves more highlight headroom.

It would be great if Dx0, or somebody, actually measured sensitivity with the noise-based measures also defined in the ISO12232 standard and called S40 and S10, based respectively on getting SNR of 40:1 or 10:1 for midtones. That would be closer to what a film's ISO speed measures.
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Re: DxO Mark interpretation: use of "true ISO" in graphs is dodgy
« Reply #12 on: April 14, 2012, 03:21:34 am »

I have just one disagreement with Dx0, apart from the consensus view that the individual measurements are far more objective and useful than the aggregate score. This is the use of a sensitivity ("ISO") measurement based on highlight headroom to calibrate comparisons of noise levels and dynamic range. The only valid correction to cameras' stated ISO speed in such comparisons would be to compensate for different cameras giving different shutter speeds under the same lighting, aperture and ISO speed settings, so that the curves line up vertically for equal shutter speeds. What Dx0 instead does is report a lower ISO speed even if a camera gives the same or higher shutter speed, simply because it positions the midtones at a lower output levels, which leaves more highlight headroom.

It would be great if Dx0, or somebody, actually measured sensitivity with the noise-based measures also defined in the ISO12232 standard and called S40 and S10, based respectively on getting SNR of 40:1 or 10:1 for midtones. That would be closer to what a film's ISO speed measures.


Sorry! Can't make head nor tail of this suggestion of yours, BJL. When shutter speed is not an issue to freeze subject movement (or camera shake in the absence of a tripod), maximum image quality is captured with an ETTR exposure in relation to the specific areas of the scene one does not want blown.

Bernard in the following thread at: http://www.luminous-landscape.com/forum/index.php?topic=65415.0  has provided an excellent example at: http://www.flickr.com/photos/bernardlanguillier/6929054304/sizes/o/in/photostream/  of a stitched jpeg from the D800 showing remarkable clarity and detail, and clean shadows, but a totally blown sky.

If the sky is bland and considered to be unimportant within the composition, then it's quite understandable that cleaner shadows take priority.

Exposure in relation to a standard for noise in the midtones might sometimes be in conflict with exposure to gain maximum detail in the highlights. An ISO rating in relation to a sensor's full-well capacity is preferred, in my view. But perhaps I've misunderstood your point.

Logged

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600

When shutter speed is not an issue to freeze subject movement (or camera shake in the absence of a tripod), maximum image quality is captured with an ETTR exposure ...
Ray, that is all fine with respect to base ISO speed, and exposing to as to make most use of the full well capacity of the sensors. Indeed, as far as I can tell, the ISO definition of sensitivity based on placement of midtones relative to maximum signal (which is what Dx0 uses) is intended primarily or exclusively for detemnining the base ISO speed: the minimum before you get into the realm of cramped highlights, such as happens with extended low settings like 50.

I am instead complaining about the opposite case, when shutter speed is an issue to freeze subject movement or such. Then, the only comparisons that make sense are of how good or bad the performance is (for noise, DR, etc.) at the shutter speed needed, like comparisons of two cameras both at 1/1000, then both at 1/2000, and so on. So the calibration should be, in essence, so that the horizontal axis is based on shutter speed.
And indeed, in tests at equal f-stop and equal ISO speed setting with the same subject and same ilumination, some cameras give a lower shutter speed than others, which is "cheating" because they can gather more light but will blur the action more at the same "ISO" setting.

The "ISO" calibration that Dx0 does is nothing to do with the cameras' choices of shutter speeds, but only on where midtones are placed in the JPEGs.

For low light/high shutter speed performace, I would prefer tests using manual mode, with all cameras tested at the same sequence of shutter speeds on the same subject with the same illumination.


P. S. I will leave aside for now whether apertures should be chosen for equal aperture ratio or for equal effective aperture diameter and thus for equal DOF! Ideally, I would like to see both kinds of tests: I like the fact that some of the sample scene comparisions at DPReview are done with apertures chosen to give roughly equal DOF, so as to compare equal compositions.


Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365

Ray, that is all fine with respect to base ISO speed, and exposing to as to make most use of the full well capacity of the sensors. Indeed, as far as I can tell, the ISO definition of sensitivity based on placement of midtones relative to maximum signal (which is what Dx0 uses) is intended primarily or exclusively for detemnining the base ISO speed: the minimum before you get into the realm of cramped highlights, such as happens with extended low settings like 50.

I am instead complaining about the opposite case, when shutter speed is an issue to freeze subject movement or such. Then, the only comparisons that make sense are of how good or bad the performance is (for noise, DR, etc.) at the shutter speed needed, like comparisons of two cameras both at 1/1000, then both at 1/2000, and so on. So the calibration should be, in essence, so that the horizontal axis is based on shutter speed.
And indeed, in tests at equal f-stop and equal ISO speed setting with the same subject and same ilumination, some cameras give a lower shutter speed than others, which is "cheating" because they can gather more light but will blur the action more at the same "ISO" setting.

The "ISO" calibration that Dx0 does is nothing to do with the cameras' choices of shutter speeds, but only on where midtones are placed in the JPEGs.

For low light/high shutter speed performace, I would prefer tests using manual mode, with all cameras tested at the same sequence of shutter speeds on the same subject with the same illumination.


P. S. I will leave aside for now whether apertures should be chosen for equal aperture ratio or for equal effective aperture diameter and thus for equal DOF! Ideally, I would like to see both kinds of tests: I like the fact that some of the sample scene comparisions at DPReview are done with apertures chosen to give roughly equal DOF, so as to compare equal compositions.


Sorry! Still can't see the problem. All the dots on the graphs are joined with a continuous line. Whilst it's true that the dots are positioned on the graphs in relation to the actual, measured ISO, and don't line up vertically, it's quite easy to make a reasonably accurate guess as to, for example, the DR at equal shutter speeds.

To take a specific example from the following DXOMark graph. By placing the cursor over the dots, the D800 is shown as having a DR of 12.83EV at the manufacturer's nominated ISO of 400, and the D7000 as having a DR of 11.97EV at the manufacturer's nominated ISO of 400, the difference being 0.86EV.

However, the dots don't line up vertically because they are positioned according to the actual, measured ISOs of 297 and 328 respectively.

Surely this is not a problem. It's easy to see that the DR of the D7000 at a position on the graph which vertically lines up with the D800 reading, would be about 12.2EV instead of 11.97, plus or minus a small error which is really irrelevant, and that the difference in DR at the same shutter speed, at the true ISO of 297, is approximately 0.6EV in favour of the D800.

Does one need to be more precise than that?

http://www.dxomark.com/index.php/Cameras/Compare-Camera-Sensors/Compare-cameras-side-by-side/(appareil1)/792%7C0/(brand)/Nikon/(appareil2)/680%7C0/(brand2)/Nikon/(appareil3)/483%7C0/(brand3)/Canon
Logged

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
Re: DxO Mark interpretation and application
« Reply #15 on: April 14, 2012, 10:53:02 am »

Ray,
    I agree that the main part of the problem can be fixed by careful readers in the way you say: by comparing the values at the dots, ignoring the curves that are potentially shifted horizontally in a poorly justified way. My issue there is just to warn people that the graphs are potentially misleading, especially to the many forum commentators who trust the Dx0 ISO speed measurements and thus also criticise some cameras for "overstating" sensitivity ("ISO speed") when in fact they might simply be allowing a little more highlight headroom, and it is the cameras that go in the other direction (Dx0 measured ISO speed values higher than stated by the camera maker) that are doing something dodgy, by allowing less highlight headroom than the ISO standard requires, and so risking blown highlights. (Blown not because the wells are full, but because the highlight signal is amplified until it hits the ceiling of the amplifier, or of the ADC, or otherwise gets pushed beyond maximum digital output level.)

My other, secondary, issue is a complaint about all low-light performance comparisons I have seen, not specifically of Dx0: some cameras (Canon is often accused, for example) give lower shutter speeds than others at the same "ISO speed" setting. This is hard to explain as an error, and so sounds very much like deliberate gaming of high ISO noise level testing, by giving the sensor a bit more light than the competition. Real world comparisons with moving subjects might be a good way to expose this dodge! If ISO speeds were corrected in that way, there would be a different horizontal movement of the curves.

As I will keep saying, it mystifies me why the very useful ISO standard definitions for noise based sensitivity, S40 and S10, are so rarely measured, published or discussed, despite the almost morbid obsession of so many camera forum participants with noise comparisons.

Some references on ISO12232:2006 noise-based speed measures:
http://en.wikipedia.org/wiki/Film_speed#Noise-based_speed
and pages 18 and 19 of this useful slideshow overview of many aspects of photography related standards and measurement methods from an ISO participant.
http://www.rps-isg.org/DF2008/DigitalPhotographyStandards.pdf
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Re: DxO Mark interpretation and application
« Reply #16 on: April 14, 2012, 10:45:26 pm »

As I will keep saying, it mystifies me why the very useful ISO standard definitions for noise based sensitivity, S40 and S10, are so rarely measured, published or discussed, despite the almost morbid obsession of so many camera forum participants with noise comparisons.

I suspect it's because of the ETTR concept which didn't apply in the days of film. I still can't grasp the purpose of your suggested reorganization along the lines of noise results in relation to specific shutter speeds.

In the case of certain Canon DSLRs, we know that an ETTR exposure at ISO 200, or 400, or 800 etc, has lower noise than the same exposure at ISO 100. I can't see the point in providing noise comparisons at specific shutter speeds that are not related to specific ISO settings.

Furthermore, with certain Canon models, this improvement in noise that can result from the ISO setting, in addition to the choice of exposure, applies only to the one-stop increments of ISO. With some models, an ETTR at ISO 125 produces the same noise characteristics as a shot underexposed by 2/3rds of a stop at ISO 200, and an ETTR at ISO 160 produces the same noise as a shot underexposed by 1/3rd of a stop at ISO 200.

I'm sorry to say, BJL, if DXO were to adopt your suggestion, I think confusion would prevail.

However, if all cameras were like the Nikon D7000, your suggestion might be workable. Unfortunately, not everyone is so privileged as to own a D7000.  ;D

Checking the DXO graphs again, for the D7000, I see that SNR at ISO 100 is 41.1dB. 5 stops down at ISO 3200, SNR is 26.7dB. At 18% grey, a 3dB difference in SNR is equivalent to one stop of exposure. 5 stops' lower exposure should result in SNR at ISO 3200 being 15dB down.

In fact, it's 14.4dB down; a trivial improvement of 0.6dB if one uses ISO 3200. A similar situation applies to DR, except there's no difference at all.

For this reason, the only purpose I see in using a higher-than-base ISO on the D7000, is so I can review the image on the camera's LCD screen, or show it off to someone else.
Logged

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
Elevated ISO speed is for attaining a needed shutter speed
« Reply #17 on: April 14, 2012, 11:48:18 pm »

I suspect it's because of the ETTR concept which didn't apply in the days of film. I still can't grasp the purpose of your suggested reorganization along the lines of noise results in relation to specific shutter speeds.
...
I can't see the point in providing noise comparisons at specific shutter speeds that are not related to specific ISO settings.


1. ETTR is mostly irrelevant to high ISO settings! It is about making the most of the full well capacity, which is not close to happening a the elevated ISO speeds that I am addressing.

2. The main reason that elevated ISO speeds are used is to get high enough shutter speeds, and in that situation, the only noise comparisons that matter are at equal shutter speed. I am not saying that ISO setting is irrelevant, only that I am interested in noise levels at equal shutter speeds. Of course, it is often best to then choose an appropriate ISO speed, like the one that places the mid-tones at a suitable level.

Let me offer a somewhat extreme hypothetical example:
- We wish to photograph a certain scene at 1/1600s to freeze motion, with f/2.8 lenses, wide open.
- According to a light meter, the Exposure Index (ISO speed setting) should be 1600 (E.g. the illumination is about 5 stops dimmer than midday sun.)
- At EI 1600, camera A indeed gives shutter speed 1/1600s, but camera B gives shutter speed 1/800s, and needs EI=3200 to get the needed shutter speed of 1/1600s.
- At EI 1600 on both cameras, the results from camera A are a bit noisier than those from camera B,
  but
- Once we get 1/1600s on camera B (say by raising the ISO speed to 3200, or by underexposing by one stop at EI=1600 and then adjusting levels up in post), the results from camera B are noisier.

If one does the common thing of comparing noise levels at equal ISO speed of 1600, camera B seems better, but when one needs to photograph such a scene at f/2.8 and 1/1600s, camera A has less noise.

That full one stop difference is a bit of an exaggeration to simplify the numbers, but there seem to be examples of different cameras that vary by up to 2/3 stop in the shutter speed given at equal EI setting.
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Re: Elevated ISO speed is for attaining a needed shutter speed
« Reply #18 on: April 15, 2012, 06:40:26 am »


1. ETTR is mostly irrelevant to high ISO settings! It is about making the most of the full well capacity, which is not close to happening a the elevated ISO speeds that I am addressing.


Sorry to disagree again. ETTR is also about making the most of the analogue amplification of the signal in accordance with a particular ISO setting (instruction). In fact, with most Canon cameras, up to ISO 1600 and certainly up to ISO 800, dynamic range falls only marginally each time one halves exposure and increases ISO. It falls only 1/2 a stop between ISO 100 and ISO 800 on the 5D2. Beyond ISO 800 DR begins to fall more rapidly, falling 0.61EV between ISO 800 and ISO 1600. Above ISO 1600 there is little benefit. One might as well underexpose at ISO 1600.

With such cameras, noise readings which are related only to shutter speed could be confusing and misleading. There's a huge difference in DR (or shadow noise) between an ETTR exposure at ISO 800 and the same exposure at ISO 100, on most Canon DSLRs.


Quote
Let me offer a somewhat extreme hypothetical example:
- We wish to photograph a certain scene at 1/1600s to freeze motion, with f/2.8 lenses, wide open.
- According to a light meter, the Exposure Index (ISO speed setting) should be 1600 (E.g. the illumination is about 5 stops dimmer than midday sun.)
- At EI 1600, camera A indeed gives shutter speed 1/1600s, but camera B gives shutter speed 1/800s, and needs EI=3200 to get the needed shutter speed of 1/1600s.
- At EI 1600 on both cameras, the results from camera A are a bit noisier than those from camera B,
but
- Once we get 1/1600s on camera B (say by raising the ISO speed to 3200, or by underexposing by one stop at EI=1600 and then adjusting levels up in post), the results from camera B are noisier.

At ISOs above the manufacturer's nominated 1600, I get the impression there is generally little benefit to increasing ISO, and that seems to be true for both Canon and Nikon cameras. In other words, the same exposure at either ISO 1600 or ISO 3200 will produce images with about the same amount of noise and DR. For this reason it's probably advisable to play it safe and use ISO 1600, to avoid the risk of blowing out highlights at ISO 3200.

However, there are exceptions. The Nikon D3s appears to be the star performer at high ISO, so one might expect there would be an advantage using an ETTR at ISO 3200 instead of the same exposure at ISO 1600.

Referring to DXOMark again, we find this is indeed the case. At half the ISO 1600 exposure, used at ISO 3200, we get a drop in DR of only 0.57EV, ie, half a stop as opposed to a predicted full stop.

DXOMark provides us with the SNR and DR test results in relation to the real or actual or measured ISO ratings. The position of the dots on the graph is in accordance with the actual, tested ISO values, which is as it should be, in my view. The manufacturer's nominated ISO is also mentioned, but this has no bearing on the position of the dots.

I presume if one were to find two cameras with identical ISO ratings, say ISO 297 instead of the manufacturer's ISO 400, then any exposure which was an ETTR with one camera would also be an ETTR with the other camera, assuming identical scenes and lighting conditions, and assuming equal T-stops for the lenses used.

Variations in lens T-stop values is another reason why noise in relation to exposure would not be meaningful. Not only would the noise and DR values at a given exposure change according to the selection of ISO, but would also change according to the T-Stop value at the particular F/stop used.


Logged

Bart_van_der_Wolf

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 8913
Re: Elevated ISO speed is for attaining a needed shutter speed
« Reply #19 on: April 15, 2012, 08:09:38 am »

There's a huge difference in DR (or shadow noise) between an ETTR exposure at ISO 800 and the same exposure at ISO 100, on most Canon DSLRs.

Underexposing by 3 stops at ISO 100 can hardly be called ETTR. You're arguing for argument's sake.

ETTR is all about optimizing (= maximizing without compromising required highlight data) the number of recorded photons. This is most effective at base ISO where the DR (engineering definition) is the largest. Increasing ISO has nothing to do with that, although it may have other useful effects.

Cheers,
Bart
Logged
== If you do what you did, you'll get what you got. ==
Pages: [1] 2   Go Up