DxOMark's new resolution magnitude: P-Mpix

[Ray]

Whatever the P-Mpix is, it isn't simply a report on sharpness/res/acutance.

Dale,
As I understand, sharpness and acutance as it relates to perceived detail is always a combination of lens resolution and sensor resolution, assuming adequate lighting and shutter speed.

The P-Mpix ratings from DXO appear to basically combine these two variables, lens resolution and sensor resolution, to provide an overall score for general, perceived sharpness, which I assume would apply at any print size (always comparing equal size prints of course).

I don't often bother using prime lenses because of the inconvenience of a fixed focal length, but I was seduced by the glowing reports of the exceptional quality of the Sigma Art 35/1.4, so I decided to get one.

Now, what I find interesting about the P-Mpix ratings for the Sigma 35/1.4, and my most used Nikkor lens, the 24-120/F4, is that a tripling of pixel count (from the 12mp of the D3 to the 36mp of the D800) more than offsets the lower resolution of the 24-120 zoom, compared with the Sigma prime. The rating for the zoom used with the D800 is 12 P-Mpix, and that for the Sigma Art used with the D3, is slightly less at 11 P-Mpix.

What's also interesting is that the rating for the Sigma 35mm when used with the 16mp D7000 is the same as that for the 12mp D3, ie, 11 P-Mpix.
Since I've hardly used my Sigma 35mm so far, this would seem to be an appropriate time to do a bit of testing in preparation for future decision-making as I take photos on my travels. So I took the following shots from the balcony of my hotel room, using the 24-120 zoom with my D800E and the Sigma Art 35/1.4 on the D7100.

Now, DXO haven't tested either of these lenses on the D7100. I'm making a reasonable assumption that the P-Mpix would jump from 11 to around 13 or 14 as a result of the 50% increase in pixel count. I would therefore expect to see a slight, but perhaps barely noticeable improvement from the Sigma on the D7100, compared with the 24-120 zoom on the D800.
The equivalent focal length on the full-frame format is around 50mm, and to achieve an equivalent DoF as well as FoV I used F5.6 at 50mm, and F4 with the Sigma 35mm.

It so happens that Photozone rate these lenses on the Nikon D3X as being sharpest at F4, in the case of the Sigma, and sharpest at F5.6 in the case of the Nikkor zoom at 50mm. This fact, if it applies to my particular lenses, which might be different due to quality control issues, should make my test more legitimate.

I have to say that I'm very surprised at the results, shown in attached 100% crops. Exactly the same sharpening, clarity and vibrance has been applied to both images in ACR, yet I cannot discern any significant or meaningful difference in resolution and detail in any part of the images.

I would have expected the Sigma to at least be sharper at the edges of the cropped frame, yet it isn't. In fact, the zoom at 50mm appears very slightly sharper in the bottom left corner. Perhaps my copy of the Sigma is below standard, or perhaps my copy of the Nikkor 24-120/F4 is above standard. Looks like I'll need to do more testing.

If the color or contrast of these test images doesn't look quite right, it's because the images have been processed on an uncalibrated laptop.

Any explanations as to what's going on here would be welcome.

[Dale_Cotton2]

Ray: might be better to take this off-line then report back if anything we come up with relates to the P-Mpix theme of this thread. You can reach me via the e-mail link on my website.

[Hans Kruse]

Dale,
As I understand, sharpness and acutance as it relates to perceived detail is always a combination of lens resolution and sensor resolution, assuming adequate lighting and shutter speed.

The P-Mpix ratings from DXO appear to basically combine these two variables, lens resolution and sensor resolution, to provide an overall score for general, perceived sharpness, which I assume would apply at any print size (always comparing equal size prints of course).

I don't often bother using prime lenses because of the inconvenience of a fixed focal length, but I was seduced by the glowing reports of the exceptional quality of the Sigma Art 35/1.4, so I decided to get one.

Now, what I find interesting about the P-Mpix ratings for the Sigma 35/1.4, and my most used Nikkor lens, the 24-120/F4, is that a tripling of pixel count (from the 12mp of the D3 to the 36mp of the D800) more than offsets the lower resolution of the 24-120 zoom, compared with the Sigma prime. The rating for the zoom used with the D800 is 12 P-Mpix, and that for the Sigma Art used with the D3, is slightly less at 11 P-Mpix.

What's also interesting is that the rating for the Sigma 35mm when used with the 16mp D7000 is the same as that for the 12mp D3, ie, 11 P-Mpix.
Since I've hardly used my Sigma 35mm so far, this would seem to be an appropriate time to do a bit of testing in preparation for future decision-making as I take photos on my travels. So I took the following shots from the balcony of my hotel room, using the 24-120 zoom with my D800E and the Sigma Art 35/1.4 on the D7100.

Now, DXO haven't tested either of these lenses on the D7100. I'm making a reasonable assumption that the P-Mpix would jump from 11 to around 13 or 14 as a result of the 50% increase in pixel count. I would therefore expect to see a slight, but perhaps barely noticeable improvement from the Sigma on the D7100, compared with the 24-120 zoom on the D800.
The equivalent focal length on the full-frame format is around 50mm, and to achieve an equivalent DoF as well as FoV I used F5.6 at 50mm, and F4 with the Sigma 35mm.

It so happens that Photozone rate these lenses on the Nikon D3X as being sharpest at F4, in the case of the Sigma, and sharpest at F5.6 in the case of the Nikkor zoom at 50mm. This fact, if it applies to my particular lenses, which might be different due to quality control issues, should make my test more legitimate.

I have to say that I'm very surprised at the results, shown in attached 100% crops. Exactly the same sharpening, clarity and vibrance has been applied to both images in ACR, yet I cannot discern any significant or meaningful difference in resolution and detail in any part of the images.

I would have expected the Sigma to at least be sharper at the edges of the cropped frame, yet it isn't. In fact, the zoom at 50mm appears very slightly sharper in the bottom left corner. Perhaps my copy of the Sigma is below standard, or perhaps my copy of the Nikkor 24-120/F4 is above standard. Looks like I'll need to do more testing.

If the color or contrast of these test images doesn't look quite right, it's because the images have been processed on an uncalibrated laptop.

Any explanations as to what's going on here would be welcome.

I looked at your examples and I see a clear difference in the crops. The D7100 with the Sigma 35 is not quite sharp in corners compared to the D800 with the 24-120. 

Now the D7100 has been tested with the Sigma 35 f/1.4 and the MPix rating is 16Mpix compared to the 12Mpix of the 24-120 on the D800. And field map attached.

[BJL]

DxOMark has defined a composite magnitude called Perceptual Mpix (or P-Mpix) to represent the output resolution/sharpness one can expect from a given combo: lens + sensor. ...

I wonder at what aperture is this magnitude defined. The one that provides max resolution for each lens?

Some details from http://www.dxomark.com/About/Lens-scores/Metric-Scores

The resolution score is computed as follows:
For each focal length and each f-number, we first compute sharpness and then weight it throughout the field, tolerating less sharpness in the corners than in the center. This gives one number for each focal and aperture combination.
Then, for each focal length, we select the maximal value of sharpness over the range of available apertures. We average this value over the whole range of focal length to obtain the DxOMark resolution score that we report (in P-MPix).

So yes, for each focal length, the results for best aperture are used … but there is averaging over the field (center to edge) and for zooms, averaging over focal lengths.

Ironically, DPReview published more detailed data derived from DXOMark testing, or at least makes it easier to find than at the DXO site, allowing one to examine values at any tested aperture ratio. For example, here is the new Zeiss 55/1.4: http://www.dpreview.com/reviews/lens-widget-fullscreen?compare=false&lensId=zeiss_otus_55_1p4&cameraId=nikon_d7100&version=0&fl=55&av=1.4&view=mtf-ca

[NancyP]

The one thing DXO doesn't have is a p value! How many lenses are tested for lens scores? How many bodies are tested for sensor scores? The singular of data is "anecdote".

[Hans Kruse]

I looked at your examples and I see a clear difference in the crops. The D7100 with the Sigma 35 is not quite sharp in corners compared to the D800 with the 24-120. 

Now the D7100 has been tested with the Sigma 35 f/1.4 and the MPix rating is 16Mpix compared to the 12Mpix of the 24-120 on the D800. And field map attached.

Shot Nikon (and Sony) decide to make a full frame sensor with the same pixel pitch as the D7100 it would be 54MP and if scaled up the Sigma resolution would be 36MP. Maybe not quite but it indicates that with such a sensor a 35mm camera could finally rival the lower rez MF cameras as the Leica S and Pentax 645D.

[Ray]

Shot Nikon (and Sony) decide to make a full frame sensor with the same pixel pitch as the D7100 it would be 54MP and if scaled up the Sigma resolution would be 36MP. Maybe not quite but it indicates that with such a sensor a 35mm camera could finally rival the lower rez MF cameras as the Leica S and Pentax 645D.

If Nikon were to offer a 54 mp upgrade to the D800, I certainly hope they would address the banding problems that are the main disappointment with the D7100.

I was very pleased with the D7000's extremely wide DR and ISO-less nature. When shooting at shutter speeds and apertures that require an increase in the ISO setting for a good ETTR, one could simply underexpose at base ISO. After processing the underexposed image in Photoshop, the result would be virtually as good as the same shot taken at a higher ISO, but with the advantage of no risk of blown highlights.

The sensor in the D7100 appears to be from a manufacturer other than Sony. I've see reports that Toshiba are the manufacturer.
Now, according to DXOMark, the D7100 has similar noise and DR characteristics to the D7000, except at its base ISO of 100. The DR difference between ISO 100 and 200 on the D7000, is virtually a full stop, or to be precise, 0.96 EV. However, on the D7100 that DR difference between ISO 100 and 200 is only 1/3rd of a stop.

For this reason, I would prefer to underexpose, say 3 stops at ISO 200 than 4 stops at ISO 100, when using the D7100. To put it another way, if one underexposes 3 stops at ISO 200, instead of using ISO 1600 for an ETTR, one loses only 1/3rd of a stop of DR, which is not significant in my books. The freedom from the risk and concerns about blown highlights, and the wasted time in trying to get it right, makes that trade-off worth it, in my view.

However, if one underexposes 4 stops at ISO 100, one gets a full stop of worse DR compared with ISO 1600. A full stop, or a 1 EV difference, can be very significant if one wants detail in those shadows. So for this reason I use ISO 200 as a base ISO when shooting at relatively fast shutter speeds that would require a higher ISO for an ETTR exposure.

Alas! As I process the D7100 images I took recently on a photographic trip to Nepal, I'm finding lots of banding in the deep shadows, which I never noticed in any of my D7000 images. If the subject is static, no problem. One can bracket exposure for HDR purposes. However, if one is trying to 'capture the moment', one gets only one shot.

The following image was underexposed by about 3 stops at ISO 200. I can't help wondering if that very noticeable banding in the shadows would have been significantly less if I'd used ISO 1600. I guess I'll have to do some testing to find out.

[Hans Kruse]

It seems to me that you are exposing to the left  I would argue that would get much better results on casual shooting where you need to catch the moment in rapidly changing light by using auto ISO on manual and set the aperture and shutter speed you need and exposure compensation to -1 on matrix metering. Using Lightroom or ACR with the automatic highlight recovery I would expect that only would get blown out highlights seldomly and you could lift the exposure in pp without the banding you are seeing now. Of course keep en eye on the ISO chosen by the camera from time to time and compensate by shutter speed needed and aperture needed.

[BJL]

... I would argue that would get much better results on casual shooting ... by using auto ISO ... set[ting] the aperture and shutter speed you need and ... using ... exposure compensation to -1 on matrix metering.

Agreed! With sensors that suffer very little from read noise, and so approach the ideal of being "ISO-less" we can probably avoid a lot of agonizing by just choosing the aperture and shutter speed we want (unless this blows highlights in the photosites themselves) and letting the camera count the resulting photons, with the only care needed being not to amplify the signals from some photosites into clipping by the analog amplifier of ADC. For the latter, a modest "under-amplification" like your suggested -1 will usually be enough to avoid clipping problems. JPEG/print/screen display levels can then be handled later; often with simple auto-levels in PP, at least as a starting point.

Of course Ray is doing this "highlight protection", but his example takes it to the extreme of three stops of protection, and in running as dar away as possible from one problem, he runs into others, like banding.

By the way, increasing ISO speed setting without increasing exposure time or aperture size is not "exposing to the right", because the sensor gets no more exposure; I call it "amplify to the right", or ATTR: pushing the numerical  raw level histogram to the right.  With good modern almost ISO-less sensors that do column-parallel ADC, this ATTR does not seem very useful.


There are other options for highlight protection, such as:
- the Hasselblad/Kodak sensor approach of just counting photons with a good enough ADC that no analog amplification is needed, with all display levels adjustments handled later, in the digital domain,
and
- the approach of some cameras from Olympus and others of systematically using less analog gain and so placing mid-tones a stop or so lower relative to the maximum raw level, and then balancing this with more digital gain in default raw conversion. This is sort of like "-1 comp in exposure , +1 push in digital processing".

[Ray]

It seems to me that you are exposing to the left  I would argue that would get much better results on casual shooting where you need to catch the moment in rapidly changing light by using auto ISO on manual and set the aperture and shutter speed you need and exposure compensation to -1 on matrix metering.

You certainly would with a Canon camera, Hans. However, if the camera is truly ISO-less, or as close as matters, as the D7000 is, then why make matters more complicated than they need be. When one is trying to catch a scene or event which is not only unexpected and rapidly changing, but which also appears in continually varying  lighting  conditions as one walks along the path, one might not even have time to set the the most appropriate shutter speed.

As you can see from the above images, that shutter speed of 1/200th is not ideal for a moving subject like a mule carrying a cage of squabbling chickens whilst lurching from side to side as it walks. I would have preferred a 400th. In such circumstances one also has to be very careful that the mule does not knock one off the path. 

[Ray]

Of course Ray is doing this "highlight protection", but his example takes it to the extreme of three stops of protection, and in running as dar away as possible from one problem, he runs into others, like banding.

BJL,
As I'm sure you realize, the 3 stops underexposure is not for the purpose of highlight protection. The underexposure is an unavoidable consequence of a particular combination of aperture and shutter speed in the particular lighting conditions. It is the method I've used to achieve this which also happens to give maximum highlight protection as a bonus. The main purpose of this method or approach is to achieve maximum image quality on the spur of the moment without wasting time making more than the simplest of camera adjustments.

Of course, you are right that I've inadvertently run into the problem of banding as a result of this approach, hence the purpose of my post, to bring this to the attention of others. But to be logical again, such banding is a property of the Toshiba sensor in the D7100. There was no such banding in the D7000 images. It seems that Nikon, with the D7100, have taken one step forward in relation to sensor resolution, and one step backward in relation to deep-shadow noise. As I've mentioned, I sure hope that Nikon fix this problem if they ever offer  us  a 54 mp full-frame.

The question that now needs to be resolved is whether or not such banding issues remain when one exposes to the right of the histogram, as opposed the the left of the histogram, using the same exposure on the D7100. My only other experience of annoying banding was with the Canon 5D many years ago, but this was the sort of camera that produced a very significant improvement when exposing to the right of the histogram at any ISO above base, compared with underexposing at base using the same exposure.

By the way, increasing ISO speed setting without increasing exposure time or aperture size is not "exposing to the right", because the sensor gets no more exposure; I call it "amplify to the right", or ATTR: pushing the numerical  raw level histogram to the right.  With good modern almost ISO-less sensors that do column-parallel ADC, this ATTR does not seem very useful.

I think we're into semantics here. ETTR means exposing to the right of the histogram. The only histogram that's available to view on a DSLR is the one that is displayed in relation to the ISO setting one has made, whatever that setting may be. As Hans has mentioned, with a wink, I've exposed to the left in my example above. If I'd used ISO 1600 instead ISO 200, the exposure would have been to the right of the histogram. As you should note, I'm a fairly logical sort of person.  

[Fine_Art]

Ray,

Topaz denoise has a decent de-banding filter, in case you have not seen it already.

[Ray]

Ray,

Topaz denoise has a decent de-banding filter, in case you have not seen it already.

Thanks! I'll check that out. I recall using Topaz many years ago, but now that both ACR and Photoshop have noise-reduction filters, I rely upon them almost exclusively. One gets used to the convenience of familiarity. I'm a bit reluctant to go to the expense and trouble of using new software if it provides only a very marginal improvement, or if it's likely I would use it very rarely.

I'd much prefer it if Nikon would get its act together. 

[Fine_Art]

I thought I would give that banding a try with IP. It is more difficult to use than the commercial, one click NR packages. The detail kept can be good if you are ok with a grain look left in.

[BJL]

Ray, can you expain what lead to this "histogram three stops too far to the left" problem?

I agree that one cannot alway use manual setting of shutter speed and aperture, so I suppose you were in a mode like aperture or shutter priority, but then how did you end up with those very low raw levels? Was it shutter priority plus ISO 200 leading to hitting the maximum aperture available? Anyway, you surely realize that we are not yet dealing with completely "ISO-less" cameras, and with recent Nikons, there is something to be gained by extra amplification up to about ISO speed 400 or 800, so long as highlight clipping is avoided. So wouldn't something like S mode combined with auto ISO limited to about 400 or 800 be safer and still simple enough?

[Fine_Art]

Thanks! I'll check that out. I recall using Topaz many years ago, but now that both ACR and Photoshop have noise-reduction filters, I rely upon them almost exclusively. One gets used to the convenience of familiarity. I'm a bit reluctant to go to the expense and trouble of using new software if it provides only a very marginal improvement, or if it's likely I would use it very rarely.

I'd much prefer it if Nikon would get its act together. 

Understood. It's not as if many of your shots would be lost to that kind of problem to begin with.

Yes, the toshiba chip seemed good at detail at the expense of noise.

[BJL]

I think we're into semantics here. ETTR means exposing to the right of the histogram.

Semantics indeed: you are confusing the meaning and underlying goal of exposing to the right with one method of measuring that exposure, the raw histogram. The original goal, let me remind you yet again, is maximizing S/N ratios by maximizing the light gathered at each photosite, within the constraint of not overfilling any photosite. The relevant histogram to be pushed to the right is of photosite occupancy levels, which is roughly the one seen at base ISO-speed setting.

Taking an exposure situation where you are stuck with substantially underfilled photosites even in the highlights (as in your light-constrained Nepalese examples) and moving the ADC output histogram to the right by using a higher analog gain (higher ISO setting) does nothing of the kind, and does nothing with respect to photon shot noise effects. At best, it can help to overcome the weaknesses of some cameras (particularly Canon DSLRs) with respect to read noise at low gain settings, but is useless in a truly ISO-less sensor, and is useless beyond about ISO 400 with most recent "non-Canon" sensors.

[Hans Kruse]

You certainly would with a Canon camera, Hans. However, if the camera is truly ISO-less, or as close as matters, as the D7000 is, then why make matters more complicated than they need be. When one is trying to catch a scene or event which is not only unexpected and rapidly changing, but which also appears in continually varying  lighting  conditions as one walks along the path, one might not even have time to set the the most appropriate shutter speed.

As you can see from the above images, that shutter speed of 1/200th is not ideal for a moving subject like a mule carrying a cage of squabbling chickens whilst lurching from side to side as it walks. I would have preferred a 400th. In such circumstances one also has to be very careful that the mule does not knock one off the path.  

My recommendation stands nevertheless and I'm sure you will get better results that way, try it Maybe an unfair comment, but your comment on being logical got me thinking that you should have tested your approach on the D7100 before using it on a trip where you can't go and retake the pictures.

Another reason my suggestion is a much better approach is that using that you can even see and review your pictures on your camera whereas heavily underexposed pictures will just be black. So although the ISO-less cameras when truly so does allow for the technique you used, it's actually not very practical IMHO.

[Hans Kruse]

Ray, can you expain what lead to this "histogram three stops too far to the left" problem?

I agree that one cannot alway use manual setting of shutter speed and aperture, so I suppose you were in a mode like aperture or shutter priority, but then how did you end up with those very low raw levels? Was it shutter priority plus ISO 200 leading to hitting the maximum aperture available? Anyway, you surely realize that we are not yet dealing with completely "ISO-less" cameras, and with recent Nikons, there is something to be gained by extra amplification up to about ISO speed 400 or 800, so long as highlight clipping is avoided. So wouldn't something like S mode combined with auto ISO limited to about 400 or 800 be safer and still simple enough?

Are you suggesting that digital amplification is better than analog amplification since you suggest limit the ISO to 400 or 800? (assuming no highlight clipping).

[hjulenissen]

Are you suggesting that digital amplification is better than analog amplification since you suggest limit the ISO to 400 or 800?

I would suggest that if in-camera ISO adjustment is done by a perfect multiplication (followed by quantisation and clipping to 14 bits), or at least appears to work like this analysed as a black-box, then you would (from a purely image-quality-perspective) be better off by doing this operation in your raw converterter, where you can tailor the degree of amplification, preserve the colour of highlights when wilfully clipping, have a floating-point pipeline, do non-linear/signal-dependant amplification etc. The Nikon D7000 seems to be close to this ideal, other cameras seems to be reasonably close above some ISO value.

Of course, this comes at the cost of operating the camera in a way that it probably was not designed for, including overriding light metering and unusable LCD preview/histogram.

(assuming no highlight clipping).

I believe that highlight-clipping headroom is the main reason to limit ISO. If you are certain that there are no highlights, then there may be no strong IQ arguments for it, but I would say that having more headroom is always a good thing.

-h