Pages: 1 [2] 3   Go Down

Author Topic: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)  (Read 11490 times)

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13985
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #20 on: April 24, 2018, 03:50:35 am »

Everybody will still be able to complain that DXO Mark is not a good test or that it demonstrates that one brand is the best...

Wow... that is great news if I have ever heard some!

Cheers,
Bernard

Rado

  • Full Member
  • ***
  • Offline Offline
  • Posts: 247
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #21 on: April 24, 2018, 06:07:05 am »

DXO Labs is the company that develops their raw converter and also bought Nik plugins last year, right? If they go under it will be a bigger loss for people who like to make photos than losing the testing site.
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10387
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #22 on: April 24, 2018, 08:56:25 am »

The Photons to Photos PDR-vs-ISO chart shows no such advantage. The 1Ds3, 5D2, D3 and D700 all stick close to each other in a bunch, with thr lines crossing each other multiple times.

The D700 was a later and cheaper version of the D3 but with the similar specs. The 5D2 was released a year later than the Nikon D3, but still didn't match the DR of the Nilon D3, at the lower ISOs.

The PDR rating (Photographic Dynamic Range) is based on a personal, subjective, opinion of what is acceptable noise in an image. Personal opinions and taste can vary enormously. The DXOMark graphs are purely objective and simply provide information as to which cameras produces less noise in the deep shadows.

Sometimes noise can be an attraction in its own right, as in the attached image which I took with the 5D in 2007. Some folks might consider it as crap, but I quite like it. The noise creates a tapestry effect which I find interesting. The shot was taken in Ayutthaya, Thailand, from a river boat at night, at ISO 1600.

Quote
And the excitement surrounding the D3 had nothing to do with high ISO - merely the fact that Nikon had finally released a full-frame camera.

That's definitely not true. I see from your profile that you didn't join LL until 2012. The D3 was released towards the end of 2007. At that time there were many posts on LL praising the spectacular high-ISO performance of the D3.

In fact, the founder of this site, Michael Reichmann, was of the opinion that the D3 had significantly lower noise at high ISOs than any other camera available at the time, as were many other authorities.

I recall arguing with Michael about this issue because I had taken the trouble to do my own tests, comparing my Canon 5D with a Nikon D3, in a Nikon shop in Bangkok. This was before DXOMark results were available.

However, I only compared the cameras using the highest ISO of the 5D, which was ISO 3200. The D3 had ISO setting up to 25,600, which fooled a lot of people into assuming that the D3 was supreme at extremely high ISOs.

By bracketing all exposures to reveal the differences in ISO sensitivity, I was able to determine that the D3 had only a 0.5 EV DR advantage at ISO 3200 and beyond. To compare the 5D with the D3 at ISO 6400, 12,800 and 25,600, I underexposed the 5D by 1 stop, 2 stops and 3 stops at ISO 3200, then raised the shadows in Photoshop.

Unfortunately, I didn't have the time, nor feel the need, to compare the cameras at lower ISOs. All the hype was about the really high ISOs because few, and perhaps no cameras, at that time, had ISO settings as high as 12,800 and 25,600.

When the DXOMark results became available, I was pleased to see that my assessment of approximately a 0.5 EV DR advantage of the D3 at ISO 3200 and beyond was confirmed.

However, what I didn't realize at the time was that the real strength of the D3 was not at the really high ISOs, but at the more moderate ISOs of 400 and 800.



Logged

shadowblade

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2839
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #23 on: April 24, 2018, 04:05:00 pm »

The D700 was a later and cheaper version of the D3 but with the similar specs. The 5D2 was released a year later than the Nikon D3, but still didn't match the DR of the Nilon D3, at the lower ISOs.

The PDR rating (Photographic Dynamic Range) is based on a personal, subjective, opinion of what is acceptable noise in an image. Personal opinions and taste can vary enormously. The DXOMark graphs are purely objective and simply provide information as to which cameras produces less noise in the deep shadows.

No, it's not subjective at all. If it were subjective, you wouldn't be able to calculate it from DxO data.

Sure, they've set the benchmark for acceptable SNR at an arbitrary level (I believe it's 20:1). But you need to have a benchmark for any test - DxO itself sets it at 1:1 (which is why DxO numbers consistently read higher than PDR ratings - the benchmark SNR is different). And what the numbers show is the number of stops above the acceptable noise floor at any given ISO - something which can be measured or calculated, not a subjective 'I think this one's better' opinion.

Quote
Sometimes noise can be an attraction in its own right, as in the attached image which I took with the 5D in 2007. Some folks might consider it as crap, but I quite like it. The noise creates a tapestry effect which I find interesting. The shot was taken in Ayutthaya, Thailand, from a river boat at night, at ISO 1600.

I can add noise in post-processing if I want it. Can even make it film-like if I really want - just shoot a piece of film with the lens cap on, develop it, scan it and apply it as a layer in Photoshop. But I can't remove noise without also removing detail, nor easily recover detail from beneath the noise floor.

Quote
That's definitely not true. I see from your profile that you didn't join LL until 2012. The D3 was released towards the end of 2007. At that time there were many posts on LL praising the spectacular high-ISO performance of the D3.

Misleading. I've been reading this since 2004. Just had no reason to actually sign up until 2012, when I wanted to post something.

Quote
In fact, the founder of this site, Michael Reichmann, was of the opinion that the D3 had significantly lower noise at high ISOs than any other camera available at the time, as were many other authorities.

Opinion only.

Show me the numbers demonstrating it.

Quote
However, what I didn't realize at the time was that the real strength of the D3 was not at the really high ISOs, but at the more moderate ISOs of 400 and 800.

http://photonstophotos.net/Charts/PDR.htm#Canon%20EOS%205D%20Mark%20II,Nikon%20D3

Where's this mid-ISO advantage you speak of? The lines are constantly criss-crossing
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10387
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #24 on: April 25, 2018, 08:06:13 am »

No, it's not subjective at all. If it were subjective, you wouldn't be able to calculate it from DxO data.


You seem to be contradicting yourself. You write that PDR is not subjective at all, then follow with the statement "the benchmark SNR is different. And what the numbers show is the number of stops above the acceptable noise floor at any given ISO. "

The concept of an acceptable noise floor is surely subjective. Also, acceptable noise can vary according to the particular circumstances of a shot. Sometimes when taking a shot one might not notice some interesting detail in the deep shadows which is later revealed during processing. If the camera one was using was, say, a Canon 50D instead of a Nikon D7000, at base ISO, then the noise in that particular shadow would definitely be less acceptable than the noise in a D7000 shot.

Quote
I can add noise in post-processing if I want it. Can even make it film-like if I really want - just shoot a piece of film with the lens cap on, develop it, scan it and apply it as a layer in Photoshop. But I can't remove noise without also removing detail, nor easily recover detail from beneath the noise floor.

True. However, if you happen to discover during processing, some rare species of bird lurking in the deep shadows, you'll be able to recover more detail if you had been using the camera that DXO measurements show as having the better DR.

Quote
Misleading. I've been reading this since 2004. Just had no reason to actually sign up until 2012, when I wanted to post something.

Very strange then that you didn't notice the discussions about the high ISO advantages of the Nikon D3. Unfortunately, those threads no longer seem to be available. At least I can't find any threads relating to the Nikon D3 when I do a forum search.

Quote
Opinion only.
Show me the numbers demonstrating it.

Numbers? Like PDR numbers? I do my own tests to provide the evidence. The reason why I accept the relevance and accuracy of the DXOMark DR measurements is because I have always found them to be consistent with my own results whenever I've taken the trouble to compare the DR of my own cameras.

As I've already mentioned, I compared my 5D with the Nikon D3 in a Nikon store in Bangkok before DXO graphs were available on the internet, and determined that at ISO 3200 and higher, the D3 had approximately a 0.5 EV advantage. Because the exposure bracketing was limited to 1/3rd stop increments, I could only be certain the DR advantage was at least 1/3rd of a stop, but was also confident it wasn't as high as 2/3rds of a stop, so estimated it to be approximately 1/2 a stop, which is what DXOMark graphs later showed to be the case.

Quote
   http://photonstophotos.net/Charts/PDR.htm#Canon%20EOS%205D%20Mark%20II,Nikon%20D3
Where's this mid-ISO advantage you speak of? The lines are constantly criss-crossing.

Yes, I don't understand how such a graph could be useful. It's all over the place. I much prefer the DXO graphs.  ;D

However, by introducing the 5D Mk2 you are straying from the point. The D3 had better high-ISO performance than any other DSLR available at that time. The Canon 5Dmk2 was introduced at least a year later, and was certainly an improvement over the original 5D. It equaled the DR performance of the D3 at ISO 1600 and above, and was only marginally below the DR rating of the D3 between base ISO and ISO 800....only about 0.4 EV worse, or perhaps 0.5 EV after adjusting for the different ISO sensitivities, according to the attached image of the DXO graph.

If I hadn't already bought a Nikon D700, before the 5Dmk2 became available, I would probably have upgraded from the 5D to the 5Dmk2. However, my favourite lens at the time was the excellent Nikkor 14-24/F2.8, which I was trying to use on my 5D with an adapter, which was unfortunately not behaving as I'd expected. The low-priced D700 seemed a logical choice, in order to get the full benefits of the Nikkor zoom.

Also, around the time the Canon 5D Mk2 became available, the 24mp Nikon D3X was announced, which we both agree had significantly higher DR at low ISOs, than any of its competitors at the time, so I held off upgrading my Canon equipment, and have still to this day held off upgrading.
Logged

shadowblade

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2839
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #25 on: April 25, 2018, 08:14:48 pm »

You seem to be contradicting yourself. You write that PDR is not subjective at all, then follow with the statement "the benchmark SNR is different. And what the numbers show is the number of stops above the acceptable noise floor at any given ISO. "

The concept of an acceptable noise floor is surely subjective. Also, acceptable noise can vary according to the particular circumstances of a shot. Sometimes when taking a shot one might not notice some interesting detail in the deep shadows which is later revealed during processing. If the camera one was using was, say, a Canon 50D instead of a Nikon D7000, at base ISO, then the noise in that particular shadow would definitely be less acceptable than the noise in a D7000 shot.

Subjectivity is different from having a benchmark. You need some sort of benchmark if you're going to make any measurements at all. It doesn't matter what that benchmark is, merely that it's clearly defined.

The metre is also an arbitrary benchmark. It lets you make meaningful statements like, 'It is 12000 metres from here to the mountain peak.' That doesn't make it subjective - quite the opposite. It tells you exactly how far it is. You could equally say, 'It is 39370 feet from here to the mountain peak.' Again, that's not subjective - assuming you've actually measured or calculated it, it is an objective statement, because the foot is a clearly defined length. But how useless would those same statements be if you hadn't defined the metre or the foot in the first place?

PDR and DxO are both using benchmarks. I believe DxO uses a 1:1 SNR, whereas PDR uses 20:1. It's just like one using metres and the other feet. But they're both defined standards, mathematically interconvertible, and, therefore, not subjective.

Quote
True. However, if you happen to discover during processing, some rare species of bird lurking in the deep shadows, you'll be able to recover more detail if you had been using the camera that DXO measurements show as having the better DR.

Wrong.

You'll be able to recover more detail if you're using a camera with higher DR (at that particular ISO). It doesn't particularly matter who measured it - whether it was DxO or anyone else - or even if it was measured at all, merely that it does have the higher DR. There's nothing special about DxO's DR measurements over anyone else measuring the same thing.

Quote
Very strange then that you didn't notice the discussions about the high ISO advantages of the Nikon D3. Unfortunately, those threads no longer seem to be available. At least I can't find any threads relating to the Nikon D3 when I do a forum search.

And they were ultimately proven incorrect.

The 'high ISO advantages' were all about people looking at the pixel level rather than the whole-image level, forgetting that the 1Ds3/5D2 had 75% more pixels than the D3/D700. Obviously, pixel-for-pixel, the 1Ds3 is going to be noisier. Almost in the same sentence, you'd have people saying that the D3/D700 images looked 'plasticky' in comparison, probably due to having less actual detail.

Quote
Numbers? Like PDR numbers? I do my own tests to provide the evidence. The reason why I accept the relevance and accuracy of the DXOMark DR measurements is because I have always found them to be consistent with my own results whenever I've taken the trouble to compare the DR of my own cameras.

So you're saying that the PDR numbers are wrong? Yes or no?

Because PDR numbers can actually be calculated using DxO data. In fact, the PDR website does it themselves - as well as their own measurements, the PDR website also publishes a list of calculated values derived from the DxO data. These calculated numbers match up very closely with the measured PDR values for the same cameras, with discrepancies well within the margins of sensor variation.

The reason for discrepancies between the PDR values (after taking the different benchmark SNR into account) is that DxO's DR figures don't take into account all sources of noise (and thus all factors impacting SNR), even though they are part of DxO's measured data set. This primarily affects the bottom end of the photon transfer curve, i.e. deep shadows. This applies to both the results calculated from DxO data (which are probably the best ones for comparisons between the two systems, since they are derived from the same data set) and the independently-measured PDR ones.

Quote
As I've already mentioned, I compared my 5D with the Nikon D3 in a Nikon store in Bangkok before DXO graphs were available on the internet, and determined that at ISO 3200 and higher, the D3 had approximately a 0.5 EV advantage. Because the exposure bracketing was limited to 1/3rd stop increments, I could only be certain the DR advantage was at least 1/3rd of a stop, but was also confident it wasn't as high as 2/3rds of a stop, so estimated it to be approximately 1/2 a stop, which is what DXOMark graphs later showed to be the case.

So you're admitting that you never actually did a valid comparison, then.

If you compared it in a store, you're comparing processed JPEG output, not RAW output. That's no basis to make a

Quote
However, by introducing the 5D Mk2 you are straying from the point. The D3 had better high-ISO performance than any other DSLR available at that time. The Canon 5Dmk2 was introduced at least a year later, and was certainly an improvement over the original 5D. It equaled the DR performance of the D3 at ISO 1600 and above, and was only marginally below the DR rating of the D3 between base ISO and ISO 800....only about 0.4 EV worse, or perhaps 0.5 EV after adjusting for the different ISO sensitivities, according to the attached image of the DXO graph.

1Ds3 and 5D2 are essentially equivalent sensor-wise, as are D3 and D700.

Quote
Also, around the time the Canon 5D Mk2 became available, the 24mp Nikon D3X was announced, which we both agree had significantly higher DR at low ISOs, than any of its competitors at the time, so I held off upgrading my Canon equipment, and have still to this day held off upgrading.

Given its price, the D3x was probably more a competitor to the lower-end medium-format cameras than the 5D2/A900, which were a third of the price. It was also equally useless at high ISO. Exmor wouldn't be ready for the prime time for another generation.
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13985
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #26 on: April 26, 2018, 05:52:55 am »

It is pretty well documented that many a pro switched back to Nikon due to the D3’s better image quality and AF.

Not only had it low noise, high DR and very good colors at high iso, its shadows were mostly pattern free, which made the actual DR gap significantly higher than the figures suggest.

Having shot the D3 for a few years I have extended first hand experience to back up this view.

Cheers,
Bernard

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10387
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #27 on: April 26, 2018, 06:50:48 am »

It is pretty well documented that many a pro switched back to Nikon due to the D3’s better image quality and AF.

Not only had it low noise, high DR and very good colors at high iso, its shadows were mostly pattern free, which made the actual DR gap significantly higher than the figures suggest.

Having shot the D3 for a few years I have extended first hand experience to back up this view.

Cheers,
Bernard

Yes. That's also my impression, Bernard. I get a sense that Shadowblade is in some sort of denial about the issue, for some reason.
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10387
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #28 on: April 26, 2018, 07:24:30 am »

Subjectivity is different from having a benchmark. You need some sort of benchmark if you're going to make any measurements at all. It doesn't matter what that benchmark is, merely that it's clearly defined.

Can't disagree with that. A benchmark is a benchmark, and is a necessary standard. The subjectivity is not in the benchmark itself, but in the reasons for selecting the benchmark, which in the case of the PDR system, is described as 'acceptable image quality'.

DXO is not making any subjective claims that at its DR limit of a 1:1 SNR, the image quality will be acceptable.  It's the notion of acceptability that's the subjective aspect. I thought that was clear.

For example, if the DXO graph indicates that the DR of a particular model of camera is 14 stops, at base ISO, it means that some detail can still be discerned if you underexpose a shot by 13 stops. You can confirm that this is true by starting with an ETTR exposure of a test chart containing various sizes of letters and numbers. After underexposing by 13 stops, the image at lowest 14th stop in the range will still contain legible numbers and letters. Only the smaller numbers and letters will be obscured by noise.

Quote
Wrong.
You'll be able to recover more detail if you're using a camera with higher DR (at that particular ISO).

Of course. Where have I stated otherwise? I'm puzzled as to why you think my statement was wrong.

Quote
And they were ultimately proven incorrect.
The 'high ISO advantages' were all about people looking at the pixel level rather than the whole-image level, forgetting that the 1Ds3/5D2 had 75% more pixels than the D3/D700. Obviously, pixel-for-pixel, the 1Ds3 is going to be noisier. Almost in the same sentence, you'd have people saying that the D3/D700 images looked 'plasticky' in comparison, probably due to having less actual detail.

I'm sure you understand that DXOMark provides graphs for noise at both the pixel level, and the full image level. The 'screen' option, top left corner, represents noise at the pixel level, and the 'print' option represents noise for the full image, down-sampled to a standard size.
However, at the time the D3 was released, DXOmark data or graphs were not published. It was only later that their measurements were published.

According to the following news article dated November 2008:
https://www.cnet.com/news/dxo-sheds-light-on-camera-sensor-performance/

"DxO Labs, a French company with deep experience measuring cameras' technical performance, has launched a Web site called DxOMark.com that features detailed information on the performance of the image sensor at the heart of many higher-end digital cameras."

The D3 was available about a year earlier.

No doubt some people failed to make the distinction between pixel noise and over all image noise, but I'm sure Michael Reichmann wasn't confusing these two issues. My disagreement with him on this issue was in regard to the extent of the superior performance of the D3 at high ISOs.

Because the D3 had higher ISO settings than any other camera at the time, up to 25,600, it was assumed by many folks, probably by people who only shot in jpeg mode using auto exposure, that the performance of the D3 must therefore be significantly better than those cameras that didn't have ISO settings higher than 3200.

I was merely trying to correct such misinformation and point out that at ISO 3200 and beyond, the DR advantage of the D3 was no more than 0.5 EV, compared with the older Canon 5D, and not nearly 1 to 2 stops better that many photography sites claimed at the time.
I've seen no reliable evidence that later proves the D3 was not the champion in the noise stakes, at the time it was released of course. Compared with later models, the situation changes.

Quote
It doesn't particularly matter who measured it - whether it was DxO or anyone else - or even if it was measured at all, merely that it does have the higher DR. There's nothing special about DxO's DR measurements over anyone else measuring the same thing.

Really! It doesn't matter who measures it. We're all equally competent. Right? C'mon now!  ;D

What is special about DXO is that they provide information on the full range of camera noise characteristics, whereas the PDR graphs represent only the noise characteristics at 20:1 SNR and above, which are subjectively considered to be acceptable.

The DXO graphs will also provide more reliable information on the ISO-invariant nature of a camera. For example, if the DR is shown as 14 EV at ISO 100, and 13 EV at ISO 200, and 12 EV at ISO 400, then one can reasonably deduce that an image underexposed by 2 stops at ISO 100 will be of equal quality to a one-stop underexposure at ISO 200, and a full exposure at ISO 400, after processing and raising shadows in Photoshop.

Quote
So you're saying that the PDR numbers are wrong? Yes or no?
Not wrong. Just less useful because they are arranged in a more subjective way, and the often higgledy-piggledy nature of the graphs are a bit off-putting and not particularly useful, in my very humble opinion of course.  ;)

Quote
The reason for discrepancies between the PDR values (after taking the different benchmark SNR into account) is that DxO's DR figures don't take into account all sources of noise (and thus all factors impacting SNR), even though they are part of DxO's measured data set. This primarily affects the bottom end of the photon transfer curve, i.e. deep shadows. This applies to both the results calculated from DxO data (which are probably the best ones for comparisons between the two systems, since they are derived from the same data set) and the independently-measured PDR ones.

You have a point there. My main current criticism of DXOMark is that they no longer provide the Full SNR plots across the entire range from the brightest highlights to the deepest shadows. They used to do that. I don't know why they no longer publish those graphs. Perhaps they realized that most practical photographers don't need or want to go into so much technical detail.

Quote
So you're admitting that you never actually did a valid comparison, then.
If you compared it in a store, you'r comparing processed JPEG output, not RAW output. That's no basis to make a

Absolutely not. Goodness gracious me! You think after all the time I've spent on this forum I'd be so dumb as to compare dynamic range characteristics by examining in-camera jpegs? Crikey!  ;D

I put my own SD card in the Nikon D3, took dozens of bracketed shots with each camera on a tripod, in RAW mode, photographing the dark and shady areas inside the store, then processed the RAW images in Photoshop, on my laptop, back in the hotel room in Bangkok.

The reason I was very interested in the claimed high-ISO performance of the D3 at the time, is because I was photographing cultural performances on stage where flash was not allowed because it disturbed the performers. Although the 5D had a reputation for relatively low noise at high ISO, even better than the 1Ds Mk3 at ISO 3200, the noise was still a concern. Refer attached 5D image shot at ISO 3200 without flash, taken in January 2008 which would probably have been the month I visited the Nikon store in Bangkok to do some comparisons.

If the reports of the exceptionally good performance of the D3 at high ISO were true, then that would have been a more ideal camera for my purposes, in those circumstances. There are also other circumstances where exceptionally good high-ISO performance could be of benefit, such as shooting in an Art Gallery where no flash nor tripods are allowed.

Quote
1Ds3 and 5D2 are essentially equivalent sensor-wise, as are D3 and D700.

Not according to the attached DXO graph. The yellow line represents the DR of the 1Ds3. As you can see, the DR of the 5D2 at ISO 1600 is almost a stop better than that of the 1Ds3.
Also, the DR of the Nikon D3, at ISO 800, is almost a stop better than the 1Ds3 at ISO 800.

Don't you think we're beginning to flog a dead horse? Perhaps we should let this topic rest. There should be no shame or embarrassment in admitting that I've completely debunked most of your points.  ;D

Logged

shadowblade

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2839
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #29 on: April 26, 2018, 07:28:29 am »

It is pretty well documented that many a pro switched back to Nikon due to the D3’s better image quality and AF.

Not only had it low noise, high DR and very good colors at high iso, its shadows were mostly pattern free, which made the actual DR gap significantly higher than the figures suggest.

Having shot the D3 for a few years I have extended first hand experience to back up this view.

Cheers,
Bernard

Show me the numbers.

People switch systems all the time, for all sorts of reasons, in both directions (and now in all six possible directions). Multiple anecdotes do not constitute evidence, nor statistics. There are equally anecdotes in the opposite direction - the D3's lack of native ISO below 200, lack of detail/smearing of detail in shots ('plasticky appearance' was a common complaint), etc.

Neither set of anecdotes has any bearing on the market's verdict. As far as I know, Canon had the lion's share of the pro market back then, and still has it now (although it's no longer as dominant as it used to be).
Logged

shadowblade

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2839
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #30 on: April 26, 2018, 08:24:03 am »

Can't disagree with that. A benchmark is a benchmark, and is a necessary standard. The subjectivity is not in the benchmark itself, but in the reasons for selecting the benchmark, which in the case of the PDR system, is described as 'acceptable image quality'.

DXO is not making any subjective claims that at its DR limit of a 1:1 SNR, the image quality will be acceptable.  It's the notion of acceptability that's the subjective aspect. I thought that was clear.

For example, if the DXO graph indicates that the DR of a particular model of camera is 14 stops, at base ISO, it means that some detail can still be discerned if you underexpose a shot by 13 stops. You can confirm that this is true by starting with an ETTR exposure of a test chart containing various sizes of letters and numbers. After underexposing by 13 stops, the image at lowest 14th stop in the range will still contain legible numbers and letters. Only the smaller numbers and letters will be obscured by noise.

Ignore the explanation text and look at the data. Explanation text is always subjective, whether it's a semi-literate rambling on Reddit or a journal article published in Nature or Lancet.

PDR may think that 20:1 makes for acceptable image quality. DxO may think that 1:1 represents this, or it may think nothing at all, and just decided to choose 1:1 as a benchmark without bothering to explain why.

The explanation text merely explains why PDR chose 20:1, instead of 10:1, 6:1, 1:1 or some other number. It does not invalidate it. It falls into the same category as DxO saying that their 'low-light ISO' rating is defined as the ISO at which DR > 9 stops (at 1:1 SNR) and colour depth > 18 bits (their version of 'acceptable quality').

None of that changes the fact that PDR is using an objective SNR of 20:1, while DxO is using 1:1. They've each defined their yardstick and used it to measure the same thing. Therefore, the results are objective, comparable and interconvertible data sets.

Quote
Of course. Where have I stated otherwise? I'm puzzled as to why you think my statement was wrong.

Because of the next part of your own quote:

'you'll be able to recover more detail if you had been using the camera that DXO measurements show as having the better DR.'

What matters is the camera's DR, not what DxO's measurements (or anyone else's measurements) say the DR is. There's nothing magical about DxO's numbers that makes them any better than anyone else who also publishes how they got their results.

Quote
I'm sure you understand that DXOMark provides graphs for noise at both the pixel level, and the full image level. The 'screen' option, top left corner, represents noise at the pixel level, and the 'print' option represents noise for the full image, down-sampled to a standard size.
However, at the time the D3 was released, DXOmark data or graphs were not published. It was only later that their measurements were published.

According to the following news article dated November 2008:
https://www.cnet.com/news/dxo-sheds-light-on-camera-sensor-performance/

"DxO Labs, a French company with deep experience measuring cameras' technical performance, has launched a Web site called DxOMark.com that features detailed information on the performance of the image sensor at the heart of many higher-end digital cameras."

The D3 was available about a year earlier.

No doubt some people failed to make the distinction between pixel noise and over all image noise, but I'm sure Michael Reichmann wasn't confusing these two issues. My disagreement with him on this issue was in regard to the extent of the superior performance of the D3 at high ISOs.

I'm referring to the endless arguments all over the internet (at DPR, photo.net, POTN and various phoography-related websites) where people would put 100% crops side-by-side, then complain about the Canon. This even included the image comparison tools at DPR. I'm sure they would have a field day now with te D850, A7r3 and 5Ds...

Quote
Because the D3 had higher ISO settings than any other camera at the time, up to 25,600, it was assumed by many folks, probably by people who only shot in jpeg mode using auto exposure, that the performance of the D3 must therefore be significantly better than those cameras that didn't have ISO settings higher than 3200.

They still make that same mistake. Most people have a very poor grasp of mathematics, let alone basic knowledge of electronics.

There's a reason I rarely shoot the A7r3 at any ISO other than 100 or 640, no matter how dark it is (unless it's so dark I need to boost it just in order to compose in the viewfinder).

Quote
I was merely trying to correct such misinformation and point out that at ISO 3200 and beyond, the DR advantage of the D3 was no more than 0.5 EV, compared with the older Canon 5D, and not nearly 1 to 2 stops better that many photography sites claimed at the time.
I've seen no reliable evidence that later proves the D3 was not the champion in the noise stakes, at the time it was released of course. Compared with later models, the situation changes.

You won't find any reliable evidence in either direction, since, noise-wise, the D3 and 1Ds3 were essentially equal.

Better than the 1D3? Certainly - that was crippled in comparison by having a sensor only 59% the size of a full-frame one. Better for shooting action than the 1Ds3? If you needed more than 5fps, certainly - the D3 could reach 9fps. But in terms of raw DR, against the 1Ds3? Not at mid-high ISO, and only at low ISO due to the Canon sensor's tartan pattern noise (which doesn't show up on a DR test and was drowned out at high ISO).

Quote
Really! It doesn't matter who measures it. We're all equally competent. Right? C'mon now!  ;D

Nope. Assuming they can take accurate measurements (and it's not exactly difficult to do) and publish their methodology, it doesn't matter whether it's done by DxO or some dodgy website operating out of the middle of Russia.

Quote
What is special about DXO is that they provide information on the full range of camera noise characteristics, whereas the PDR graphs represent only the noise characteristics at 20:1 SNR and above, which are subjectively considered to be acceptable.

That's because PDR uses exactly the same dataset as DxO - it literally uses DxO's own dataset! They also re-measure some of the sensor characteristics, giving a second table, but that is as much a validation/quality-control measure as it is about actually producing new data.

If you think the core function of PDR is producing raw measurements, you're completely missing the point. Rather, it is a re-interpretation of DxO's own datasets (not the pretty charts and figures on DxO's websites, which are DxO's own interpretation of the data, but the actual data points underlying them), taking into account more factors than DxO itself does in its own interpretations of the data. And these corrections are able to explain certain discrepancies between DxO's graphs and actual sensor performance, such as why DxO shows the D7100 and D7200 as having such high base-ISO DR (higher than many full-frame sensors)

Quote
The DXO graphs will also provide more reliable information on the ISO-invariant nature of a camera. For example, if the DR is shown as 14 EV at ISO 100, and 13 EV at ISO 200, and 12 EV at ISO 400, then one can reasonably deduce that an image underexposed by 2 stops at ISO 100 will be of equal quality to a one-stop underexposure at ISO 200, and a full exposure at ISO 400, after processing and raising shadows in Photoshop.
Not wrong. Just less useful because they are arranged in a more subjective way, and the often higgledy-piggledy nature of the graphs are a bit off-putting and not particularly useful, in my very humble opinion of course.  ;)

Neater, certainly. Not more reliable. What shows up on DxO as ISO invariance (since they only take read noise into account) is often revealed, after taking FWC and other factors into account, as close-but-not-quite ISO invariance on PDR.

Neat graphs don't necessarily indicate reliable results. Scientific lab-work is surprisingly messy. A scientist who shows you a results table or chart with neat lines that completely follow a predicted curve probably made up the results.

Quote
I put my own SD card in the Nikon D3, took dozens of bracketed shots with each camera on a tripod, in RAW mode, photographing the dark and shady areas inside the store, then processed the RAW images in Photoshop, on my laptop, back in the hotel room in Bangkok.

The store let you do that?

Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10387
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #31 on: April 26, 2018, 08:45:55 am »

The store let you do that?

Of course they did. They were hoping to get a sale. Perhaps the staff also thought I was a person of some stature and importance.  ;)

I don't wish to continue the discussion about DXOMark versus PDR. If you find the PDR graphs more meaningful and relevant, then that's fine by me. Best of luck!
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13985
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #32 on: April 26, 2018, 02:03:37 pm »

Show me the numbers.

People switch systems all the time, for all sorts of reasons, in both directions (and now in all six possible directions). Multiple anecdotes do not constitute evidence, nor statistics. There are equally anecdotes in the opposite direction - the D3's lack of native ISO below 200, lack of detail/smearing of detail in shots ('plasticky appearance' was a common complaint), etc.

Why on earth do you think I would use time gathering data to support an obvious fact to convince you to spend more time trying to deny the obvious? Why would that be an appealing proposition?

The platics look was certainly not a complaint from people using the camera but some fanboys were most certainly doing their thing on DPreview.

Neither set of anecdotes has any bearing on the market's verdict. As far as I know, Canon had the lion's share of the pro market back then, and still has it now (although it's no longer as dominant as it used to be).

As the recent world press award demonstrated with remarkable clarity...  ;D

Cheers,
Bernard

shadowblade

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2839
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #33 on: April 26, 2018, 05:21:36 pm »

Why on earth do you think I would use time gathering data to support an obvious fact to convince you to spend more time trying to deny the obvious? Why would that be an appealing proposition?

The platics look was certainly not a complaint from people using the camera but some fanboys were most certainly doing their thing on DPreview.

As the recent world press award demonstrated with remarkable clarity...  ;D

Cheers,
Bernard

It's obviously not 'obvious' to anyone but you and a few other vocal Nikon fanboys. Performance data doesn't support it, unless you cherry-pick some tests/analyses (DxO and PDR run slightly different analyses on literally the same test results) while ignoring other, equally-valid ones. Market share data doesn't support it either - as if 60% of the market (and probably 90% of the sports/action shooters, judging by the average sporting event sidelines) are stupid and deliberately buy the worse system year after year after year (Nikon's decade-long low-ISO advantage doesn't come into it for sports - this is about ISO 400 and above).
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13985
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #34 on: April 26, 2018, 06:53:51 pm »

You must the right then.

Cheers,
Bernard

shadowblade

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2839
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #35 on: April 26, 2018, 08:00:34 pm »

You must the right then.

Cheers,
Bernard

Well, you haven't brought up any figures to support your argument, beyond various appeals to unnamed authority and repeatedly saying 'Nikon is just better' in about ten different ways. And you've yet to show anything that refutes the market share evidence, or the PDR figures showing no difference in end-use high-ISO performance. '[Insert prominent person] says it's better' isn't an argument. Even Einstein and Hawking need to substantiate their hypotheses.

Incidentally, I happen to think the D3 actually is better than the 1Ds3 for action photography whenever required resolution or focal length aren't limiting factors. Just that it has nothing to do with high-ISO image quality - rather, it's due to having 9fps vs 5fps for the 1Ds3.
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13985
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #36 on: April 26, 2018, 09:32:22 pm »

Well, you haven't brought up any figures to support your argument, beyond various appeals to unnamed authority and repeatedly saying 'Nikon is just better' in about ten different ways. And you've yet to show anything that refutes the market share evidence, or the PDR figures showing no difference in end-use high-ISO performance. '[Insert prominent person] says it's better' isn't an argument. Even Einstein and Hawking need to substantiate their hypotheses.

Incidentally, I happen to think the D3 actually is better than the 1Ds3 for action photography whenever required resolution or focal length aren't limiting factors. Just that it has nothing to do with high-ISO image quality - rather, it's due to having 9fps vs 5fps for the 1Ds3.

Just to set the record straight, I haven't written that Nikon was better. I just wrote that the Nikon D3 caused a clear market shift at the time because of its superior high iso image quality and AF.

We just got data last week showing that more than 50% of the World press award shooters in 2017 used Nikon cameras, to me that's as good a proof of their current market positioning among top photographers as there'll ever be.

I am still unclear why you think you need to do that, but feel free to disagree and to re-write history all you want.

Moving on to a more productive usage of my time. ;)

Cheers,
Bernard
« Last Edit: April 26, 2018, 09:58:36 pm by BernardLanguillier »
Logged

shadowblade

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2839
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #37 on: April 26, 2018, 10:11:17 pm »

Just to set the record straight, I haven't written that Nikon was better. I just wrote that the Nikon D3 caused a clear market shift at the time because of its superior high iso image quality and AF.

We just got data last week showing that more than 50% of the World press award shooters in 2017 used Nikon cameras, to me that's as good a proof of their current market positioning among top photographers as there'll ever be.

I am still unclear why you think you need to do that, but feel free to disagree and to re-write history all you want.

Moving on to a more productive usage of my time. ;)

Cheers,
Bernard

Given that the D3 was Nikon's first foray into full-frame, of course it caused a market shift. Previously, there had only been one option if you wanted full-frame; now there were two. There were similar shifts when Canon first came out with CMOS, when the 5D2, then the D800 raised the stakes resolution-wise, and when Sony entered the game.

The press award winners represent a very small sample size, and one that varies wildly year to year. For the record, here are the statistics from last year's winners. It swings completely in the opposite direction, and is equally meaningless:

https://petapixel.com/2017/02/16/cameras-captured-winning-shots-world-press-photo-2017/

Show me a chart comparing the cameras used by all the entrants, rather than just the winners, and it would actually be meaningful.
Logged

Stephen Starkman

  • Guest
Re: DXO Labs (kin of DXO Mark) in receivership (like US Chapter 11?)
« Reply #38 on: May 03, 2018, 09:59:28 pm »

....best example of thread drift I've seen in awhile...

And so many (not all) folks are so antagonistic.

Logged

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
DXO Labs (but not DXO Mark!) in receivership
« Reply #39 on: May 03, 2018, 10:42:16 pm »

....best example of thread drift I've seen in awhile...

And so many (not all) folks are so antagonistic.
And sadly, the last two thread I have started have drifted into the same food fight.

Should I reiterate that DXO Mark is now a separate company and so not involved in this receivership, and then close the thread? Se new title!
Logged
Pages: 1 [2] 3   Go Up