Luminous Landscape Forum
Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: Guillermo Luijk on November 17, 2008, 12:35:34 pm
-
A new site by DxO Labs has appeared: DxO Mark (http://www.dxomark.com/), comparing sensors and cameras with a rigurous approach (they claim to focus on pure RAW undemosaiced data).
Some of the statements such as: "Contrary to conventional wisdom, higher resolution actually compensates for noise" (copy & paste: www.dxomark.com/index.php/eng/Insights/More-pixels-offsets-noise!) make me thing it can become a quite interesting place for reference, contrarily to common JPEG sites such as DPreview.
Just the overall DR tests, not per-pixel DR (the Y-scale seems rounded but plots seem correct). All new FF's have similar DR but into that group the A900, with the highest pixel count, won:
(http://img98.imageshack.us/img98/1081/rangodinamicolc8.gif)
BR
-
I found this (http://www.luminous-landscape.com/reviews/accessories/dxomark.shtml) introduction to that site on the web.
-
I found this (http://www.luminous-landscape.com/reviews/accessories/dxomark.shtml) introduction to that site on the web.
oops! I did a search about dxomark.com on the forum but forgot to check the main page. Anyway this thread can be used for discusion.
-
Just the overall DR tests, not per-pixel DR (the Y-scale seems rounded but plots seem correct). All new FF's have similar DR but into that group the A900, with the highest pixel count, won:
(http://img98.imageshack.us/img98/1081/rangodinamicolc8.gif)
Good performance by the A900, although this is probably within measuring errors of the other 3.
By the way, is that pixel level DR or image level DR? I would think that pixel level DR is more important since image level DR will make the larger image look better at a given print size thanks to the smoothing effect of downsizing on shadow noise, right?
Cheers,
Bernard
-
A new site by DxO Labs has appeared: DxO Mark (http://www.dxomark.com/), comparing sensors and cameras with a rigurous approach (they claim to focus on pure RAW undemosaiced data).
Some of the statements such as: Contrary to conventional wisdom, higher resolution actually compensates for noise (http://www.dxomark.com/index.php/eng/Insights/More-pixels-offsets-noise)! make me thing it can become a quite interesting place for reference, contrarily to common JPEG sites such as DPreview.
Just the overall DR tests, not per-pixel DR (the Y-scale seems rounded but plots seem correct). All new FF's have similar DR but into that group the A900, with the highest pixel count, won:
(http://img98.imageshack.us/img98/1081/rangodinamicolc8.gif)
BR
I noticed that this site uses the cameras lowest ISO, which they list as ISO 100 on the A900. ISO 200 is the actual base ISO of the A900, and most tests I've seen give ISO 200 nearly a +1EV stop of DR compared to ISO 100, so the A900 may actually have even better DR than this new test shows. What do you think?
-
I noticed that this site uses the cameras lowest ISO, which they list as ISO 100 on the A900. ISO 200 is the actual base ISO of the A900, and most tests I've seen give ISO 200 nearly a +1EV stop of DR compared to ISO 100, so the A900 may actually have even better DR than this new test shows. What do you think?
Never mind. They tested a range of ISOs. Strange that ISO 100 is higher than 200
-
By the way, is that pixel level DR or image level DR? I would think that pixel level DR is more important since image level DR will make the larger image look better at a given print size thanks to the smoothing effect of downsizing on shadow noise, right?
I think DR measures are at image level, they speak about that here: www.dxomark.com/index.php/eng/Insights/More-pixels-offsets-noise!
(just copy & paste the address including the ! in the end since it seems the LL forum engine does not allow the ! symbol for links).
Otherwhise the 1Ds3 and A900 would surely have worse per-pixel SNR and DR than the Nikons.
-
Some of the statements such as: "Contrary to conventional wisdom, higher resolution actually compensates for noise" .........
I thought this fact had been conventional wisdom for quite some time and that only a few diehards like Gabor (Panopeeper) objected to trading resolution for lower noise.
This new site seems an excellent idea to me. I'm all in favour of more rigorous and more meaningful testing which relates more directly to the final product, which is usually a print, or image on screen, of a particular size. You can't have a sizeless image or print.
-
Iliah Borg over at dpreview doesn't seem to think a lot of the results from dxomark so far...
http://forums.dpreview.com/forums/read.asp...essage=30054057 (http://forums.dpreview.com/forums/read.asp?forum=1021&message=30054057)
-
Iliah Borg over at dpreview doesn't seem to think a lot of the results from dxomark so far...
http://forums.dpreview.com/forums/read.asp...essage=30054057 (http://forums.dpreview.com/forums/read.asp?forum=1021&message=30054057)
As Emil says, it's a brand new site in beta stage and we still don't know deeply what methods they are using.
Also for example if you plot DR of the 40D vs 50D (the so much criticised review in DPreview), for both cameras DR is slightly higher for ISO200 than for ISO100. I can only think of that happening if saturation points at those ISO values vary so much to compensate SNR loss at a given RAW exposure when going from ISO100 to ISO200. Looking at the SNR plots I rather would think they didn't make a very accurate calculation in the low end (deep shadows). Perhaps they applied the same data processing to a big group of samples (they have reviewed many cameras) and the measures for some of them were not enough accurate at that critical point. Or simply they did not have data reaching the low end and some approximation was done.
See here the Canon 50D strange behaviour in the low end:
(http://img440.imageshack.us/img440/3327/canon50dgk9.gif)
And for the case of the Nikon D90 there also seems to be an unexpected behaviour in the 4th curve (ISO 1600) that could lead to a such suspicious DR curve plot:
(http://img374.imageshack.us/img374/7073/nikond90jf8.gif)
So surely there are things to be fine tuned in the results and methods, after all it's a massive beta version of the site, but IMO the beginning seems promising compared to other comparisions using procedures sometimes out of any logic (DPreview).
Let's see if Emil and Iliah hang around here.
-
I'd like to see popular MFDB's included in the test results.
Marc
-
Canon DR usually goes up from ISO 100 to 200 in current models for a combination of two reasons: (1) read noise goes down by almost a factor of two; and (2) the sensor saturates well before the max raw level 16383 at ISO 100. This means that one loses less than a stop at the highlight end in going from ISO 100 to 200, while getting nearly a stop at the shadow end.
It does seem that there are some glitches in the measurements at low signal in their SNR plots. I would think it better to use the measurements to fit the data to the universal noise vs signal formula, taking as many points are required to get a good chi-squared. Then plot the universal formula with the fitted parameters as the curve, with the data points to show the goodness of fit. And use the fit rather than the measured data to determine the DR etc, so that individual data points (especially in shadows) don't skew the results. My guess though is that they're using the measured points directly, or with simple spline-type fits rather than fits to the universal noise formula, and so if they don't have enough data they'll have more substantial errors in the results.
-
Since the D3 and D700 use the same sensor but get different DR evaluations, I see an obvious problem with this site's credibility.
I also wonder why anyone would care if their camera is better or worse at some extraordinarily negligible level of detail. When cameras had clear image quality advantages, then comparisons were useful, but at this point it is not worth the efforts some are putting into trying to determine which camera is "better".
-
Canon DR usually goes up from ISO 100 to 200 in current models for a combination of two reasons: (1) read noise goes down by almost a factor of two; and (2) the sensor saturates well before the max raw level 16383 at ISO 100. This means that one loses less than a stop at the highlight end in going from ISO 100 to 200, while getting nearly a stop at the shadow end.
Emil,
This seems to be very useful information to have. I think that most photographers would tend to assume that best results with respect to tonality, DR and noise will be achieved at base ISO. As a consequence, one might choose a merely adequate shutter speed at ISO 100 for the sake of better DR, in preference to a shutter speed twice as fast at ISO 200. The faster shutter speed is likely to produce at least marginally sharper results. However, if the lower ISO of 100 has no benefit with regard to DR, one might wonder what purpose is served by using it, unless one needs a slower shutter speed for a particular effect.
-
Since the D3 and D700 use the same sensor but get different DR evaluations, I see an obvious problem with this site's credibility.
I also wonder why anyone would care if their camera is better or worse at some extraordinarily negligible level of detail. When cameras had clear image quality advantages, then comparisons were useful, but at this point it is not worth the efforts some are putting into trying to determine which camera is "better".
This DR test over the new FF cameras demonstrates that regarding DR, the differences among them are negligible. And that's a _very_ useful information to focus on other characteristics to make a choice. If you plot the result of the Canon 350D, or the Fuji S5 Pro vs these FF cameras, you will see important differences in DR.
I don't see a problem in the D3 and D700 comparision, their DR results are so close that could well be due to differences in the units under test (just look at the Y-scale). The point is not only which camera got a higher or lower DR in the test, but how much higher or how much lower. Again the conclusion is 'they have the same DR' and that allows to pay attention to other parameters (e.g. price).
-
Canon DR usually goes up from ISO 100 to 200 in current models for a combination of two reasons: (1) read noise goes down by almost a factor of two; and (2) the sensor saturates well before the max raw level 16383 at ISO 100. This means that one loses less than a stop at the highlight end in going from ISO 100 to 200, while getting nearly a stop at the shadow end.
Emil, please correct me if I am wrong: what you mean with this is that:
1. These Canon cameras _almost_ double their S/read noise ratio when going from ISO100 to ISO200, and that is equivalent to having very close curve plots in the low end (never crossing curves however), and that is equivalent to admitting that almost all read noise is produced after the ISO amplification.
2. On the other side, the difference in saturation point (lower at ISO100 than at ISO200) means more extra DR in the right end than what we loose in the left end when going from ISO100 to ISO200.
Being * the saturation points for ISO100 and ISO200 this is what we are talking about:
(http://img505.imageshack.us/img505/8462/isoswp7.gif)
Questions:
1. If the saturation points were the same, we would _never_ get more DR at a higher ISO no matter how much improvement in S/read noise we could get from ISO100 to ISO200. The more we could expect in that case would be to get the same DR. Right?
2. If it is true that for a given camera a higher ISO provides more DR, unless we need extra long exposure times, it would be completely stupid to use the lower ISO for any application (this is what Ray asked).
3. Why is that big difference in the Canon's saturation points? at ISO100 my cheap old 350D saturates at 4095, while the 5D saturates at 3692. I never understood this. It seems as if the output of the ISO amplification gets clipped before entering the ADC, and even that clipping point depends on the ISO set.
BR
-
This DR test over the new FF cameras demonstrates that regarding DR, the differences among them are negligible. And that's a _very_ useful information to focus on other characteristics to make a choice. If you plot the result of the Canon 350D, or the Fuji S5 Pro vs these FF cameras, you will see important differences in DR.
I don't see a problem in the D3 and D700 comparision, their DR results are so close that could well be due to differences in the units under test (just look at the Y-scale). The point is not only which camera got a higher or lower DR in the test, but how much higher or how much lower. Again the conclusion is 'they have the same DR' and that allows to pay attention to other parameters (e.g. price).
Agreed. In fact, I'd say that, other than making the choice between the lower noise of the D3 vs. the resolution of the 1Dsiii/A900, there really isn't much between the FF cameras. It seems that price, handling, camera options and system options are the more relevant reasons to make a purchasing decision.
-
Agreed. In fact, I'd say that, other than making the choice between the lower noise of the D3 vs. the resolution of the 1Dsiii/A900, there really isn't much between the FF cameras. It seems that price, handling, camera options and system options are the more relevant reasons to make a purchasing decision.
And there you have it in a nutshell.
-
Emil, please correct me if I am wrong: what you mean with this is that:
1. These Canon cameras _almost_ double their S/read noise ratio when going from ISO100 to ISO200, and that is equivalent to having very close curve plots in the low end (never crossing curves however), and that is equivalent to admitting that almost all read noise is produced after the ISO amplification.
Yes the curves shouldn't cross; the examples you showed where they do represent flaws in the data.
Being * the saturation points for ISO100 and ISO200 this is what we are talking about:
(http://img505.imageshack.us/img505/8462/isoswp7.gif)
Questions:
1. If the saturation points were the same, we would _never_ get more DR at a higher ISO no matter how much improvement in S/read noise we could get from ISO100 to ISO200. The more we could expect in that case would be to get the same DR. Right?
Correct.
2. If it is true that for a given camera a higher ISO provides more DR, unless we need extra long exposure times, it would be completely stupid to use the lower ISO for any application (this is what Ray asked).
Because, though you lose a bit of DR at ISO 100 on Canons (at the cost of a stop in Tv/Av as well), you gain in S/N throughout the range. Any particular feature in the image will be less noisy.
3. Why is that big difference in the Canon's saturation points? at ISO100 my cheap old 350D saturates at 4095, while the 5D saturates at 3692. I never understood this. It seems as if the output of the ISO amplification gets clipped before entering the ADC, and even that clipping point depends on the ISO set.
It's to do with the native ISO of the sensor relative to the gain applied by the variable gain amplifier. Canon has chosen their lowest gain setting such that the sensor clips before raw saturation in the newer sensors. Since the newer ones are more efficient, if the same amplification is used and the well capacity remains about the same, the photosite will saturate more rapidly (one might say that the "native" ISO of the sensor is higher on the newer camera sensors). Of course these things are never kept fixed from models to model (due in particular to changing pixel pitch), but it is true that the native sensor ISO has increased and so one has the possibility that the sensor saturates earlier for a given ISO setting.
-
It's to do with the native ISO of the sensor relative to the gain applied by the variable gain amplifier. Canon has chosen their lowest gain setting such that the sensor clips before raw saturation in the newer sensors. Since the newer ones are more efficient, if the same amplification is used and the well capacity remains about the same, the photosite will saturate more rapidly (one might say that the "native" ISO of the sensor is higher on the newer camera sensors). Of course these things are never kept fixed from models to model (due in particular to changing pixel pitch), but it is true that the native sensor ISO has increased and so one has the possibility that the sensor saturates earlier for a given ISO setting.
I see, but then if a given sensor saturates at ISO100 at (let's say) ADU=13000, which is less than 1 f-stop far from RAW saturation (ADU=16383), any other ISO should clip at RAW saturation. Does this happen?
For instance the 40D which saturates at 13823 (from my tests) at ISO100, should saturate at 16383 for any other higher ISO value. Does it?
-
I see, but then if a given sensor saturates at ISO100 at (let's say) ADU=13000, which is less than 1 f-stop far from RAW saturation (ADU=16383), any other ISO should clip at RAW saturation. Does this happen?
For instance the 40D which saturates at 13823 (from my tests) at ISO100, should saturate at 16383 for any other higher ISO value. Does it?
Close enough. John Sheehy did some analysis of the 30D a while back and the clipping levels were not exactly 4095, but close enough that it doesn't affect things on the log scale of EV. The difference between 13823 and 16383 however is close to 1/4 stop. I did some 40D tests which gave clipping at 16383 for the green channel at ISO 200 and up, IIRC.
-
Canon DR usually goes up from ISO 100 to 200 in current models for a combination of two reasons: (1) read noise goes down by almost a factor of two; and (2) the sensor saturates well before the max raw level 16383 at ISO 100. This means that one loses less than a stop at the highlight end in going from ISO 100 to 200, while getting nearly a stop at the shadow end
1. The DR of both the Canon 40D and 50D is larger at ISO 100 than at ISO 200.
2. The saturation level plays no role in the DR; the ISO 100 values are scaled down, probably from the base ISO, which appears to be 1/4 EV over 100.
-
Looks like I'll have to carry out some testing, when I have the time, comparing shadow and highlight detail at ISOs 100 and 200. I know that cameras do vary from copy to copy (of the same model), just as lenses do. Years ago, after collecting my new 5D, I wasn't happy with the severe banding in the darkest shadows. I returned it and tried another copy which was better. I wouldn't say much better, but noticeably better.
-
Forgive me for continuing to piss all over this thread, but I find these discussions not to be useful to me and question their purpose.
Iliah Borg over at dpreview doesn't seem to think a lot of the results from dxomark so far...
It's more fundamental than whether or not DxO is "accurately" measuring the characteristics of these sensors. What is the point of dissecting RAW data absent any conversion of that data? If fixating on differences in sensors makes any difference at all to the outcome of your photographs, then equal attention should be paid to RAW converters: http://forums.dpreview.com/forums/read.asp...essage=30062493 (http://forums.dpreview.com/forums/read.asp?forum=1034&message=30062493) You can't simply divorce the data coming from the sensor from the data that comes from conversion of that file; after all, the later is what you will ultimately be working with when you craft your photograph. This leads me to question what some people in these photography forums are attempting to craft: better photographs, or better ways of measuring RAW data. For me, the tools I care about are the ones that deliver files I can use to make better photographs with, and DxO Mark is a diversion from that goal.
-
Tony has a point. We all have to use a RAW converter of one type or another and that choice of RAW converter might be based on lots of factors that are not directly related to getting the last ounce of detail from highlights or shadows. We might use a converter because we like the results, the work-flow, and like the ease of getting those results, or perhaps because we are simply familiar with the interface.
We should not forget that DXO are in the business of creating their own RAW converter. Their tests, which they are now sharing with everyone, are presumably necessary for their own purposes.
-
Forgive me for continuing to piss all over this thread, but I find these discussions not to be useful to me and question their purpose.
It's more fundamental than whether or not DxO is "accurately" measuring the characteristics of these sensors. What is the point of dissecting RAW data absent any conversion of that data?
Because it provides a measure of the camera's capabilities rather than the sum total of the camera's capabilities filtered through a particular raw converter's capability. Usually the scientific method is to strip away those aspects of the subject under study that don't pertain to the subject under study. If you got a lousy result from camera A with raw converter X, you can't know if the issue is camera A or raw converter X. If you study the raw data of camera A without going through conversion, and you find it to be poorer quality than that of camera B, then no raw converter is going to make camera A's output look better than camera B's provided it is doing an equally competent job on each. The latter issue should be the subject of a separate investigation, which to my knowledge is yet to be done in a scientific manner for commercial raw converters.
Better to isolate the variable under study than to lump it in with all sorts of other unknowns.
If fixating on differences in sensors makes any difference at all to the outcome of your photographs, then equal attention should be paid to RAW converters: http://forums.dpreview.com/forums/read.asp...essage=30062493 (http://forums.dpreview.com/forums/read.asp?forum=1034&message=30062493) You can't simply divorce the data coming from the sensor from the data that comes from conversion of that file; after all, the later is what you will ultimately be working with when you craft your photograph. This leads me to question what some people in these photography forums are attempting to craft: better photographs, or better ways of measuring RAW data. For me, the tools I care about are the ones that deliver files I can use to make better photographs with, and DxO Mark is a diversion from that goal.
I agree that equal attention should be paid to raw converters, and that hasn't been done yet. That doesn't mean it can't be done, and it doesn't mean that raw data analysis is without meaning or worth.
-
I find these discussions not to be useful to me and question their purpose
Well, obviously the purpose of pleasing you is failed miserably. I guess you need to ask Michael to close the thread.
What is the point of dissecting RAW data absent any conversion of that data?
It is called abstraction. The information gained this way reflects the characteristics of the camera, independently of the raw converter.
If this is not done, then one could believe, that the camera can do only as good as it appears from certain specific conversions. For example DPReview conducted tests of the 50D with the preliminary version of ACR, which contained some error (IMO it still does). The 40D raw data interpretation was wrong from the outset. I reported the error, which has been turned into another one. Measuring the DR based on that interpretation of the raw data leads to an incorrect result (but DPReview can not measure the DR anyway).
Do you find it useful to evaluate a camera based exclusively on a specific interpretation of the raw data?
-
Well, obviously the purpose of pleasing you is failed miserably. I guess you need to ask Michael to close the thread.
No need to be flippant; I'm just expressing my point of view. No, I would never presume to ask Michael to close any thread in his forums.
If this is not done, then one could believe, that the camera can do only as good as it appears from certain specific conversions.
I'm certainly not advocating that; in fact I'm thinking you and I agree more than you think we do as far as that is concerned. However, a camera's RAW files are only as good as what they can be converted to. While that may require a better RAW conversion algorithm; if none exists then that becomes very problematic to a photographer who doesn't have the knowledge or means to create an ideal RAW converter -- then you move on to another camera -- but I don't think this a really a problem with the recent DSLRs and associated RAW converters we currently have available to us.
Do you find it useful to evaluate a camera based exclusively on a specific interpretation of the raw data?
Yes, specifically with the "interpretation" from the converter I would be using -- which for me would be the one that delivered the best results. Some people complain that superior results from a particular converter are attained because of NR being applied (often they are wrong), but I don't care as what matters to me is that the level of noise and the amount of detail are optimized, and what I find using NX with my D300 is that I have both less noise and more detail than what I get with other converters. Should I dismiss the results I know I can get because some website uses a different (putatively neutral) converter and then claims that some other camera does a better job than mine using that other converter? Should I care if someone claims another camera gets better DR than mine because some "abstract" analysis says so, particularly when I can get what I consider extremely good results with the camera I have?
-
The strength of the DxO Mark method and similar methods is that it allows for accurate characterization of a sensor and/or optical system.
The weakness of the method is that final image quality (e.g., what is seen in a print or on a display, such as in a web gallery) depends on much more than raw-level characterization.
For example, a raw-level characterization may show more promise for the Panasonic DMC-LX3's sensor compared to a Canon PowerShot G10's sensor, but raw files rendered using these makers' respective software (SilkyPix and Digital Photo Professional) may yield opposite results. This may seem initially like a bogus comparison because different software products are being used, but on the other hand it's a common scenario that will apply to many users (i.e., owners of these two cameras will simply use the software that comes with the camera).
This is the classic tradeoff of the scientific method. By isolating specific variables, you can get a very good understanding of that variable within a limited context. The downside is that it's insufficient to explain the big picture or the broader context, without a (much more difficult and time-consuming) examination of the whole context. For example, we have detailed models of the human eye, which can explain certain visual phenomena but is insufficient to model color appearance. As another example, there has been much study of the individual contributions of various nutrients (vitamin B, antioxidants, etc.) to health, which unfortunately are poor predictors in general because they do not take into account the much larger complex system (i.e., how all of these elements interact).
Don't get me wrong. I applaud DxO for their efforts in launching this site and for making the (summarized) data available. It is certainly the right way to go when developing raw conversion software. But if you don't develop raw conversion software, and you're just a good ol' photographer, you have to ask yourself if the numbers/data are going to be relevant to your workflow. Or, put another way, if the doctor tells you to eat fewer eggs because lower cholesterol means a smaller chance of a heart attack, you should ask if that is really the factor that matters.
-
As a Sony user, I can say that the two worst RAW converters I've seen for the A900 files are ACR and....Sony's own IDC! Since nearly every review site uses one of those two converters for RAW development with the Sony, I think it's good to show what the sensor is capable of. Aperture and Capture One are two examples of converters that don't mush the Sony RAWs, and I'm hoping the new Bibble acts similarly.
-
I'm certainly not advocating that; in fact I'm thinking you and I agree more than you think we do as far as that is concerned. However, a camera's RAW files are only as good as what they can be converted to. While that may require a better RAW conversion algorithm; if none exists then that becomes very problematic to a photographer who doesn't have the knowledge or means to create an ideal RAW converter -- then you move on to another camera -- but I don't think this a really a problem with the recent DSLRs and associated RAW converters we currently have available to us.
The point of an analysis without the raw converter would then be to determine the potential for improvement in raw conversion.
This kind of data may not be useful to regular photographers per se, but it could be useful to those making raw converters or otherwise working directly on raw data. It could also be useful to camera sensor makers.
-
This is the classic tradeoff of the scientific method. By isolating specific variables, you can get a very good understanding of that variable within a limited context. The downside is that it's insufficient to explain the big picture or the broader context, without a (much more difficult and time-consuming) examination of the whole context.
I agree that results based solely on measures over RAW data providing an objective view of the hardware of the camera, need some extra interpretation effort to extrapolate results to what we can expect to achieve in our real photographs. However I prefer by far these results than those more obvious DPreview samples. After all we are not always going to use our camera to take JPEG pictures of a stamp with the queen of England.
BR
-
Yes the curves shouldn't cross; the examples you showed where they do represent flaws in the data.
Emil, I was thinking that perhaps they already corrected the saturation point of the camera at each ISO to fit 100% in the X-axis of their plots, contrarily to your plots that strictly display ADU levels on that axis where the right end is always RAW saturation (ADU=2^14-1).
In that case their curves could cross, and in fact this would show in a very intuitive way in the plots what you explained on how some cameras can reach a higher DR for a given higher ISO, but at the cost of worse SNR in most of the range.
I think it could be a good idea that you show the sat point for each ISO (typically just ISO100 should differ from 2^14-1) in your graphs, specially for proper DR measures on them.
BR
-
Emil, I was thinking that perhaps they already corrected the saturation point of the camera at each ISO to fit 100% in the X-axis of their plots, contrarily to your plots that strictly display ADU levels on that axis where the right end is always RAW saturation (ADU=2^14-1).
In that case their curves could cross, and in fact this would show in a very intuitive way in the plots what you explained on how some cameras can reach a higher DR for a given higher ISO, but at the cost of worse SNR in most of the range.
I think it could be a good idea that you show the sat point for each ISO (typically just ISO100 should differ from 2^14-1) in your graphs, specially for proper DR measures on them.
BR
I agree, but I don't know the saturation point for some of the cameras.
-
I agree, but I don't know the saturation point for some of the cameras.
Saturation and clipping are more complex than one might initially think. To test the saturation of my Nikon D3, I grossly overexposed an image of a lightbox at various ISOs with the camera set to record losslessly compressed NEFs. The indicated exposure for ISO 200 was 1/1000 s at f/2.8 and I used 2 s at f/2.8, so I am reasonably certain that all pixels are saturated. I then used Iris to separate the Green 1, Green 2, Red and Blue channels and examined the histograms of the ISO 200 and ISO 3200 images in ImageJ:
[attachment=9786:Histograms.gif]
Clipping can occur when the sensor is saturated or when the ADC overflows. The maximum value for a 14 bit data number is 16383. The green channels at ISO 200 never reach 16383. Due to PRNU (Pixel Response Nonuniformity (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/index.html#prnu)), the pixels saturate at various levels as shown in the histograms. The camera gain is such that the ADC does not overflow. For the red channel at ISO 200, the gain is such that at pixel saturation, the ADC overflows for all pixels as shown in the histogram. For the blue channel, the gain is such that some pixels cause overflow of the ADC but others do not and PRNU is again observed.
I thought that these results were interesting and worth posting.
Bill
-
Saturation and clipping are more complex than one might initially think.
The strangest saturation point I have found is that of the R sensor in the Fuji Super CCD. Looking at the histograms obtained with increasing exposure one could think it never saturates since blown information never concentrates at any particular point but along a wide range of values in the form of a gaussian distribution. See here the EV histogram (each of them normalised to its maximum) with subsequently increasing exposure of +1EV:
(http://img177.imageshack.us/img177/9695/serievp8qv3.jpg)
If we look at the last histogram in linear mode we see all the saturated areas concentrated in a gaussian bell distribution in the right half:
(http://img512.imageshack.us/img512/8274/dscf5074hissc7.gif)
One could think that's information compression (which would mean a huge DR!), but unluckily it is not, it seems to be just noise. Yes, noise in the highlights but noise after all with no texture or detail on it. And the three RGB channels of that bell are aligned in the RAW data (i.e. prior to WB which is the way I calculated these histograms), that is why once WB is applied, R and B relative exposures increase with respect to G and this causes the typical Fuji magenta highlights for having G shortage that becomes visible when applying a strong exposure correction down (what others call highlight recovery). So when a magenta cast appears on a Fuji image it just means one thing: saturation of the R sensor (of course in this situation the S sensor saturated time ago).
These images were obtained from the same RAF file, first with ACR and strong exposure correction down by -4EV, second using a not so strong exposure correction to try to avoid the magenta cast, and last with Zero Noise (DCRAW) using as saturation point the beginning of the gaussian bell, i.e. deliberately clipping to pure white all the 'information' that could be packed into that gaussian distribution:
(http://img235.imageshack.us/img235/3545/acr0cq2.jpg)
The Zero Noise result displays the same information as ACR with the -4EV correction down but without the channel misalignment thanks to the specific saturation point set in DCRAW. Also ACR and DCRAW show differences in the WB applied to the R sensor, when both apply the same WB to the S sensor (i.e. the shadows of any RAF).
This particular behaviour of the R sensor in the Super CCD can easily lead to being optimistic (~0.5 f-stops) in the calculation of the DR for the Fuji cameras if this effect is not properly taken into account.
-
The strangest saturation point I have found is that of the R sensor in the Fuji Super CCD. Looking at the histograms obtained with increasing exposure one could think it never saturates since blown information never concentrates at any particular point but along a wide range of values in the form of a gaussian distribution. See here the EV histogram (each of them normalised to its maximum) with subsequently increasing exposure of +1EV:
This particular behaviour of the R sensor in the Super CCD can easily lead to being optimistic (~0.5 f-stops) in the calculation of the DR for the Fuji cameras if this effect is not properly taken into account.
Guillermo,
These data are interesting, and I am surprised that no one has commented. I'm not that familiar with the Fuji super CCD and the R sensor component, but I would imagine that it would behave similarly to other CCDs with only one type of pixel.
In performing a sensor analysis of my D3 (CMOS sensor) using paired images at various exposures using the method of Roger Clark (http://www.clarkvision.com/imagedetail/evaluation-1d2/index.html), one can make some interesting observations. I used Iris to separate the green 1 channel and cropped the image to 201 x 201 pixels. I then subtracted one pair from the other and added an offset of 200 to avoid negative numbers and determined the standard deviations. The standard deviations of the single frames represents total noise, while the difference standard deviation removes variations due to non-uniformity of the light field (e.g. due to vignetting in the lens), dust specs on the sensor, pixel to pixel sensitivity variations and other effects.
This table shows the results of some of the data. Sigma3 is the standard deviation of the subtracted frames and it is repeated for the second pair.
[attachment=9821:Table1c.gif]
When the sensor saturates, there is an abrupt decrease in noise since all pixels are driven to their maximum value. The standard deviations of the individual frames represents mainly pixel nonuniformity and this is removed when one looks at the difference image:
[attachment=9820:ImageJ_Histo.gif]
The individual image histogram has a double Gaussian appearance since the D3 uses multiple readout channels. I think that the histogram of the subtracted frames represents quantization error.
It would be interesting if you would perform a similar analysis on your data and let us know the results. From the appearance of your histograms, I assume that you are using demosaiced data from DCRaw. It might be better to use data directly from a raw channel if this is possible.
Bill
-
There is something seriously wrong with the conclusions this website offers. I was reading a post at another photography forum that referred to the D300 ISO performance on this page: http://www.dxomark.com/index.php/eng/DxOMark-Sensor (http://www.dxomark.com/index.php/eng/DxOMark-Sensor) and when I went there I noticed that the D90 ISO performance (which uses the same sensor) was rated about half a stop better. I believe people relying on this sort of information in making a DSLR purchase are going to be seriously mislead as there is simply no way the D90 is a half stop better than the D300, and this makes me highly suspicious of all of the charts offered by this website. To put it bluntly, this site as it currently stands is simply bogus.
-
... there is simply no way the D90 is a half stop better than the D300, and this makes me highly suspicious of all of the charts offered by this website. To put it bluntly, this site as it currently stands is simply bogus.
Tony,
How do you know there is no way? What inside information do you have to support this conclusion. What test comparisons have you done?
-
Tony,
How do you know there is no way? What inside information do you have to support this conclusion. What test comparisons have you done?
A reputable reviewer has done the tests: http://www.bythom.com/nikond90review.htm (http://www.bythom.com/nikond90review.htm)
I know who I trust.
-
Hi,
There are amplifiers in the signal path. The Sony chip has on chip ADCs but Nikon normally bypasses these and uses a 14 bit external ADC instead. Even the chip itself could be better as the D90 is a later model than.
I don't think that DXO is a bogus site just because you or I don't understand their methodology or don't agree with their results. On the contrary, I really appreciate that their share their results.
Best regards
Erik
There is something seriously wrong with the conclusions this website offers. I was reading a post at another photography forum that referred to the D300 ISO performance on this page: http://www.dxomark.com/index.php/eng/DxOMark-Sensor (http://www.dxomark.com/index.php/eng/DxOMark-Sensor) and when I went there I noticed that the D90 ISO performance (which uses the same sensor) was rated about half a stop better. I believe people relying on this sort of information in making a DSLR purchase are going to be seriously mislead as there is simply no way the D90 is a half stop better than the D300, and this makes me highly suspicious of all of the charts offered by this website. To put it bluntly, this site as it currently stands is simply bogus.
-
Hi,
There are amplifiers in the signal path. The Sony chip has on chip ADCs but Nikon normally bypasses these and uses a 14 bit external ADC instead. Even the chip itself could be better as the D90 is a later model than.
I don't think that DXO is a bogus site just because you or I don't understand their methodology or don't agree with their results. On the contrary, I really appreciate that their share their results.
Best regards
Erik
The comparison is to the D90, another Nikon camera using the same sensor and it only records 12 bits whereas the D300 can record 14 bits; I believe both use a 16 bit pipeline, I know the D300 does. To quote the relevant portion (for the purposes of this discussion) of Thom Hogan's review of the D90:
"...for the main types of noise and high ISO values, the D90 is essentially a mimic of the D300....Go read what I wrote in my D300 review about noise handling; the D90 is nearly identical."
That sure doesn't look like there's a half stop difference to me -- perhaps Thom Hogan just doesn't see the differences that DXOMark is asserting are there.
These geeky, technical analysis' of cameras are already flawed even if they were accurate: http://theonlinephotographer.typepad.com/t...l-vindicat.html (http://theonlinephotographer.typepad.com/the_online_photographer/2008/11/i-feel-vindicat.html). However, the data presented by DXOMark is flawed and that makes it particularly egregious to offer it to for the purpose of "informing" some unwitting person who wants to buy the "better" DSLR.
-
I noticed that the D90 ISO performance (which uses the same sensor) was rated about half a stop better
A note without any further qualification/comparison: the sensors of the D90 and D300 are not identical. Although the overall dimension in pixels is the same, the masked areas are different as are the spectral responses.
This does not preclude, that the the sensors' electronics are the same. On the other hand, the different noise characteristics speak against the "same sensor" even regarding the electronics.
-
This does not preclude, that the the sensors' electronics are the same. On the other hand, the different noise characteristics speak against the "same sensor" even regarding the electronics.
Okay, you prefer to believe some numbers presented by DXOMark over what Thom Hogan observes. It appears to me that in the real world (the world based on knowledgeable photographer's experiences with the cameras) the D90 and D300 have very nearly identical image quality; only in fantasy land does the D90 have noticeably superior image quality.
-
Okay, you prefer to believe some numbers presented by DXOMark over what Thom Hogan observes. It appears to me that in the real world (the world based on knowledgeable photographer's experiences with the cameras) the D90 and D300 have very nearly identical image quality; only in fantasy land does the D90 have noticeably superior image quality.
For instance, in the fantasy land that is the DPR Forums
http://forums.dpreview.com/forums/read.asp...essage=30005750 (http://forums.dpreview.com/forums/read.asp?forum=1019&message=30005750)
-
Okay, you prefer to believe some numbers presented by DXOMark over what Thom Hogan observes
I prefer to believe my own measurements over what Thom Hogan observes. I came to said result and posted it much before this DXOMark comparison appeared.
It has to be noted, that my measurements relate to the noise at a given ISO, but that is only half of the subject; the other half is, how correct the nominal ISO settings are. When shooting in low-light situation, it does not solve the problem that one camera has much less noise @ ISO 1600 than the other, if the first camera requires one stop higher ISO in the same setting. (Of course, one full stop difference would be quite extreme). I can not measure this; I am using raw files made by others, mostly by Imaging Resources. I guess DXOMark publishes such a comparison as well.
On the other hand, from the point of maximum dynamic range, only the noise counts.
Anyway, the possible incorrectness of the nominal ISO values does not mitigate the fact, that the sensors of D90 and D300 are not the same; in fact they are very different.
Samples of my measurements are for example in D90 vs D300 (http://www.panopeeper.com/Demo/DR_Measurement_NikonD90vsD300.tif), a layered TIFF.
-
I see that you both are referring to comparison files from Imaging Resources. First thing I notice about these files is that they are exposed differently; for instance, at ISO 800 the D300 is exposed at 1/125 while the D90 is exposed at 1/100, that's a 1/3 of stop difference. I suspect that I do not shoot the same way Imaging Resources shoots, I utilize uni-WB to try to achieve a maximum EV and then adjust EV with the RAW converter. Nonetheless, despite the 1/3 EV difference, looking at the ISO 800 images I see practically nothing to distinguish them from each other -- so digging around in shadows to isolate differences is an exercise in pixel peeping that does nothing to elucidate which camera would be a better choice for an individual photographer -- as I stated earlier in this thread, all of these cameras will deliver more than adequate files for the vast majority of photographers under the vast majority of applications.
I wish all of you enjoyment in your pursuit of whatever it is you are searching for (my suspicion is that real photography is not the goal of these pointless exercises). I just hope that people looking for a "better" camera choice are not swayed by these flawed analysis'.
-
I see that you both are referring to comparison files from Imaging Resources. First thing I notice about these files is that they are exposed differently; for instance, at ISO 800 the D300 is exposed at 1/125 while the D90 is exposed at 1/100, that's a 1/3 of stop difference. I suspect that I do not shoot the same way Imaging Resources shoots, I utilize uni-WB to try to achieve a maximum EV and then adjust EV with the RAW converter. Nonetheless, despite the 1/3 EV difference, looking at the ISO 800 images I see practically nothing to distinguish them from each other -- so digging around in shadows to isolate differences is an exercise in pixel peeping that does nothing to elucidate which camera would be a better choice for an individual photographer -- as I stated earlier in this thread, all of these cameras will deliver more than adequate files for the vast majority of photographers under the vast majority of applications.
I wish all of you enjoyment in your pursuit of whatever it is you are searching for (my suspicion is that real photography is not the goal of these pointless exercises). I just hope that people looking for a "better" camera choice are not swayed by these flawed analysis'.
I'm happy for you that you don't need to push the performance of your cameras to the limit. BTW, uni-WB does nothing to the RAW file. It may help you to set the exposure if you're not in the habit of metering the scene before you shoot, by checking the histogram after you shoot.
There is a difference, it's quite obvious in the IR images, and much more than the 1/3 stop difference in exposure. Agreed though that not all people need the difference, any of the cameras in the thread I linked is more than capable of making great images at low to moderate ISO.
-
I'm happy for you that you don't need to push the performance of your cameras to the limit.
Nonsense. I do push my cameras to their performance limits, and that is more than what I typically need. How do you find the current cameras lacking?
BTW, uni-WB does nothing to the RAW file. It may help you to set the exposure if you're not in the habit of metering the scene before you shoot, by checking the histogram after you shoot.
It definitely helps, it is essential. What's more, other settings are involved too, including most importantly brightness. I find that the true native ISO of the D300 is closer to ISO 125 compared to what the D200 does.
There is a difference, it's quite obvious in the IR images, and much more than the 1/3 stop difference in exposure.
Laughable. I have an IR converted D200 and wouldn't waste my time doing IR shots with a CMOS sensor camera.
Perhaps you meant Imaging Resource, well I find their shooting conditions to be uncontrolled as demonstrated by using different EVs, different focal lengths, and who knows what else is different. With so many differences, it's difficult to determine what the sensor's actual differences are.
Agreed though that not all people need the difference, any of the cameras in the thread I linked is more than capable of making great images at low to moderate ISO.
That's my point, for the vast majority of users all of these cameras are more then up to the job.
-
I see that you both are referring to comparison files from Imaging Resources. First thing I notice about these files is that they are exposed differently; for instance, at ISO 800 the D300 is exposed at 1/125 while the D90 is exposed at 1/100, that's a 1/3 of stop difference
This is totally irrelevant regarding the noise, as is the lens, the distance, the subject, the illumination. ("Calibrating" the ISO is a different issue.)
despite the 1/3 EV difference, looking at the ISO 800 images I see practically nothing to distinguish them from each other -- so digging around in shadows to isolate differences is an exercise in pixel peeping that does nothing to elucidate which camera would be a better choice for an individual photographer -- as I stated earlier in this thread, all of these cameras will deliver more than adequate files for the vast majority of photographers under the vast majority of applications
I'm afraid you are not in the position to judge this; you did not ask the "vast majority" of photographers, nor did I. You are suggesting, that noise and dynamic range are non-issues with these cameras; I don't doubt that this is true regarding your photography, but looking around on several forums I am convinced, that this can not be said generally.
People are shooting in low light like in concerts or with very short shutter time without professional lighting like indoor sports in a high school gym, thus they are forced to go with high ISO. Some of the shots are still strongly underexposed; some areas can be in the seventh, eighths or ninths EV (from saturation), and in that region the noise is unavoidable with today's DSLRs (at ISO 1600 and higher), the question is only how bad. These nonsensical looking tests are analysing just that: how is the noise in the very deep shadows.
The other side of the same coin is the dynamic range. I am very often in situations, that highlights are already clipping but other areas of the image are far underexposed @ ISO 100 or 200. How much I can reclaim from the shadows while showing the details is limited by the noise. I can determine with my measurements the difference between two sensors in stops at a given noise level.
-
You are suggesting, that noise and dynamic range are non-issues with these cameras; I don't doubt that this is true regarding your photography, but looking around on several forums I am convinced, that this can not be said generally.
There is not enough difference between a D90 and a D300 to get excited about. That's the word I got back from Thom Hogan tonight when I asked him directly about this. If you want to shoot in low light, then a D700 or D3 is your tool of choice, and anyone who thinks they will get noticeably better performance from a D90 than a D300 is misinformed. For some, that misinformation will come from DXO Mark, Imaging Resource, and even this forum (at least this thread). Arguing that someone who is not getting results they are satisfied with at ISO 1600 with a D300 would be satisfied with a D90 instead is setting them up for disappointment; the same is true at base ISO as it pertains to DR.
Regarding the issues of not using the same lens, the same focal length, or the same EV; that highlights the lack of rigorous control at Imaging Resource and which is manifested by the 1/3 of a stop discrepancy in their comparative files. If you do not see that as a flaw than you are simply not paying attention. If I increase the illumination of the scene and then set the D300 to expose at 1/100 instead of 1/125 (just as the D90 was,), then the already negligible differences are practically erased.
(http://photos.imageevent.com/tonybeach/mypicturesfolder/sharing//D90-D300.jpg)
Can you see a difference in the lighting between these two? This probably explains the 1/3 of a stop difference in EVs between the D90 and D300 files. Also, when measuring noise, focus is an important consideration -- actually, you need to defocus the scene to avoid measuring actual details and confusing them with sensor noise, which is why using different lenses and having different focus points are important to getting accurate measurements.
-
There is not enough difference between a D90 and a D300 to get excited about. That's the word I got back from Thom Hogan tonight when I asked him directly about this
I would be very excited about half stop difference.
anyone who thinks they will get noticeably better performance from a D90 than a D300 is misinformed
Based on the raw image I have it is you, who is misinformed or uninformed.
Arguing that someone who is not getting results they are satisfied with at ISO 1600 with a D300 would be satisfied with a D90 instead is setting them up for disappointment; the same is true at base ISO as it pertains to DR
On what do you base your assessment? On how two anyway not noisy images compare as "whole"?
Regarding the issues of not using the same lens, the same focal length, or the same EV; that highlights the lack of rigorous control at Imaging Resource and which is manifested by the 1/3 of a stop discrepancy in their comparative files
You don't understand the very basics of measuring the noise. If you give me two shots with the same ISO, containing smooth, unicolored, evenly lit areas in the very shadows, the best if several such spots in different darkness, I can measure and compare the noise, no matter which camera, which lens, which scenery, which illumination.
The layers of the TIFF I linked above show the principle of measurement: the noise is measured relatively to the actual pixel level in an area, on the raw channel. It does not matter which color, which lens, whatever. What matters is how high the pixel level and how high the noise is. If I find such areas with closely matching pixel levels, then the noise level shows which sensor is better. If I find areas with the same noise level, then the difference between the relative pixel levels shows how "much" one sensor is better than the other. (One needs to take several samples for a reliable result.)
I uploaded a compilation of such measurements as an Excel chart (http://www.panopeeper.com/Demo/NoiseAnalysis.xls) (the 5D2 data is not reliable); this is only as demo, it does not include the D300 nor the D90.
Can you see a difference in the lighting between these two?
This is irrelevant. For example at ISO 200 neither the D300 nor the D90 images from IR can be characterized as noisy at all. However, those images are suitable to measure the noise level and predict, how the sensors compare at ISO 200 if the shot contained severally underexposed areas. That is, what dynamic range is about.
Pick very dark areas (smooth, etc. as described above) from any ISO 1600 image and measure the noise; the consolidated result shows, what you can expect in a really low light situation. If the noise in one sensor is the same as in the other but on a half stop darker area, then the first one can create an equal image in lower light, or a less noisy image in the same light (assumed, that the ISO calibrations are equal).
-
Tony, just for having the D90 a 12 bit sensor while being the D300 14 bit, one could think of differences in the ADC which could make better any of the two models, specially regarding noise. Different bitdepth should make us think of different camera hardware and thus characteristics.
I particular regarding DR, I wouldn't just look at the final DR figure which is obtained just using a specific SNR threshold criteria. DxO Mark provides full SNR plots for each camera and there it's easy to see the behaviour regarding noise at any ISO and at any EV (unfortunately they chose an uncomfortable log10 scale for the X-axis rather than log2=EV, but converting it is not difficult):
(http://img406.imageshack.us/img406/280/compmq1.gif)
Just by looking at that graph, and assuming measurements were properly done, it's easy to see that the D90 beats the D300 at any ISO and at any RAW exposure. And the difference increases the higher the ISO value, which means the D90 performs specially better in very low light conditions where a high ISO value is needed than the D300, and could thus deliver a noticeably higher DR for certain applications like indoor shooting.
A point that should be studied however is if RAW noise reduction is being applied. I think DxO Mark's RAW measurements, specially regarding noise, should be attached to some noise reduction and/or texture quality (contrast, detail) analysis. Camera vendors seem to be interested in making the RAW data less RAW everytime, and this can affect all these measurements.
BR
-
Just by looking at that graph, and assuming measurements were properly done, it's easy to see that the D90 beats the D300 at any ISO and at any RAW exposure. And the difference increases the higher the ISO value, which means the D90 performs specially better in very low light conditions where a high ISO value is needed than the D300, and could thus deliver a noticeably higher DR for certain applications like indoor shooting.
Considering that there's a year between the two cameras, this shouldn't be surprising.
Camera manufacturers have been known to make progress from one year to the next, and Nikon has been less shy than e.g. Canon in allowing the cheaper, more recent camera to catch up technologically.
A point that should be studied however is if RAW noise reduction is being applied. I think DxO Mark's RAW measurements, specially regarding noise, should be attached to some noise reduction and/or texture quality (contrast, detail) analysis. Camera vendors seem to be interested in making the RAW data less RAW everytime, and this can affect all these measurements.
That's food for thought, but I wonder what useful information we would gain from that approach.
-
Based on the raw image I have it is you, who is misinformed or uninformed.
From Imaging Resource?
(http://photos.imageevent.com/tonybeach/mypicturesfolder/sharing//Clipped.jpg)
These are screenshots from PhotoBola Raw Image Analysis, and they show where (on the bottom) the D300 file is clipped as a result of its being 1/3 stop underexposed compared to the D90 image; 1/250 compared to 1/200.
I actually wonder if you are a photographer; I have never seen a photograph of yours, just endless pixel peeping nonsense. I know you are not a scientist, as no competent scientist would draw conclusions from such flawed comparisons.
-
I actually wonder if you are a photographer; I have never seen a photograph of yours, just endless pixel peeping nonsense.
I never saw a photograph of yours and that doesn't mean you are not a photographer.
Did you try http://www.panopeeper.com/ (http://www.panopeeper.com/)?
-
I would like to see DxO labs attach some sort of noise thresholds to these DR tests... Imaging Resource's DR numbers, which I agree are less scientific, are shown at four different noise thresholds, and by far the closest match to the DxO numbers is at the most noisy of the four (really reaching into the shadows for detail most people won't want to print), which is generally within 0.2 stop, and often 0.1 stop, of he DxO number. IR's "high quality" DR numbers range from 1 to 2 stops less than this, and that varies substantially by camera - since DxO is already doing noise profiling, wouldn't it be fairly easy for them to report low-noise DR as well? They don't even need all four of IR's numbers, just the extreme (which they have) plus the low noise number on the other end...
-Dan
-
I would like to see DxO labs attach some sort of noise thresholds to these DR tests...
They use the SNR=0 dB criteria. Anyway they provide full SNR plots, so you can calculate the DR according to any criteria you like best (e.g. the more useful in photographic terms SNR=12 dB=2EV).
-
I never saw a photograph of yours and that doesn't mean you are not a photographer.
He has posted at least five to [a href=\'index.php?showtopic=29488\']ObPhoto thread #1[/a].
-
They use the SNR=0 dB criteria. Anyway they provide full SNR plots, so you can calculate the DR according to any criteria you like best (e.g. the more useful in photographic terms SNR=12 dB=2EV).
What's the rationale behind that?
-
These are screenshots from PhotoBola Raw Image Analysis, and they show where (on the bottom) the D300 file is clipped as a result of its being 1/3 stop underexposed compared to the D90 image; 1/250 compared to 1/200.
Tony, obviously you are not prepared to deal with the fact, that measuring the noise on the raw data has nothing common with judging an image based on its visual appearance.
As I posted, the difference in exposure is irrelevant. In fact, noise and DR comparison can be done (and I am often doing that) based on totally unrelated images, i.e. different sceneries, different illuminations, different exposures, etc.
Anyway, I don't know the details of the D2X comparisons, I was referring to my own measurements. However, there is one flaw on their site for sure: they should allow access to the detailed data only after a test verifying that the viewers understand what they are seeing.
-
What's the rationale behind that?
In practice, there is no distinction between SNR=1 and SNR=0; SNR=1 is the threshold of detectability of signal for an isolated pixel, but measuring the noise at zero signal is technically slightly easier. As I said, though, the two choices yield essentially identical values for the DR. SNR=1 is the lower threshold for detectability and so it leads to the most optimistic measure of DR; if you like, it is an upper bound for DR. And as Guillermo said, you can use the SNR plots to determine the DR according to any stricter criterion you may have.
-
What's the rationale behind that?
pardon? the rationale behind calculating the DR by yourself, or behind chosing 12 dB as threshold?
-
pardon? the rationale behind calculating the DR by yourself, or behind chosing 12 dB as threshold?
The rationale behind chosing 12 dB as the threshold.
-
The rationale behind chosing 12 dB as the threshold.
I am not an expert but I have found it a good value since when I have seen images (or areas of a given image) with 12 dB of SNR (S/N>=2EV, i.e. S being at least 4 times the std deviation of noise both in RAW levels) I could still distinguish detail, even if quite grainy. Of course this degree of noise will affect more to fine details than to large plain areas, but as a general rule it can be considered a good criteria.
Moreover using that criteria on Emil's plots one gets DR values (between 8-10 EV) that make sense with those figures I obtained time ago less scientifically just by visual inspection on scenes like this:
Canon 350D - ISO100
(http://www.guillermoluijk.com/article/digitalp02/zonasecuencia.jpg)
the 8th f-stop accounting from saturation is signed as -7EV so I estimated a practical usable DR for the photographer of 8 f-stops on this camera.
And later I read on Emil's article that he also suggests this figure (2EV=12 dB) so I consider it a valid tradeoff in photoghraphic applications for several reasons. No idea if the specialised literature could agree with this value.
-
The rationale behind chosing 12 dB as the threshold.
There is nothing rational on that, except that something has to be picked as the basis of comparison.
Expectations vastly differ depending on several parameters, like shooting subjects, presentation forms and sizes, personal preference. Particularly the affected area in the image and its relative lightness has to be taken into account; one tolerates much less noise in the bright areas than in the shadows, and much of that in the shadows gets cut off anyway by black point or by the lower end of the S curve.
-
Has anybody read and has opinions about that article:
http://www.astrosurf.com/buil/nikon_test/test.htm (http://www.astrosurf.com/buil/nikon_test/test.htm)
It claims that the Nikon bodies are applying some kind of raw noise reduction before saving the raw file, and thus reduce also the kind of details that can be captured.
Thus, comparison of noise levels becomes irrelevant.
Cited also on this site:
http://www.photo-lovers.org/fpsensor.html.en (http://www.photo-lovers.org/fpsensor.html.en)
-
Has anybody read and has opinions about that article:
http://www.astrosurf.com/buil/nikon_test/test.htm (http://www.astrosurf.com/buil/nikon_test/test.htm)
It claims that the Nikon bodies are applying some kind of raw noise reduction before saving the raw file, and thus reduce also the kind of details that can be captured.
Thus, comparison of noise levels becomes irrelevant.
Cited also on this site:
http://www.photo-lovers.org/fpsensor.html.en (http://www.photo-lovers.org/fpsensor.html.en)
Nikon does do NR on raw data for exposures of longer than 1/4 or 1 second, depending on the model. It can't be turned off. Important for astrophotography, but not important for many other photographic situations. As far as I am aware this is the only NR that Nikon does to RAW data that can't be turned off by the user.
-
As far as I am aware this is the only NR that Nikon does to RAW data that can't be turned off by the user.
The problem I see here is not actually whether present Nikons do this or not, but the trend of the camera vendors. If they start to play unfair pre-processing their RAW data to obtain cleaner RAW files, it will become difficult to make fair comparisions since detail and sharpness tests should be done in addition to noise measures. And I cannot imagine a standard 'detail' test that can be applied in equal conditions to any camera model (it should play with differences in the optics, in the sensor format,...).
-
The problem I see here is not actually whether present Nikons do this or not, but the trend of the camera vendors. If they start to play unfair pre-processing their RAW data to obtain cleaner RAW files, it will become difficult to make fair comparisions since detail and sharpness tests should be done in addition to noise measures. ).
Agreed.
But where does one draw the line?
Example 1: Bad pixels (i.e., pixels that are always effectively clipped to white or black). They contribute to standard noise measurements unless you use robust statistics designed to ignore them. Is it ok for a camera to fix bad pixels before writing out the raw file?
Example 2: CFA sensors often have 2 green pixels. Sometimes the two green pixel types differ significantly in response. Is it ok for a camera to balance the greens internally before writing out the raw file? (Keep in mind that green-balancing is effectively a form of noise reduction.)
Example 3: Tiling issues. Large sensors are sometimes composed of multiple pieces which have differing responses. Is it ok for a camera to "balance" the signals across the components before writing out the raw file? (Essentially performing an in-camera calibration.)
Example 4: Lens corrections, such as distortion. Is it ok for a camera to apply geometric distortion correction automatically before writing out the raw file?
(I submit these as real-world examples of things that camera makers either already do, or will be doing.)
-
Example 4: Lens corrections, such as distortion. Is it ok for a camera to apply geometric distortion correction automatically before writing out the raw file?
(I submit these as real-world examples of things that camera makers either already do, or will be doing.)
I doubt that any camera is or will be doing geometric distortion correction; actually, it can not be done at that stage. However, vignetting correction is realistic.
-
Not long ago, I would have said exactly the same thing.
-
Not long ago, I would have said exactly the same thing.
Well, I am prepared for a surprize, but as of now, I don't see any way to convert a red pixel into a green one in that stage, i.e. without anticipating the WB.
-
(...)
And later I read on Emil's article that he also suggests this figure (2EV=12 dB) so I consider it a valid tradeoff in photoghraphic applications for several reasons. No idea if the specialised literature could agree with this value.
Thanks, to both you and Gabor, I see why it's an approximate, sensible limit.