Luminous Landscape Forum
Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: Guillermo Luijk on November 18, 2007, 02:49:25 pm
-
Hi all, I have a question for those brilliant minds around here.
We all know that:
- The higher the ISO for a given exposure, the lower the SNR.
- The lower the exposure for a given ISO, the lower the SNR as well.
But if we are forced to use some exposure parameters (aperture and exposure time), what is recommended to obtain the best possible SNR (Signal to Noise Ratio), i.e. less visible noise, to push the ISO number? or to allow certain underexposure?
One could think both options would be the same in terms of visible noise, but this is not the case. I have done a test shooting the same scene with the same exposure parameters changing only the ISO, and the result was that the higher the ISO, the lower the visible noise, specially in the shadows.
Test: 3 shots at 70mm, 0.3s, f/8 using ISO 100, 200 and 400.
This is the scene:
(http://img407.imageshack.us/img407/5108/escenaln8.jpg)
And these are crops of normal, dark and very dark areas of the scene from top to bottom.
(development done with no noise reduction in DCRAW balancing exposures in PP):
(http://img140.imageshack.us/img140/921/compvr2.jpg)
I am sorry for the large file size due to noise. It is easy to see that the higher ISO is, the better SNR we achieve.
The fist conclusion I make of this is that it is better to use high ISO values than leaving our shot underexposed. Also that exposing to the right makes sense not only at ISO100 but at any ISO value, so ISO may be used to get ETTR when needed for being forced to use certain fixed exposure parameters (aperture and exposure time).
I wonder if it's the same for even higher ISO values, but I guess it is.
My question is: what explanation do you think this phenomemon has?
1. The camera applies some noise reduction in the signal amplification process when using high ISO (i.e. with large amplification values)?
2. There is some noise source that is added after amplification, so the higher the signal level is (thanks to ISO) the less that 'last minute' noise source will affect?
Regards
-
Hi!
This is really an issue I wanted to investigate myself. I don't see any obvious answer. What camera are you using, BTW?
Two possible explanations would be that if the dynamic range of the AD-converter is less than the dynamic range of the sensor amplification befor AD-conversion would reduce noise. The other possible explanation I see would be that the sensor is doing some kind of noise reduction before outputting RAW.
It seems to me that no resolution is lost when going from 100 to 400 ISO. That may indicate that no excessive reduction in luminance noise is done in the sensor. I have no further ideas.
Best regards
Erik
Hi all, I have a question for those brilliant minds around here.
We all know that:
- The higher the ISO for a given exposure, the lower the SNR.
- The lower the exposure for a given ISO, the lower the SNR as well.
But if we are forced to use some exposure parameters (aperture and exposure time), what is recommended to obtain the best possible SNR (Signal to Noise Ratio), i.e. less visible noise? Push the ISO number? or to allow certain underexposure?
One could think both options would be the same in terms of visible noise, but this is not the case. I have done a test shooting the same scene with the same exposure parameters changing only the ISO, and the result was that the higher the ISO, the lower the visible noise, specially in the shadows.
Test: 3 shots at 70mm, 0.3s, f/8 using ISO 100, 200 and 400.
This is the scene:
(http://img407.imageshack.us/img407/5108/escenaln8.jpg)
And these are crops of normal, dark and very dark areas of the scene from top to bottom.
(development done with no noise reduction in DCRAW balancing exposures in PP):
(http://img140.imageshack.us/img140/921/compvr2.jpg)
I am sorry for the large file size due to noise. It is easy to see that the higher ISO is, the better SNR we achieve.
The fist conclusion I make of this is that it is better to use high ISO values than leaving our shot underexposed. Also that exposing to the right makes sense not only at ISO100 but at any ISO value, so ISO may be used to get ETTR when needed for being forced to use certain fixed exposure parameters (aperture and exposure time).
I wonder if it's the same for even higher ISO values, but I guess it is.
My question is: what explanation do you think this phenomemon has?
1. The camera applies some noise reduction in the signal amplification process when using high ISO (i.e. with large amplification values)?
2. There is some noise source that is added after amplification, so the higher the signal level is (thanks to ISO) the less that 'last minute' noise source will affect?
Regards
[a href=\"index.php?act=findpost&pid=153903\"][{POST_SNAPBACK}][/a]
-
It is rather simple, actually: underexposure will always produce more noise, especially in the shadows.
It has to do with the linearity with which the sensor captures light (try googling "linear RAW capture" or read this: http://www.adobe.com/digitalimag/pdfs/linear_gamma.pdf) (http://www.adobe.com/digitalimag/pdfs/linear_gamma.pdf)). In other words, and very simplified, the vast majority of information is captured in the brightest steps, and conversely, the least amount of tonal information is available in the darkest areas.
-
Hi slobodan, I think we have already gone a bit beyond your point. It is well known that overexposed images have greater signal to noise ratio, but this is clear when you achieve overexposure through real exposure camera parameters (aperture, exposure time).
ISO is some kind of artificial electronic amplification of the captured signal, and we wonder what's the reason inside the sensor electronics to improve visible noise when making use of ISO to achieve the overexposure. It's not that clear.
BTW Erik I have a 350D (CMOS).
Regards.
-
Hi!
I checked this site:
http://www.clarkvision.com/imagedetail/dig...rmance.summary/ (http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary/)
It seem that the 350D has a well capacity of 43000 electrons an a read noise of 3.7 leading to a dynamic range of 13.5 steps, which is a bit more than the DR for a 12bit converter which is 12 steps. So in this case amplifying the signal before conversion may actually reduce noise.
That said I'm still surprised!
Regarding the experiment it seems to me that you have eliminated all probable "cheat" factors by using a standard RAW-converter instead of a commercial one and keeping exposure fixed.
Best regards
Erik
Hi slobodan, I think we have already gone a bit beyond your point. It is well known that overexposed images have greater signal to noise ratio, but this is clear when you achieve overexposure through real exposure camera parameters (aperture, exposure time).
ISO is some kind of artificial electronic amplification of the captured signal, and we wonder what's the reason inside the sensor electronics to improve visible noise when making use of ISO to achieve the overexposure. It's not that clear.
BTW Erik I have a 350D (CMOS).
Regards.
[a href=\"index.php?act=findpost&pid=153960\"][{POST_SNAPBACK}][/a]
-
Hi!
I checked this site:
http://www.clarkvision.com/imagedetail/dig...rmance.summary/ (http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary/)
It seem that the 350D has a well capacity of 43000 electrons an a read noise of 3.7 leading to a dynamic range of 13.5 steps, which is a bit more than the DR for a 12bit converter which is 12 steps. So in this case amplifying the signal before conversion may actually reduce noise.
That said I'm still surprised!
Regarding the experiment it seems to me that you have eliminated all probable "cheat" factors by using a standard RAW-converter instead of a commercial one and keeping exposure fixed.
Best regards
Erik
[a href=\"index.php?act=findpost&pid=153966\"][{POST_SNAPBACK}][/a]
I got to the same website this afternoon. However I had to stop as I was having a terrible headache hehe. Will give it a second try.
Yes I used DCRAW which by default performs no noise reduction and scaled all images by the same white balance factors. Next I balanced the ISO100 image to match the ISO400 with a +2EV curve, and the ISO200 image with a +1EV curve.
The crops are 50% the real size, but resized using nearest neighbour (no interpolation) so SNR remains statistically the same.
Could you please explain the mechanism why SNR is improved thanks to the DR compression? I am an engineer so you can talk in electronic terms.
-
Hi,
Sorry, I have no good explanation. My reasoning is that noise floor is below the threshold of the sensor, so some of the noise may just originate from the LSB going either one or zero. It does not really see to be a reasonable explanation to me, but I felt I could mention it anyway.
Best regards
Erik
I got to the same website this afternoon. However I had to stop as I was having a terrible headache hehe. Will give it a second try.
Yes I used DCRAW which by default performs no noise reduction and scaled all images by the same white balance factors. Next I balanced the ISO100 image to match the ISO400 with a +2EV curve, and the ISO200 image with a +1EV curve.
The crops are 50% the real size, but resized using nearest neighbour (no interpolation) so the statistical SNR is safely kept.
Could you please explain the mechanism why SNR is improved thanks to the DR compression? I am an engineer so you can talk in electronic terms.
[a href=\"index.php?act=findpost&pid=153974\"][{POST_SNAPBACK}][/a]
-
Hi,
Sorry, I have no good explanation. My reasoning is that noise floor is below the threshold of the sensor, so some of the noise may just originate from the LSB going either one or zero. It does not really see to be a reasonable explanation to me, but I felt I could mention it anyway.
Best regards
Erik
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=153977\")
Makes sense. However I have just received a quite convincent answer. the reason for the improvement is my second point: a noise source being added after signal amplification and before A/D (probably in the A/D itself):
...that noise source is probably in the ADC, although other analog circuitry could contribute as well. (I'll call it "ADC noise" regardless.)
Most cameras do not apply "lossy" noise reduction algorithms to their RAW data.
> I wonder if it's the same for even higher ISO values, but I guess it is.
Actually it isn't. Increasing the ISO amplifies both the signal and the sensor read noise (pre-amplification) relative to the ADC noise. When sensor read noise becomes the dominant contribution to shadow noise, there is no further advantage for higher ISO at the same absolute exposure.
Roger Clark has some interesting information about this stuff:
[a href=\"http://www.clarkvision.com/imagedetail/evaluation-1d2/index.html]http://www.clarkvision.com/imagedetail/eva...-1d2/index.html[/url]
--
Alan Martin
In simple terms I think this is the idea: the input to the ADC is:
OUTPUT = (S_in + N_read) * ISO + N_adc
SNR= (S_in*ISO) / (N_read*ISO + N_adc) which becomes higher the higher ISO is.
But if we push ISO a lot amplified N_read becomes dominant over N_adc:
N_read*ISO >> N_adc -> SNR ~ S_in / N_read which cannot be improved any more.
-
Hello,
There was a discussion about this somewhere on LL forum (can't find the thread now). The conclusion was as you put it - it's better to have analog amplification in camera (using higher ISO) than a digital curve applied in post-production.
If I remember correctly, it was also stated, that it's better to use whole multiples when going to higher ISOs - for base 100 you should use 200, 400, 800, 1600 and not intermediate values (like 320, 640 etc). That is apparently due to implementation of this feature in cameras - the circuitry implementing whole multiples seems to be better than for intermediate values.
Don't know if this is really true, I see MR uses a lot of intermediate ISO values when shooting, so this might not be an issue in real-world situations.
Allan
-
Hi all, I have a question for those brilliant minds around here.
We all know that:
- The higher the ISO for a given exposure, the lower the SNR.
- The lower the exposure for a given ISO, the lower the SNR as well.
But if we are forced to use some exposure parameters (aperture and exposure time), what is recommended to obtain the best possible SNR (Signal to Noise Ratio), i.e. less visible noise, to push the ISO number? or to allow certain underexposure?
I wonder if it's the same for even higher ISO values, but I guess it is.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=153903\")
Hi Guillermo,
Your test confirms what [a href=\"http://www.clarkvision.com/imagedetail/evaluation-1d2/index.html]Roger Clark[/url] has predicted.
According to his analysis, signal:noise in the highlights is determined by photon counting statistics, but read noise predominates in the deep shadows. Since the exposure (integration time) was the same for both ISOs, the noise should be the same in the highlights, since the number of photons collects was the same. However, at high ISO the read noise decreases and shadow noise is lessened. From a visual examination of your results, it looks to me that the noise differences in the two ISOs are mainly in the shadow areas as predicted.
In another essay (http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary/), Roger discusses unity gain. He says that above unity gain, there will be no further improvement by increasing the ISO setting. However, he notes that you lose 1 stop of dynamic range for each doubling of ISO.
For the Canon 1D MII, he suggests that ISO 1600 is optimum and has the lowest read noise, but that ISO 800 is nearly as good and has almost double the dynamic range.
Bill
-
Here is a test I did on the Canon G9. ISO 1600 shots were properly exposed, while ISO 800 shots were under-exposed by one stop. Raw images were processed per the Photoshop preset described at
http://forums.dpreview.com/forums/read.asp...essage=25612433 (http://forums.dpreview.com/forums/read.asp?forum=1010&message=25612433)
Exposure was increased in the ISO 800 Raw image by +1. JPEG images were postprocessed using Noise Ninja with default settings. The ISO 800 image was modified with Curves to correct exposure. Images were downsized 50% and cropped.
JPEG 1600:
(http://epaperpress.com/temp/800x_1.jpg)
JPEG 800:
(http://epaperpress.com/temp/800x_2.jpg)
Raw 1600:
(http://epaperpress.com/temp/800x_3.jpg)
Raw 800:
(http://epaperpress.com/temp/800x_4.jpg)
Conclusion:
ISO 800 images fared better than ISO 1600 images.
Raw images fared better than JPEG images.
-
Hi ePaper, the result is as you say, lower ISO = lower noise. But the way to get to the final images seems too confusing and incorrect to me. Let me explain:
Basic:
Clarity: 25
Detail:
Amount: 60
Radius: .8
Detail: 30
Masking: 0
Luminance: 20 (80 for Noise Reduction)
Color: 100
What is Detail for instance? Noise reduction? how do you know if noise reduction or that detail parameter are applied the same way to 2 differently exposed and ISO images?
I don't think this is the right way to make a noise comparision. I suggest to develop the RAW files with a totally neutral RAW developer (DCRAW is a fine tool for this), just adjust exposure, and show crops using nearest neighbour scaling (not interpolation as you used, which improves SNR).
If you make the RAW files available I would be glad to do all these steps.
Thanks
-
Hi ePaper, the result is as you say, lower ISO = lower noise. But the way to get to the final images seems too confusing and incorrect to me. Let me explain:
Basic:
Clarity: 25
Detail:
Amount: 60
Radius: .8
Detail: 30
Masking: 0
Luminance: 20 (80 for Noise Reduction)
Color: 100
What is Detail for instance? Noise reduction? how do you know if noise reduction or that detail parameter are applied the same way to 2 differently exposed and ISO images?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=154167\")
You'll find details for Photoshop's settings here:
[a href=\"http://photoshopnews.com/2007/05/31/about-camera-raw-41/]http://photoshopnews.com/2007/05/31/about-camera-raw-41/[/url]
For me it doesn't matter what the results are without sharpening and noise reduction applied since that's my workflow. I can understand how others may disagree. You may contact me at the Feedback link at http://epaperpress.com/whoami (http://epaperpress.com/whoami) and I'll link you to a download that contains the raw files. Be advised that they've been converted to DNG format
-
You'll find details for Photoshop's settings here:
http://photoshopnews.com/2007/05/31/about-camera-raw-41/ (http://photoshopnews.com/2007/05/31/about-camera-raw-41/)
For me it doesn't matter what the results are without sharpening and noise reduction applied since that's my workflow. I can understand how others may disagree. You may contact me at the Feedback link at http://epaperpress.com/whoami (http://epaperpress.com/whoami) and I'll link you to a download that contains the raw files. Be advised that they've been converted to DNG format
[a href=\"index.php?act=findpost&pid=154180\"][{POST_SNAPBACK}][/a]
But if through your workflow you get a poorer image from the highest ISO shot, which in neutral circumstances could be less noisy, it means you could be doing a non-optimum workflow and that you can get better results from your images.
I will go to your thread... OK I sent a feedback message. Are you the author of PTLens? wow, great software indeed!
PS: NP for DNG, DCRAW perfectly supports that format.
-
For me it doesn't matter what the results are without sharpening and noise reduction applied since that's my workflow. [a href=\"index.php?act=findpost&pid=154180\"][{POST_SNAPBACK}][/a]
Well, you got me interested. So I set all sliders that had an effect on sharpness and noise reduction to zero in ACR. Here are a few 100% crops. Note that Exposure, for the 800 image, was set at +1.
ISO 800:
(http://epaperpress.com/temp/800y_1.jpg)
ISO 1600:
(http://epaperpress.com/temp/800y_2.jpg)
It seems there is a lot of chroma noise in the 1600 image and colors are slightly different even though they had the same white balance. This is done in ACR and there may be other factors at work here that I'm not aware of.
-
Yeah, definitively your ISO800 is better than your ISO1600. I would love to try them on DCRAW but looking at this I don't think we will reach a different conclusion. This can be very interesting as it means the behaviour of ISO is different according to vendor.
What camera model do you have? CMOS or CCD? Mine is a 350D (CMOS ).
These are my RAW files in case anyone is interested in giving them a try:
http://www.guillermoluijk.com/download/iso100.cr2 (http://www.guillermoluijk.com/download/iso100.cr2)
http://www.guillermoluijk.com/download/iso200.cr2 (http://www.guillermoluijk.com/download/iso200.cr2)
http://www.guillermoluijk.com/download/iso300.cr2 (http://www.guillermoluijk.com/download/iso300.cr2)
-
What camera model do you have? CMOS or CCD? Mine is a 350D (CMOS ).
[a href=\"index.php?act=findpost&pid=154190\"][{POST_SNAPBACK}][/a]
The images posted here are for the Canon G9 that has a small 1/1.7 CCD. As we say, we're trying to squeeze blood from a turnip here!
-
The images posted here are for the Canon G9 that has a small 1/1.7 CCD. As we say, we're trying to squeeze blood from a turnip here!
[a href=\"index.php?act=findpost&pid=154195\"][{POST_SNAPBACK}][/a]
Well here is my test over your images. Although a bit more neutral it still shows that your ISO800 is better than your ISO1600 image, what makes me think the behaviour regarding ISO is in close relation to the sensor technology (I don't think this is a matter of internal PP since I believe cameras don't perform any NR over the RAW data).
As in your test, there is quite a lot more colour noise in the ISO1600 imagen than in the ISO800 image:
-
Anyone can explain this?
[a href=\"index.php?act=findpost&pid=154203\"][{POST_SNAPBACK}][/a]
Hello,
How is ISO 1600 implemented in G9? Is it true analog gain boost or just a trick, like ISO 3200 on 20D?
If it's the latter, this might explain the outcome. I have tried the experiment like this myself with my 30D and the jump in SNR between 1600 and 3200 is MUCH greater than between 800 and 1600.
Allan
-
How is ISO 1600 implemented in G9? Is it true analog gain boost or just a trick, like ISO 3200 on 20D?
[a href=\"index.php?act=findpost&pid=154324\"][{POST_SNAPBACK}][/a]
It's difficult to say without knowledge of the G9 circuitry. The G9 has an ISO knob that goes from 80 to 1600. There's a special scene setting that allows you to shoot at 3200 at reduced resolution.
At any rate, when I need 1600 I just dial in -1 on exposure compensation and shoot at 800 for better results.
-
I understand that lower read noise at high ISO in DSLRs is due to analog preamplification prior to A/D conversion. In CMOS sensors for DSLRs I believe every photosite has its own preamplifier. I doubt whether there would be room for an analog preamp on the tiny photosites of P&S cameras. I would therefore not expect the same degree of read noise reduction at higher ISOs on P&S cameras.
I remember testing these issues with my 60D and was surprised to find that an ISO 100 shot underexposed by 2 stops had hardly more noise that an ISO 400 shot with the same exposure.
However, since the D60 Canon have reduced high ISO noise very significantly in subsequent models, yet noise and DR at base ISO has improved only marginally.
Whilst wandering around a temple today in the late afternoon, I thought I might try a few high ISO shots in poorly lit areas for stacking experiments. I wandered around to the back of an old building, deep in shadow as the sun sank below the clouds, and shot off 7 frames in continuous mode at ISO 1600, f8 and 1/20th sec. with my 5D, hand held.
I've just converted the images, checked the darkest parts for noise, and couldn't find any. I mean, how frustrating . Perhaps I should have used ISO 3200 at f11.
Below is the ACR window displaying the conversion settings, followed by a very much lightened crop of the top right corner where I expected to see noise.
Needless to say, I'm very impressed with the low noise characteristics of my 5D. But I'd make one point; this image was correctly exposed to the right. The small areas of sky poking through the foliage are definitely blown, as indicated in the small spikes on the far right of the histogram. But the fact they are blown is inconsequential in this image.
Default sharpening and luminance smoothing of 25 was applied in ACR.
[attachment=3927:attachment] [attachment=3928:attachment]
I should add that I'm on an uncalibrated laptop. I think all my recent images would therefore probably appear a few shades lighter when viewed on a calibrated monitor (according to my printer results).
-
I have done more tests, ranging all possible ISO values my camera (350D) has.
(http://img509.imageshack.us/img509/9127/bodegontodosvw5.jpg)
Shot at T=1/4, f/13, 50mm
ISO PROGRESSION
From left to right, lighter to darker areas.
Crops at 50% done using nearest neighbour to preserve SNR.
(http://img142.imageshack.us/img142/6041/todosjl8.jpg)
ISO100 vs ISO1600
Crops at 100%:
(http://img509.imageshack.us/img509/7420/todos2iw4.jpg)
Improvement is so clear that noise manages to alter the general tone (colour) in the darkest areas.
The improvement is higher the darker the areas are. So in the darkest shadows the gap from ISO100 to ISO200 is clearly larger than from any other couple of ISO values. This is consisten with the theory that improvement is less noticeable once we reached very high ISOs.
-
I have done more tests, ranging all possible ISO values my camera (350D) has.
The improvement is higher the darker the areas are. So in the darkest shadows the gap from ISO100 to ISO200 is clearly larger than from any other couple of ISO values. This is consisten with the theory that improvement is less noticeable once we reached very high ISOs.
[a href=\"index.php?act=findpost&pid=154487\"][{POST_SNAPBACK}][/a]
Guillermo,
Nicely illustrated tests. According to Roger Clark's unity gain theory, above ISO 1100 (the unity gain of the 350D), there is no reason to increase the ISO any further. You can achieve any further boost that is needed in the raw converter. This will give you greater headroom and improve dynamic range. He adds that most astronomy enthusiasts do not bother to go beyond ISO 1600 with Canon cameras.
On a camera like the Nikon D200 (unity gain at ISO 800), the results would be even more marked.
With a medium format digital back using the KAF 39000 chip with a 16 bit ADC and a full well capacity of 60K electrons, theoretically one could stay at base ISO and make adjustments only with the raw converter, but I have not seen any tests that confirm this.
Bill
-
Guillermo,
I have downloaded the images from the first post, with ISO 100, 200 and 400 and analyzed them in raw format.
The ISO 100 raw image is less noisy than the ISO 400.
The stress is on raw. The noise is the consequence of the post processing, not of the original image.
A closer look at the raw images follows (not the ISO 200), but first some notes:
1. the black current level calculated from ACR based on the masked pixels is between 255 and 258 for both images. Everything under the respective black level is considered null (actually, ACR calculates one value for each row).
2. I selected a 100x100 pixel area from the top right edge of the monitor bezel. Many of these pixels, like in the other dark, noisy areas, are very close or even under this limit. However, the values with ISO 400 are in average higher, above the black level; this is very important.
One can't make very much out of nothing.
3. The following images are screen captures. The color is not the result of de-mosaicing but averaging of the pixels of a filter pattern (2x2 pixels).
4. In order to compare the raw images as "native" as possible, I have not applied any white balancing. No sharpening or any other hidden post processing has been applied.
4. Likewise, the exposure of the ISO 100 data is not adjusted. Instead, the source range of the mapping was limited to 244-500, instead of 255-4095. Therefor the dark areas appear much lighter, without changing their values.
So, here is the raw image of the ISO 100 shot; the selected area is framed with an orange rectangle:
(http://www.panopeeper.com/Demo/Canon_350D_ISO100%200EV_Noise.jpg)
and here is the ISO 400 shot:
(http://www.panopeeper.com/Demo/Canon_350D_ISO400%200EV_Noise.jpg)
There are two row of numbers of the images, the first of them relates to the raw values. They show the minimum, maximum and average pixel values within the selection. These show, that the ISO 100 image is more uniform (less noisy). For example the green of the ISO 100 shot are between 250 and 268, while the greens of the ISO 400 shot are between 246 and 291.
Well, if this is so, then where is the noise from?
It has two sources:
A. the black level.
Applying the black level correction "generates" many pixels with zero value. When the exposure correction is carried out, nonzero values get scaled, but it leaves the zero pixel values at zero.
Moving the mouse over the selected area in the ACR created images shows many 0 values as one color component. These are now noticable in the "new context", where the values of the non-zero pixels are quadrupled.
B. stretching of the scale. In order to achieve the same brightness with the ISO 100 image as with the ISO 400 one, a huge exposure correction is required. This stretches the pixel values in a four times larger space, which creates "holes" between the pixel values, and that appears as noise.
Here is a crop of the ISO 100 image with the statistics:
(http://www.panopeeper.com/Demo/Canon_350D_ISO100%202EV_Noise_ACR.jpg)
and the ISO 400 version:
(http://www.panopeeper.com/Demo/Canon_350D_ISO400%200EV_Noise_ACR.jpg)
The layered TIF of screenshots (http://www.panopeeper.com/Demo/Canon_350D_ISO100vs400_Noise.tif) (10 MB) contains these images without JPEG artifacts, plus screenshots from ACR and PS. The expanded histogram of PS can be used on a selection within the orange frame, which shows that the standard deviation of the ISO 100 shot (on that selection) is 5.2, while it is 8 in the ISO 400 image. The same on the ACR images shows the result of the minimum noise removal of ACR.
Note: the noise removal of ACR 4, 4.2 can not be completely turned off. This has been reduced in ACR 4.3, but there is still a small amount of noise removal, which is part of the de-mosaicing algorythm.
-
The ISO 100 raw image is less noisy than the ISO 400.
The stress is on raw. The noise is the consequence of the post processing, not of the original image.
Well it all depends on what we call post processing, and what we mean by "less noisy".
The ISO100 is less noisy, right, but has a worse signal to noise ratio, which is the parameter that really matters in order to be able to choose which image is preferred in terms of noise. To be fairly compared you must adjust RAW exposure so that bright levels on both images become the same. I don't consider this exposure correction as post processing, but anyway that discusion is all about semantics.
The question is the ISO100 has a lower signal level compared to its noise level than the ISO400 or the ISO800 so the ISO100 provides a less usable final image.
Yes, black levels on the 3 images was about the same: 256 levels. I always use DCRAW to get this information so as to make sure I have no noise reduction at all; something that as you say cannot be achieved using ACR.
BTW, what is this PhotoBOLA soft? mmm seems interesting.
Regards
-
PhotoBOla: I found it here: http://www.cryptobola.com/PhotoBola/Rawnalyze.htm (http://www.cryptobola.com/PhotoBola/Rawnalyze.htm)
-
The ISO100 is less noisy, right, but has a worse signal to noise ratio, which is the parameter that really matters in order to be able to choose which image is preferred in terms of noise
There is no question about this: if I have to choose between ISO 100 two stops underexposed and correct exposure with ISO 400, then I choose ISO 400, because it fills the gaps with useful data.
I don't consider this exposure correction as post processing, but anyway that discusion is all about semantics
I consider everything post-procesing, which is post, i.e. after the initial action. If I am recording raw data in-camera, then whatever I do with that data is post processing.
Sure it is about semantics, but it is about communications as well. Do you considere for example contrast adjustment in ACR post processing?
black levels on the 3 images was about the same: 256 levels
I think DCraw made it a bit too simple. It is the task of the raw processor to estimate the correction, based on the masked pixels. Unfortunately, those give very mixed, sometimes outright crazy information.
For example the masked pixels in your camera over the columns yielded 202-205 in average for every second column. However, the average for every other column is between 252 and 254. The individual values show huge variations, between 77 and 265 (only a cursery look).
ACR applies row-wise correction, calculated to be between 255 and 257 for the ISO 100 shot and between 255 and 258 for the ISO 400 shot. I have no idea, how (with what algorythm) ACR came to these results. The averages per row vary between 253 and over 3000 (yes, three thousand).
Perhaps the inherent noise reduction of ACR (which is part of the de-mosaicing) makes use of a more detailed analysis of the black noise data - although this non-optional noise reduction applies to such cameras as well, which do not deliver masked pixel values.
-
I think DCraw made it a bit too simple. It is the task of the raw processor to estimate the correction, based on the masked pixels. Unfortunately, those give very mixed, sometimes outright crazy information.
DCRAW estimates the correction by reading the level in hidden pixels, that's why it substracts a different amout of black levels for each individual RAW file. No idea how precise is the calculation of these levels, we should look at DCRAW's code or ask David Coffin.
When I have plotted undemosaiced 12-bit histograms of my RAW files I found no information below level 255:
(http://img228.imageshack.us/img228/9186/histoku1.gif)
If you substract black by 3000, you would eliminate most of the image! bearing in mind it reaches a maximum of 4095 (less in some other cameras).
-
When I have plotted undemosaiced 12-bit histograms of my RAW files I found no information below level 255
There is plenty of data below 255. The ISO 100 image has 52494 red, 50245 green and 32533 blue pixels between 245 and 254. The ISO 400 image starts at 233 (only two pixels); there are 12634 red, 8930 green and 12656 blue pixels up to 254.
Following two images show the bpttom right corner. Note, that in this view not even the averaging of the pixels of the CFA arrays has been done; every display pixel represents the corresponding image pixel in the respective color, with mapped intensity.
(The pixel statistics in the image relate to the range up to 255, which I selected for the sake of displaying everything over 254 as black.)
(http://www.panopeeper.com/Demo/Canon_350D_ISO100_under_255_channel.jpg)
(http://www.panopeeper.com/Demo/Canon_350D_ISO400_under_255_channel.jpg)
The question is, how much of this data carries useful information, not pure noise. These images show the outline of the complete image, but that reflects only the fact, that the "undercutting" values occured in the very dark areas.
If you substract black by 3000, you would eliminate most of the image! bearing in mind it reaches a maximum of 4095 (less in some other cameras).
This is not so simple. In many raw images the clipping point depends on the ISO. The Canon 20D's clipping point is always 4095, like the 350D appears (I have not seen images of the 350D with all possible ISOs, but if ISO 100 clips at 4095, then most probably all ISO settings go up to 4095). However, for example with the Canon 40D, the clipping point is between 12700 and 16383, depending in the ISO.
-
The question is, how much of this data carries useful information, not pure noise. These images show the outline of the complete image, but that reflects only the fact, that the "undercutting" values occured in the very dark areas.
This is interesting Pano. And I don't think those data are only noise, at least not all of them. They can also be very low but useful levels, coming from pixels that got very little light excitation. In fact noise is surely some levels above in many pixels.
And if they are just noise appearing on super dark areas outlining them, that is information anyway, so why should we loose it?
Looking at the pure RAW histogram of the image you are referring to (my ISO100 test of the computer), I can see there is a soft end of the histogram near level 256, and as you say, there is quite a lot of non-zero information there.
I think the reason for substracting a black level being a power of two must be simplification, I cannot find any other explanation. I will ask David Coffin but I don't think it was a problem to substract 246 (you can see I found the same value as you in my histogram for the beginning of the information , leaving aside a couple of G and B pixels with a 0 value):
Filled levels: R: 3849 (5,9% of available), range [246..4095]
(http://img402.imageshack.us/img402/1733/mg1851hisyy9.gif)
Zoom 1:1 12-bit histogram of RAW file showing clipped data due to 256 black substraction.
Definitively I am going to write to David Coffin to find out why he (and it seems ACR too) always substracts an integer power of two (I have developed some RAW files in which the black point substracted was 128, but 256 is more usual).
Luckily DCRAW allows you to develop setting the desired black point, in this case using: dcraw -k 246 would have saved those levels. But to calculate it in advance is time consuming.
PS: BTW if you like to investigate detailed histograms, and it seems you do, maybe you are interested in a little program I wrote called Histogrammar that plots histograms in the 0..65535 range. Also calculates logarithmic histograms for dynamic range calculations.
Download it from here:
Histogrammar Download (click DESCARGAR) (http://www.guillermoluijk.com/software/histogrammar/index.htm)
There is a tutorial although I guess you don't need it hehe: Histogrammar Tutorial (http://www.guillermoluijk.com/tutorial/histogrammar/index_en.htm)
-
I don't think those data are only noise, at least not all of them
Nor do I, and I am not convinced, that ACR (or even DPP) makes full use of every bit of information.
Definitively I am going to write to David Coffin to find out why he (and it seems ACR too) always substracts an integer power of two
This is not so with ACR. I uploaded three text files with the related values:
Masked pixels at the top, ISO 100 (http://www.panopeeper.com/Demo/GLuijk_iso100_MaskedTop.txt)
Masked pixels at the left, ISO 100 (http://www.panopeeper.com/Demo/GLuijk_iso100_MaskedLeft.txt)
Vertical black level array, ISO 100 (http://www.panopeeper.com/Demo/GLuijk_iso100_VertBlackLevels.txt)
The latter shows the values generated by ACR (actually, by the DMG converter, but they are supposed to work the same way). ACR ignores the masked pixels at the top and creates a value for each row.
Note, that the 350D has masked pixels at the top and the left, but not at the bottom and the right.
I have developed some RAW files in which the black point substracted was 128, but 256 is more usual)
Not with the 350D, I guess. The average black level is around 128 with the 20D, 30D, 1DMarkII, 1DsMarkII, 5D, but it is around 256 with the 400XTi and 350D. I don't have any image from a 300D, nor from the 10D.
if you like to investigate detailed histograms, and it seems you do, maybe you are interested in a little program I wrote called Histogrammar that plots histograms in the 0..65535 range. Also calculates logarithmic histograms for dynamic range calculations
These look fancier, than the histograms created by Rawnalyze:
(http://www.panopeeper.com/Demo/40D_ISO/ISO100-160-200_Histograms_1-10s.gif)
However, the space in a dialog window is a serious consideration when deciding the shape of a diagram. The histograms of Rawnalyze are 512 pixels wide, plus frame, etc. With some monitors this is nothing, but with some other monitors, it reaches the edge easily.
Another issue: I see a problem with the dynamic range without knowing the camera exactly. From where should the dynamic range be counted backwards? Most cameras' clipping points are not at the numerical limit of the respective bit depth, and it even depends on the ISO.
For example three new Canon cameras have now 14bit depth, but the maximum value is 16383 - or less than 13000, depending on the ISO. If I don't know the camera, I can't calculate in stops.
-
Not with the 350D, I guess. The average black level is around 128 with the 20D, 30D, 1DMarkII, 1DsMarkII, 5D, but it is around 256 with the 400XTi and 350D. I don't have any image from a 300D, nor from the 10D.
Yes, I think it was with the 350D, my only camera. Although it is true that many people has lent me RAW files, so actually you could be right.
However, the space in a dialog window is a serious consideration when deciding the shape of a diagram. The histograms of Rawnalyze are 512 pixels wide, plus frame, etc. With some monitors this is nothing, but with some other monitors, it reaches the edge easily.
Another issue: I see a problem with the dynamic range without knowing the camera exactly. From where should the dynamic range be counted backwards? Most cameras' clipping points are not at the numerical limit of the respective bit depth, and it even depends on the ISO.
I use a 768 (3*256 pixels) wide display, but you can zoom both axis from 1:1 up to 1:256, which is equivalent to Photoshop's 8-bit range histograms. Of course you can move to any point with a slide bar, it's easy and practical.
Regarding the dynamic range, the logarithmic mode allows to calculate the dynamic range of a scene visually by using a log scale in the X axis.
But this only makes sense when displaying linear images, like those from DCRAW. It always plots 16 f-stops histograms, which is the maximum a 16-bit linear value can represent. Wether the lowest f-stops really make sense or correspond to noise, will depend on the width of the DR, the shooting conditions and sensor quality.
This image (12MB) (http://www.guillermoluijk.com/download/11m.tif) is the result of blending 3 shots at different exposures. Noise is practically negligible at all f-stops, and I achieved to record the almost 13 f-stops of the scene's dynamic range:
(http://www.guillermoluijk.com/tutorial/histogrammar/histolinlog.gif)
It also plots the EV distributions of any scene. For instance I used this image:
(http://www.guillermoluijk.com/article/digitalp02/zonas.jpg)
in an article (http://www.guillermoluijk.com/article/digitalp02/index.htm) about dynamic range, showing that noise is the real limitation of the DR in the shadows:
(http://www.guillermoluijk.com/article/digitalp02/zonasecuencia.jpg)
(nearly 8 f-stops of usable dynamic range in my 350D)
-
showing that noise is the real limitation of the DR in the shadows
If one ignores noise, one can find details several stops under the "declared" bottom. One has to define, what is acceptable. The definition is easy, but it should be measurable, and that's not so easy.
Regarding the dynamic range, the logarithmic mode allows to calculate the dynamic range of a scene visually by using a log scale in the X axis
Calculating the range is one thing, but putting it on a correct scale is more difficult.
If you have a certain camera and you see a scaled histogram, you would expect that reflecting the camera. As the low end is not firm, one has to start at the clipping point, let's declare that 0EV or +10EV or whatever, and from there one can scale backwards. One has to "know the camera" in order to establish the 0EV point; the actual image tells nothing.
For example a fellow poster on DPReview shot a white board with the 40D *totally* overexposed (I don't know, with how many EV bias, but it must have been at least +3EV), so that all pixels are really clipped. He did this with all ISOs, and from that I established the list of clipping points. In the meantime I too got a 40D, which confirmed, that
I. there are variations between camera copies,
II. there are within a few levels, i.e. negligable,
III. the variations between ISOs are huge.
From here, one can draw an EV scale over the histogram.
It always plots 16 f-stops histograms, which is the maximum a 16-bit linear value can represent
I am often the domestic pedant (due to my education and my occupation). So, I have to remark: there is no such relationship.
This image (12MB) (http://www.guillermoluijk.com/download/11m.tif) is the result of blending 3 shots at different exposures
This is an impressive what? Like a sports place, but there is nothing for the spectators. The same with theater. Perhaps an exhibition hall? (Or the inside of a paper box?)
-
If one ignores noise, one can find details several stops under the "declared" bottom. One has to define, what is acceptable. The definition is easy, but it should be measurable, and that's not so easy.
I agree, this is the same as we were previously commenting, noise can give information even if just the presence of noise reveals the darkest shapes.
Well my point may be simplistic but I think it is right, tell me if you agree: sensor is a linear device (of course not ideal, I did some tests to plot the real curve response and as one could expect in the ends: deep shadows and blown highlights, it starts to show non linear behaviour). So I set the 0EV reference in the 65534 level of the developed RAW, and start to count backwards allocating all levels in the image. E.g. a level 32768, in the middle of the range, would correspond to the border between f-stop 0EV and -1EV.
Levels with 65535 value are considered blown, i.e. higher than 0EV, so are ignored for the log histogram. In the same way I discard 0 values since they would belong to -infinite EV.
The non-linear behaviour in the ends could lead to some errors in the precise allocations of nearly blown highlights and deep shadows, but the middle part of the range (99% of the total information) should represent a real EV distribution of the scene under observation, don't you agree?
hehe that place is a memorial for the Alqaeda victims of the 11M in Madrid, so there is nothing to do there or to sit in but observe. Strangely shooting on a tripod is allowed in that place, while it is not in the rest of the train station. I waited till nobody was there since blending images with moving parts produces undesired ghosting (BTW I am improving my blending routine with an anti-ghosting algorithm that I think will work fine for small moving parts like tree leaves or water waves).
The room must be a square of about 20x20m with a strong natural light source in the middle of the ceiling which provoques a high dynamic range scene.
-
I agree, this is the same as we were previously commenting, noise can give information even if just the presence of noise reveals the darkest shapes.
Well my point may be simplistic but I think it is right, tell me if you agree: sensor is a linear device (of course not ideal, I did some tests to plot the real curve response and as one could expect in the ends: deep shadows and blown highlights, it starts to show non linear behaviour).
[a href=\"index.php?act=findpost&pid=154902\"][{POST_SNAPBACK}][/a]
The shadows are totally linear in Canon RAW data, but they become non-linear when they are clipped at the blackpoint, snce all you have near black is a lot of zeros and some positive numbers due to noise, but all the negative numbers due to noise are clipped away. The mean of black, after clipping, is always positive with noise.
This is why I think that blackpoint clipping should be saved until just before gamma is applied to the image. Up to that point, offsets should be subtracted so that black is zero and there are negative numbers due to noise.
White balance, demosaicing, colorimetric adjustments, resampling, etc, can all be done with negative values present, and done better.
Think about what happens to the color of noise when the WB is done for incandescent - if you clip at black first, then zero is anchored, and blue noise gets exaggerated only in the positive direction. If the blackpoint is unclipped, the WB pulls the negative blues lower, and the positive blues higher, and therefore the net, low-frequency effect is no exaggeration of the blue at all, and this is especially true if you downsample your image before blackpoint clipping.
-
If one ignores noise, one can find details several stops under the "declared" bottom. One has to define, what is acceptable. The definition is easy, but it should be measurable, and that's not so easy.
Calculating the range is one thing, but putting it on a correct scale is more difficult.
[a href=\"index.php?act=findpost&pid=154844\"][{POST_SNAPBACK}][/a]
One of the issues that determines practical DR is desired resolution. Many of the scientific approaches tend to overlook this aspect, and concentrate on the precision of individual pixels as measuring devices. With a D3 or a 1Dmk3, You can probably read big, bold white fonts on a black background where white is 14 stops or more below RAW saturation, at ISO 1600. I've done it 12 stops below with my 20D (white = 1 ADU, or about 0.8 average photons). If the image were text with thin fonts, close to pixel width, then of course you have to take pixels and their noise more literally (no averaging possible), and would have relatively little usable DR.
-
Hello folks -
Please allow me to hijack your interesting discussion a bit.
I have a P45+ back. And am getting stripes in images from ISO 400.
The back could give me good images up to ISO 3200 (I like grain), were it not for the stripes.
Strange thing is, every Raw converter I've tried is writing *different* stripes !!!!
I wonder whether this has something to do with the dark pixels.
I can send people a link to file downloads of Raw images if interested.
A glance here might be worthwhile if this topic interests you.
http://luminous-landscape.com/forum/index....showtopic=20901 (http://luminous-landscape.com/forum/index.php?showtopic=20901)
As I said, I can send people Raw images or make some; the backs are well linearised, I believe and have good A/D converters.
Edmund
-
I have a P45+ back. And am getting stripes in images from ISO 400
Well, Edmund, you are one of the few lucky ones, who have not been confronted with this phenomenon before (those, who say otherwise, have not taken a really close look and they don't know, that this is omnipresent).
Anyway, instead of speculation, what about posting the raw images, so that others can take a really close look?
-
yes, I also would like to take a look at your RAW files, even at ISO400 you could have underexposed the escene and there is no sensor today that won't display some noise in an underexposed ISO400 image.
Also there seems to be some trouble with the blue channel: I think the black point of the blue channel is wrong and that gave your pic a blue appearance and a lot of blue noise in the shadows.
(http://img410.imageshack.us/img410/1148/index2tm5.jpg)
-
tell me if you agree: sensor is a linear device (of course not ideal, I did some tests to plot the real curve response and as one could expect in the ends: deep shadows and blown highlights, it starts to show non linear behaviour)
Sensors are very linear devices, perhaps even more, than you think.
Pls elaborate the circumstances you think, that they are not linear in the highlights (the shadow is a different issue, for it is difficult to draw the line).
I have seen demonstrations of non-linearity with Imatest, which turned out to be useless, at a close look.
So I set the 0EV reference in the 65534 level of the developed RAW, and start to count backwards allocating all levels in the image
The numerical range of the raw values may be {black level - 4095} or {black level - 16383} or {black level - 65534} etc. or anything in between.
What is your basis to put the 0EV at any particular point, if you don't know that particular camera?
For example the Nikon D2X tops at 3880 with greens, while the reds go to 4095I listed already the pecularities of the Canon 40D.
This depends on ISOs and on the "kinfd" of the p[ixel in the CFA.
Even worse: the two greens of the CFA (remember: red-green, green-blue, this is the most popular arrangement) can have different responses.
I reiterate it: in order to establish an EV scale, one has to be able to define a known, proven point. In other words: if you don't know the camera's characteristics (at all ISOs), then you can not scale the histogram. If you do, that is good only for the "unwashed".
-
Your wish is my command - here is the download link for the Raw file -
But please understand that I am happy with the noise and the color, it's just the %*#ing streaking that bothers me.
http://download.yousendit.com/F3434A597701B34F (http://download.yousendit.com/F3434A597701B34F)
Edmund
yes, I also would like to take a look at your RAW files, even at ISO400 you could have underexposed the escene and there is no sensor today that won't display some noise in an underexposed ISO400 image.
Also there seems to be some trouble with the blue channel: I think the black point of the blue channel is wrong and that gave your pic a blue appearance and a lot of blue noise in the shadows.
(http://img410.imageshack.us/img410/1148/index2tm5.jpg)
[a href=\"index.php?act=findpost&pid=155085\"][{POST_SNAPBACK}][/a]
-
Your wish is my command - here is the download link for the Raw file -
But please understand that I am happy with the noise and the color, it's just the %*#ing streaking that bothers me.
http://download.yousendit.com/F3434A597701B34F (http://download.yousendit.com/F3434A597701B34F)
Edmund
[a href=\"index.php?act=findpost&pid=155089\"][{POST_SNAPBACK}][/a]
I think that's a link to the TIFF Edmund, not the RAW.
-
Guillermo, I looked at the link, that's a small file, only 32MB - it's the Raw. Tiff is a container data format as we all know, and the Phase Raw files come with a TIF extension, and an embedded preview. In case of doubt point CS3, the C1 beta or Raw Developer at it, I'll be glad to repost if I made a mistake.
Edmund
I think that's a link to the TIFF Edmund, not the RAW.
[a href=\"index.php?act=findpost&pid=155092\"][{POST_SNAPBACK}][/a]
-
Edmund,
I took a look of the image, and honestly, I don't know, why you are surprized.
This shot was FOUR stops underexposed:
(http://www.panopeeper.com/Demo/Edmund_CF000598_raw_histogram.gif)
The arm of the lady is still very clean. However, the darker area ends at ZERO pixel values (the figures over the image reflect the pixels of the selected area framed in orange):
(http://www.panopeeper.com/Demo/Edmund_CF000598_%200EV.jpg)
This already shows the huge differences between the pixels; however, if you boost the exposure by 2EV, then the zeroes remain zero and the difference between the pixels gets much larger:
(http://www.panopeeper.com/Demo/Edmund_CF000598_%202EV.jpg)
I don't know why the noise appears arranged horizontally (streaks). Perhaps a totally blown (but really, really blown, in all channels) raw shot could give some hints.
If these are not the "stripes" you were referring to, then pls. describe them in detail, or post a screen shot of them.
-
Edmund,
I took a look of the image, and honestly, I don't know, why you are surprized.
This shot was FOUR stops underexposed:
[a href=\"index.php?act=findpost&pid=155117\"][{POST_SNAPBACK}][/a]
4 stops underexposed? That's interesting. I applied the auto setting in ACR and it produced a +1.7 stop exposure compensation without any adjustment of the highlight recovery slider. Since the histogram now looks as far to the right as it should be, the implication is that Edmund's shot was underexposed by 1.7 stops.
Can you explain this anomaly?
[attachment=3972:attachment]
Anyway, whether it's 4 stops or 1.7 stops underexposed, it's clear that some of these DB users need to attend a tutorial on ETTR .
-
Panopeeper,
I reduced recovery, fill, brightness and contrast to their minimum level then applied +4 EC adjustment. It seems clear that from ACR's perspective Edmunds image is definitely not underexposed by as much as 4 stops.
At +1.7 stops you can still notice faint highlight detail that shows on the far right of the enlarged histogram below. This detail is not necessarilly inconsequential to the fashion photographer who wants to retain as much detail as possible in skin highlights, so I think 4 stops underexposure is an exaggeration, bearing in mind that people have to use a RAW converter that's convenient and lends itslef to a reasonably efficient work flow. I know David Coffin's DCRAW can often extract more detail from a RAW image, but it's not easy to use, is it!
[attachment=3973:attachment] [attachment=3974:attachment]
-
Can you explain this anomaly?
I forgot to add a disclaimer. As this is the only raw image I ever saw with the Phase One P45+, I had to rely on the information Adobe's DNG converter gave, namely that the numerical range of the raw values is 0-65535, i.e. the full 16-bit range is used.
If this is not true, then a larger proportion of the total range has been used than I assumed. A totally overexposed shot would reveal the true clipping points.
Another aspect is, that there are some pixels higher than 4600, which I took (arbitrarily) as the upper limit of this shot. 1928 red and 1820 green pixels are between 4600 and 9000 something. However, I regarded these less than 0.03% pixels as negligable; these are in the lamp. If I were to shoot this scenery (and if there was enough light), I would rather expose it so high, that these few thousand pixels clip, in order to better expose the shadows.
So, yes, the 4 stop is partly speculation (because of the clipping point), partly subjective (because of ignoring those high pixels).
On the other hand, 65% of the pixels are zero. Though the de-mosaicing changes on that, still, one or more color components are zero in large parts of the image, even with +4EV. Again, 16 times 0 is still 0.
-
I forgot to add a disclaimer. As this is the only raw image I ever saw with the Phase One P45+, I had to rely on the information Adobe's DNG converter gave, namely that the numerical range of the raw values is 0-65535, i.e. the full 16-bit range is used.
If this is not true, then a larger proportion of the total range has been used than I assumed. A totally overexposed shot would reveal the true clipping points.
Another aspect is, that there are some pixels higher than 4600, which I took (arbitrarily) as the upper limit of this shot. 1928 red and 1820 green pixels are between 4600 and 9000 something. However, I regarded these less than 0.03% pixels as negligable; these are in the lamp. If I were to shoot this scenery (and if there was enough light), I would rather expose it so high, that these few thousand pixels clip, in order to better expose the shadows.
So, yes, the 4 stop is partly speculation (because of the clipping point), partly subjective (because of ignoring those high pixels).
On the other hand, 65% of the pixels are zero. Though the de-mosaicing changes on that, still, one or more color components are zero in large parts of the image, even with +4EV. Again, 16 times 0 is still 0.
[a href=\"index.php?act=findpost&pid=155143\"][{POST_SNAPBACK}][/a]
I see. However, the fact remains that with the latest version of ACR this image is definitely not underexposed by nearly as much as 4 stops. Did you see what happens with a +4 EC adjustment in ACR above?
-
I see. However, the fact remains that with the latest version of ACR this image is definitely not underexposed by nearly as much as 4 stops. Did you see what happens with a +4 EC adjustment in ACR above?
I do realize that. If Edmond provides a blown raw shot (with ISO 800, like this shot), then we can see the clipping point (or points, for the sensels with different filters may have different clipping points). Perhaps is the 65535 totally off.
In the meantime I took another look at the raw file (now in raw, not in DNG).
It contains the WB as follows:
1. coefficients 1.00, 1.37, 4.36
2. Kelvin 1003, tint 0.0185
THIS changes the situation radically. ACR uses Kelvin 2000, tint -3. I don't know, what coefficients this means, but it pushes up the pixel values a lot, and it creates artificial clipping (see the highlight clip warning).
When you add 4EV, the lamps and the arm of the lady appear clipped. ACR never shows the raw histogram, but there is a way to find out, if there was true clipping somewhere ("true" means, if ACR believes there was clipping): try to pick WB on such points. If you can (and you can on any point of this image), then ACR believes there is no clipped pixel in that small area. If ACR thinks there are clipped pixels in that 5x5 or what size area, then it does not make WB adjustment. If you click once more, the message appears, that this is "too bright".
Anyway, g'd night for now.
-
This shot was FOUR stops underexposed
hehehe I had this feeling. You took advantage on me Pano, I wanted to be first! how should I access the RAW file contained in this TIF file? I already had this problem in the past and couln't find out.
Edmund, you definitively don't deserve the good stuff you have! (just kidding ).
BTW, anyone knows the expression that interrelates the WB linear RGB coefficients with the Kelvin/Tint values? I would be very interested in that, and always when I tried to find info about it got mad with black body radiation sites that I did not understand.
Regards.
-
how should I access the RAW file contained in this TIF file? I already had this problem in the past and couln't find out
Well, if you do find it it, please tell it to me.
The TIFFs direct from the cameras are often messy. Sometimes very messy. Canon, and particularly Nikon violate TIFF rules en mass.
The PhaseOne P45+ is no exception. Nothing goes without reverse engineering effort.
anyone knows the expression that interrelates the WB linear RGB coefficients with the Kelvin/Tint values?
There is no such generic conversion; the coefficients depend on the spectral response of the filters. There are slight variations even between copies of the same maker and model; therefor, if someone takes it very seriously, one needs to calibrate ACR to his specific camera copy.
-
how should I access the RAW file contained in this TIF file? I already had this problem in the past and couln't find out.
[a href=\"index.php?act=findpost&pid=155152\"][{POST_SNAPBACK}][/a]
I don't understand the problem here. Whilst the file name has a TIF extension, the Document Type in ACR is described as Camera Raw image.
-
I don't understand the problem here. Whilst the file name has a TIF extension, the Document Type in ACR is described as Camera Raw image.
[a href=\"index.php?act=findpost&pid=155248\"][{POST_SNAPBACK}][/a]
I cannot open it from ACR (V3.X CS2). I cannot develop it from DCRAW. That's the problem. None of them recognize this file as a RAW file.
-
I cannot open it from ACR (V3.X CS2
I guess the camera support has been built in after that ACR version. Download the newest DNG converter (it comes with ACR 4.3, you just throw away ACR, which you can't use with CS2).
Convert the file in DNG and then you can process it with older ACR as well.
I thought you wanted to process the image with your own program. That does not work without knowing the information, which is not in the raw file as standard TIFF tags (for example there is no IFD at all for the main image).
-
Hi all, I have a question for those brilliant minds around here.
We all know that:
- The higher the ISO for a given exposure, the lower the SNR.
Yes, if the relative exposure (EC) stays constant, and the absolute exposure (photons collected in the sensor wells) remains the same.
- The lower the exposure for a given ISO, the lower the SNR as well.
This is always the case, as a non-defective camera would have to intentionally go out of its way to provide a lower SNR at a higher tonal level at the same ISO. The firmware would literally have to say, "if the exposure is this high, add this much noise".
But if we are forced to use some exposure parameters (aperture and exposure time), what is recommended to obtain the best possible SNR (Signal to Noise Ratio), i.e. less visible noise, to push the ISO number? or to allow certain underexposure?[a href=\"index.php?act=findpost&pid=153903\"][{POST_SNAPBACK}][/a]
That really depends on the camera. All the issues here are with read noise - shot noise depends on sensor exposure ONLY, as shot noise is really signal, in an empirical sense, even though it is noise as far as our IQ ideals are concerned. Quantization is not an issue in the RAW data of current cameras, except for extreme imaging like stacking multiple images for astrophotography.
Generally speaking, if you shoot RAW and your RAW converter allows you to bring up the shadows without suppressing them in any way, then CMOS cameras with high-ISO optimizations (like most Canons, and the Nikon D3) give much better read noise performance with higher ISOs (ignoring clipping issues), especially at the lower end of the ISO range (100 pushed to 400 is far worse compared to 400 than 400 pushed to 1600 is compared to 1600). If you look at typical read noises at different ISOs, you can see why. My Canon 20D has read noises of 2.07, 2.2, 2.45, 3.2 and 4.7 ADU, repectively, for ISOs 100, 200, 400, 800, and 1600. There is almost no difference in read noise between the lower ISOs, in ADUs. Between 800 and 1600, the difference is not quite 2x, but is a lot closer. So, if you under-expose ISO 100 to do 200, then your new, adjusted relative read noise is 4.14, almost as much as ISO 1600! If you push 800 to 1600, then your new relative read noise is 6.4, a little worse than ISO 1600. It works in the other, way, too for pulling. Pulling ISO 400 to 100 results in a relative read noise of 0.61 ADU (or 2.45 at a relative bit depth of 14 bits instead of 12, depending how you look at it). This is why extreme positive EC works nicely for subjects that have a majority of their tones concentrated in its higher levels, like a white wall where everything else is darker, or even a grey wall where everything else is darker.
For the majority of the rest of the cameras out there, the only difference in the state of RAW data at different ISOs is due to differences in gain before hitting the ADC, typically resulting in about 12 to 15x the read noise at ISO 1600 compared to 100. Shooting RAW, and using a converter that doesn't trash shadow areas, these cameras give you less back by shooting at high ISOs, considering what they give you in extended headroom through under-exposure at lower ISOs. There is also a possible issue that an under-exposed image will not be processed with enough precision in a converter (although this shouldn't really have to be an issue). You definitely don't want to under-expose JPEGs with the majority of cameras, as JPEGs tend to trash the shadows and make them less pushable.
Some cameras have even more reason to under-expose at low ISOs. My Panasonic FZ50 gets crazy stupid at ISO 1600; not only does it have far more than 16x the read noise of ISO 100, but the blackpoint is in the wrong place, causing a color cast to the deep shadow areas. ISO 100 blackpoint is right on the money, even after multiplying the RAW data by 16.
EDIT - I forgot to mention that higher ISOs on many cameras are achieved solely through digital math, so any gains from higher ISO stop at the highest analog ISO.
-
If this image from Edmund really is 4 stops underexposed then that would place the P45+ as a camera with better high-ISO noise characteristics than Canon DSLRs.
Does anyone believe this?
-
If this image from Edmund really is 4 stops underexposed then that would place the P45+ as a camera with better high-ISO noise characteristics than Canon DSLRs.
Does anyone believe this?
[a href=\"index.php?act=findpost&pid=155408\"][{POST_SNAPBACK}][/a]
There are too many unknowns to come to a good conclusion. It may be that the RAWs are reporting the wrong whitepoints, or the software is misunderstanding them. What if ISO 800 is just ISO 100 or 200 or 400, under-exposed, in the RAW? Then, ACR could be using the wrong internal gain for high-ISO shots, requiring perhaps +1.75 EC for what should really be -0.25, or -1.25, etc.
-
What if ISO 800 is just ISO 100 or 200 or 400, under-exposed, in the RAW? Then, ACR could be using the wrong internal gain for high-ISO shots, requiring perhaps +1.75 EC for what should really be -0.25, or -1.25, etc.
[a href=\"index.php?act=findpost&pid=155499\"][{POST_SNAPBACK}][/a]
The latest ACR includes the P45+. What are you saying? Adobe has got it wrong and doesn't know the difference between ISO 100 and 800?
-
The latest ACR includes the P45+. What are you saying? Adobe has got it wrong and doesn't know the difference between ISO 100 and 800?
[a href=\"index.php?act=findpost&pid=155531\"][{POST_SNAPBACK}][/a]
Adobe have been known to use some very proprietary definitions of standard numbers ... And conversely, l ISO descriptions of cameras are a bit open to doubt; I think camera manufacturers tend to be optimistic.
Edmund
-
My Canon 20D has read noises of 2.07, 2.2, 2.45, 3.2 and 4.7 ADU, repectively, for ISOs 100, 200, 400, 800, and 1600. There is almost no difference in read noise between the lower ISOs, in ADUs. Between 800 and 1600, the difference is not quite 2x, but is a lot closer. So, if you under-expose ISO 100 to do 200, then your new, adjusted relative read noise is 4.14, almost as much as ISO 1600! If you push 800 to 1600, then your new relative read noise is 6.4, a little worse than ISO 1600.
This is very interesting John. What kind of unit is ADU?.
I don't understand this table however at Roger Clark's here (http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary/):
Read Noise (electrons)
----------------------------------------------------------------------------------
Canon Canon Canon Canon Canon Canon Canon Canon Nikon Nikon Nikon Canon Canon
1DMII 5D 20D 10D 350d 300D 40D* 400D D200 D70 D50 S60 S70
------------------------------------------------------------------------------------------
ISO 50: 30.6 59.7 13.6 4.1
ISO 100: 16.6 30.1 25.3 15.9 21.6 17.9 10.0 3.4
ISO 200: 8.95 15.6 13.5 11.0 11.5 9.9 8.1 13.4 3.2
ISO 400: 5.56 8.4 7.5 10.6 7.2 6.5 7.0 7.7 6.3 4.3
ISO 800: 4.04 5.2 4.8 9.0 4.9 10 5.2 7.4 13 7.47
ISO1600: 3.90 3.7 3.6 9.0 3.7 4.3 7.4
ISO3200: 3.93 3.7
Could you explain what is he calling "read noise" here please? the figure goes down with ISO.
Thanks
-
Zoom 1:1 12-bit histogram of RAW file showing clipped data due to 256 black substraction.
Definitively I am going to write to David Coffin to find out why he (and it seems ACR too) always substracts an integer power of two (I have developed some RAW files in which the black point substracted was 128, but 256 is more usual).
My guess is that it's because using a power of two is quicker.
As long as you deal with powers of two, a shift left or shift right can divide by or multiply by powers of two easily, and it's often quicker than performing calculations.
If you shift right by N bits, then you've effectively lost the information in the N least significant bits. Then shift left by the same N bits, and you've zeroed out all of these.
It is possible that some architectures will optimize the zeroing of an entire 8-bit byte to an even quicker operation.