If instead of exposing to the right I used the same slower shutter speed but at a lower ISO setting, I would also benefit from lower noise. My question is this: which result would give me the better image - the ETTR image at the higher speed, or the "middle"exposure at a lower ISO, using the same shutter speed and f-stop?
If the slower ISO is "pushed" during processing I can well understand this. I'll try to illustrate my original question with a concrete example. Lets say the camera's lightmeter indicated a "standard" exposure of 1/250th at f8 using ISO400. The ETTR shot might then be 1/125th at f8, also at ISO400. In ACR/Lightroom you would then back off the ETTR exposure to give a processed image with lower noise than the "standard" shot. However, how would this processed ETTR image compare to an alternative "standard" shot of 1/125th at f8, but at ISO200?
If the slower ISO is "pushed" during processing I can well understand this. I'll try to illustrate my original question with a concrete example.
Your original question was already answered. Lower ISO = more visible noise.
Hi Guillermo,So are you interpreting the OP the same as I am, which seems to be the theory that rather than slow down the shutter or open the aperture to move the data to the right to achieve EttR, you would instead increase ISO which would move the data to the right in the histogram? I'm reading the question as " if the shutter speed is as slow as you want to go, and the aperture as wide as you want to go, and your exposure looks pretty good but isn't "to the right" can using ISO to move it the right help, hurt, or be irrelevant."
Your "Lower ISO = more visible noise" answer is incomplete, it should have been followed by "assuming identical exposure parameters, which effectively means underexposing the lower ISO shot much more, compared to a normal exposure for that ISO."
Cheers,
Bart
So are you interpreting the OP the same as I am, which seems to be the theory that rather than slow down the shutter or open the aperture to move the data to the right to achieve EttR, you would instead increase ISO which would move the data to the right in the histogram? I'm reading the question as " if the shutter speed is as slow as you want to go, and the aperture as wide as you want to go, and your exposure looks pretty good but isn't "to the right" can using ISO to move it the right help, hurt, or be irrelevant."
Logic says that if the original exposure is a good one (not grossly underexposing) increasing ISO will add noise that may not be offset sufficiently by moving the data up in the histogram.
Guillermo's example seems to show otherwise, but perhaps missing the point a little since in this case both exposures will provide an adequate exposure, the higher ISO one EttR, the other one "normal". This means only a about a .6 to 1.3 stop ISO increase, not a 4 stop one like the example, where the ISO 100 one seems to be very underexposed (I haven't seen that much noise in shadows at ISO 100 in a camera in a long time).
And Guillermo's right - curious enough to try it now. I've never thought of increasing ISO to move the data to the right because as I mentioned, logically seems counter to the concept. I'll have to do some tests while out shooting tomorrow.
So are you interpreting the OP the same as I am, which seems to be the theory that rather than slow down the shutter or open the aperture to move the data to the right to achieve EttR, you would instead increase ISO which would move the data to the right in the histogram? I'm reading the question as " if the shutter speed is as slow as you want to go, and the aperture as wide as you want to go, and your exposure looks pretty good but isn't "to the right" can using ISO to move it the right help, hurt, or be irrelevant."
Logic says that if the original exposure is a good one (not grossly underexposing) increasing ISO will add noise that may not be offset sufficiently by moving the data up in the histogram.
If the slower ISO is "pushed" during processing I can well understand this. I'll try to illustrate my original question with a concrete example. Lets say the camera's lightmeter indicated a "standard" exposure of 1/250th at f8 using ISO400. The ETTR shot might then be 1/125th at f8, also at ISO400. In ACR/Lightroom you would then back off the ETTR exposure to give a processed image with lower noise than the "standard" shot. However, how would this processed ETTR image compare to an alternative "standard" shot of 1/125th at f8, but at ISO200?
Similarly for any camera with a CCD sensor, such as older generations of Nikon such as the D2x, or any MFDB such as the P65+ (I am ignoring here the pixel binning features of the more recent Phase offerings, which lower image read noise at the cost of reduced resolution). For all these cameras, ISO is essentially irrelevant; the only thing that controls image noise is the exposure. In fact, there is little reason to do anything other than peg the camera at or near base ISO, and set the largest exposure you can subject to the constraints of motion blur and needed DOF. Raising the ISO will do nothing but chop off highlight headroom.
OK, your blowing my mind here ... you're saying with my IQ180 if I shoot a shot at 1/10th at f/11 at ISO 200, and I shoot the same shot by just going to ISO 50 I won't really see any difference in the final results, even though the second one is way "under exposed"?
OK, your blowing my mind here ... you're saying with my IQ180 if I shoot a shot at 1/10th at f/11 at ISO 200, and I shoot the same shot by just going to ISO 50 I won't really see any difference in the final results, even though the second one is way "under exposed"?
really hard to wrap my mind around that .... (and curious enough to test it).
Now, it is quite possible that a raw converter looks at ISO and applies different processing. That is a reasonable approach. Raw conversion is about actual photography and not about "objective testing".
I think the relevant question is why any raw converter would look only at the ISO in metadata in order to set its default processing 'look'. The ISO (International Standards Org) specification of ISO refers to output density, not to the amount of signal amplification at some intermediate stage of processing. If a raw converter is going to change its internals according to the actual ISO (which I agree can be a reasonable approach), it should use the metadata ISO together with the setting on the exposure compensation control of the converter in order to set those defaults, since that is what sets the ballpark output density (apart from sigmoidal tone curves etc). Using only the metadata makes it harder for the user to get good results using more advanced shooting techniques that take into account the camera's noise characteristics to optimize the capture.
Q1. My in-camera meter suggests an exposure of S sec at aperture A, with my chosen ISO setting. I know from experience that this reading allows at least 1.5 - 2.0 stops of headroom before there is any danger of serious highlight clipping. I wish to expose to the right in order to minimize noise in the image. In order to ettr (push the histogram to the right) I have the option to increase the exposure (by altering either S or A) or, alternatively, to increase the ISO setting. Which of these alternatives will produce an image with the least noise?
A1. Increasing the exposure by changing either shutter speed or aperture will produce the cleaner image. Increasing the ISO will add noise to the image.
Q2. I have set my shutter speed and aperture to values dictated by the shot and which I prefer not to alter. I have set the ISO to a value such that the in-camera meter shows a "proper" exposure. My only alternative to achieve ettr (push the histogram to the right) is to increase the ISO. Will doing so improve, degrade or have no effect on the noise in the image?
A2. If you are using a camera whose read noise diminishes with increasing ISO then increasing ISO up to 1600 will lessen noise. If you are not using such a camera, there is nothing to be gained by increasing ISO. In either case, be aware that increasing the ISO comes at a cost of decreasing highlight headroom and may cause some clipping.
Q3. I know that increasing the ISO of my camera, especially above X, results in very noisy images. However the exposure parameters I have chosen require an ISO of X or more to produce a properly exposed image. Can I reduce the noise in the image by shooting at an ISO less than X and compensating for the underexposure in post-processing?
A3. See Reply #1 ;)
The first part -- increasing the exposure results in a cleaner image -- is correct. Increasing the ISO for fixed exposure will not add noise to the image, as may be seen in any of the examples I gave, increasing ISO at worst does not change the noise, and in many examples results in less noise.
Do you see that this answer is in contradiction with the your first one?
The answer is no, you cannot reduce the noise through use of a lower in-camera ISO. The noise at fixed exposure will at best be about the same. The advantage of a lower ISO, if present, is not in the arena of noise but rather in added highlight headroom.
P.S. Are you perhaps the same person as one Emil Martinec at the U. of Chicago (my alma mater, class of 1952). His (your?) paper on "Noise, Dynamic Range and Bit Depth" was most enlightening and a pleasure to read; even if partly over my head.
[...]
The answer is no, you cannot reduce the noise through use of a lower in-camera ISO. The noise at fixed exposure will at best be about the same. The advantage of a lower ISO, if present, is not in the arena of noise but rather in added highlight headroom.
Hi Bart,
What happens if you measure the raw data directly, rather than filtering it through the converter (eg by sending it through dcraw -T -4 -D -v)?
I am a bit suspicious that either (a) the number of photons is not the same between shots; or more likely (b) the converter is doing something without telling you.
the first three squares (1-3 in your counting, 19-21 on the CC chart) should be entirely dominated by photon noise, and yet your results show consistent differences (I think we need to regard 400+2EV, square number two in your chart, as an anomaly).
The Camera to Print & Screen videos demonstrate the benefits of ETTR very clearly. I have also read Michael's earlier articles on this and I understand the reason why this works. The videos also point out the obvious fact that lower ISO settings also improve S/N ratios. Now, by exposing to the right, I am also using a slower shutter speed. If instead of exposing to the right I used the same slower shutter speed but at a lower ISO setting, I would also benefit from lower noise. My question is this: which result would give me the better image - the ETTR image at the higher speed, or the "middle"exposure at a lower ISO, using the same shutter speed and f-stop?
The experiment is easy to repeat though (for someone with enough time to do it, I'm a bit swamped now), even for those without Raw analysis tools. So I invite others to try it as well.
Cheers,
Bart
I started by making a few preliminary analyses on a single exposure using different methods. The results are somewhat puzzling to me and I hope you can give me some suggestions. The exposure was of a daylight lit gray card filling the field of a 105mm lens on a D700. Exposure was F/8 @ 1/1000, which was about 2 stops below the metered value.
1. Raw file opened in ACR using default values and then opened in PSCS5. The histogram shows a mean value of 85.7 with a S.D. = 1.6
2. The raw file was converted with dcraw (-v -d -r 1 1 1 1 -T -4). When the resulting tiff file was opened in PSCS5 the histogram shows a mean = 12.36 and a S.D. = 2.9
3. Using IRIS software, the raw file was converted to a .pic version of a CFA file. The color components were separated with the command CFA2RGB and the following data were extracted. Looking only at the green channel: mean = 959; S.D. = 46.4
4. Using the IRIS software I attempted to calculate a S.D. employing the difference method. Not having a duplicate exposure, I constructed two pic files by taking a 300x300 pixel window from 2 different areas of the green channel frame. I then subtracted one window from the other. Results:
window #1 mean = 966 S.D. = 15.2
window #2 mean = 984 S.D. = 14.6
win2 - win1 S.D. = 21.2 (corrected (??) 21.2/1.4 = 15.0)
If you look at the entire color plane then there is a chance that vignetting affects the result; best to choose reasonable size patches, say 100x100 or so.Or perhaps highpass filter the entire image with a really low cutoff?
ACR conversion opened in PS will have gamma applied, as well as tone curve etc unless you have zero'd out the controls in ACR. With sRGB gamma of 2.4 that brings 985/16383 up to about 79/255, close to what you are observing. Tone curve could account for the rest. Similarly, SNR has been increased by a factor of about 2.5 relative to IRIS/Rawanalyze values, presumably also because gamma raises the value of S and compresses the N.
You can have IRIS separate the channels, then subtract one green channel from the other. Many effects such as vignetting and other sources of signal variation such as uneven lighting cancel out.
I recall reading one of your posts, some time ago, that you did just that. Can you tell me where to find that message? I was unable to find the proper instruction in IRIS to obtain the two green channels as separate files. Can you enlighten me?
The IRIS command
split_cfa c1 c2 c3 c4
will split the raw data array into four separate color channels, assuming a Bayer type pattern. You can find them as c1.fit etc in whatever directory is the default for IRIS (which you can find by pulling up the preferences dialog). You can load each color plane in turn via
load c1
and so on. Find the two channels that are closest in value; these will be the two green planes. Suppose it is c1 and c3. Load c1 and then execute the command
sub c3 1000
(here 1000 is a fixed amount that is added so that the average is not zero; make it whatever you want). IRIS can then compute average and std dev of a patch -- just drag the mouse with left click to select a window, and right click to select 'Statistics' to get the mean and std dev for the selection.
dcraw -d will give the raw data, without separating the color planes the std dev is not meaningful.
If you look at the entire color plane then there is a chance that vignetting affects the result; best to choose reasonable size patches, say 100x100 or so.
Find the two channels that are closest in value; these will be the two green planes. Suppose it is c1 and c3. Load c1 and then execute the command
sub c3 1000
(here 1000 is a fixed amount that is added so that the average is not zero; make it whatever you want).
2. The raw file was converted with dcraw (-v -d -r 1 1 1 1 -T -4). When the resulting tiff file was opened in PSCS5 the histogram shows a mean = 12.36 and a S.D. = 2.9
It indeed helps to take a smallish area from the center of the image and use an aperture of f/5.6 or f/8 as that minimizes the vignetting influence. To avoid image detail from interfering, a slight defocus can be used when shooting test images. I also try to select an area that has no hot sensels, or dust bunnies (which tend to pile up in corners if the sensor is not cleaned to perfection).
Cheers,
Bart
I assume you are referring to the method of using two identical exposures. Would you also recommend taking a window from the center of the image when one is taking the difference of the two green channels from the same exposure?
I assume you are referring to the method of using two identical exposures. Would you also recommend taking a window from the center of the image when one is taking the difference of the two green channels from the same exposure?
And thanks to all for tolerating the questions of a tyro!
I use a minimum amount of 1024 to accommodate for the blackpoint offset of Canon cameras when doing a Blackframe (read noise) analysis. When doing S/N analysis at higher ISOs I use 4000. In any case, I follow the subtraction command by a stat command and check for minimum>0 or maximum<clipping level, just to make sure that the sigma/standard deviation as reported is not based on clipped noise.
My workflow is based on pre-cropped image segments, so the stat command is adequate for obtaining the standard deviation at the same time as the boundary checking with the stat command.
Cheers,
Bart
First, I fail to understand the procedure for doing a Blackframe (read noise) analysis.
If the Canon camera adds an offset of 1024 to all data, I would think that one needs to subtract 1024 from the data; particularly the mean since this is the significant datum in regard to read noise, and the S.D. is not of any importance.
From what I have read (and perhaps misunderstood), in doing a black-frame (offset) analysis, one takes several images and computes the average of the means, or alternatively, the median value.
When doing a S/N analysis, I can understand that one wishes to exclude pixels outside the range of 0 to clipping level from influencing the S.D. However, since the S.D. (and the other statistics) are computed first and the offset added after the fact, the addition of an offset should not affect the S.D. It also eludes me how one recognizes the inclusion of clipped noise by examination of the stats.
No problem, I'll explain. A Blackframe is supposed to be black because it received no exposure. However, when we analyse it there is noise. The
Wouldn't that be a BIAS frame in standard terms?
QuoteHi Mike,
No problem, I'll explain. A Blackframe is supposed to be black because it received no exposure. However, when we analyse it there is noise. The noise is produced by the camera electronics. When we take precautions to eliminate as many noise sources (e.g. thermal noise doubles for approx. each 6 degrees Celsius rise) as possible, we could assume that the remaining noise is unavoidable and linked to the action of reading out the sensor data, hence coined "Read noise".
A Blackframe is typically produced by setting the camera to it's shortest possible exposure time (to counteract thermal noise build-up), using a body cap instead of a lens (to avoid light leaks, electronic noise from the lens, and camera gain adjustments at certain apertures), and covering the eyepiece of the viewfinder (to avoid light leaking into the mirrorbox though the back).
The signal that is still recorded is the lowest signal possible and is usually random with a Gaussian distribution. It changes with the ISO (gain) setting. It is not the same as a Darkframe, which is produced with a much longer (>1 sec. typically) exposure time, as used for Darkframe subtraction. By comparing a Darkframe and a Blackframe one can quantify the (mostly thermal) contribution.
Thank you. I was aware of the difference between a Black-frame (called an Offset by the Astrophotophotographer, I believe) and a Darkframe. I am interested in the Black-frame because I wish to use it to estimate the influence of ISO on the read noise of my camera.QuoteThe offset in most Canons cameras is part of the ADC quantization, so it is not added afterwards. That's why the noise has a Gaussian distribution centered at (usually) ADU 1024. There are also values below 1024 because of the Readnoise.
Still scratching my head here. By afterwards I meant after (or even simultaneous with) conversion of the (amplified) signal from the sensor into ADU. Thus 1024 will be added to whatever read noise is associated with a given pixel. And I assume that such read noise cannot take negative values. So I fail to see how one can get values less than 1024, nor how the mean of the read noise would be at 1024 (if you are talking about values obtained from a single image).QuoteWhat you describe is a Darkframe (not Blackframe) noise reduction technique, commonly used in astrophotography where long exposure times are needed to collect enough photons to record faint signals. This is also why Canon cameras are often used in astrophotography, because the Readnoise improves predictably with averaging multiple frames and may reveal faint signals.
From my brief reading in the AstroP sites, the same method (i.e. average mean or median of several images) is recommended for determining a Offset (or what we are calling a Blackframe). How do you recommend obtaining Blackframe data? I assume it does not involve taking the difference between two images, as one would do for S/N analysis.Quote... When you subtract 2 noisy data sets with a mean value of e.g. 1024, then there is a 50% chance that an image has a value of 1024 or less. There is a equal chance of it being 1024 or higher. When we subtract an image with a higher data value from one with a lower data value we would get a negative number, which cannot be encoded in an integer number calculation, and thus result in a clipped noise distribution.
Therefore we add an offset to both datasets, which only changes the mean value but not the SD around that mean, and the result of the subtraction can be statistically evaluated. My choice of 1024 is not a must, one can use any number that doesn't add to the risk of integer value clipping, although it could also indicate an ADC problem. That's why I use the IRIS stat command after the subtraction, to check that there are no values that resulted in (probably clipped) zero despite the offset. If it would have a minimum of zero, then I redo the subtraction with a higher offset (for light exposure frames), but for Blackframes this is usually not needed (especially for lower ISO gain settings).
I am aware that when one subtracts two data sets, regardless of their respective means, there is an equal chance that any given pixel in image #1 will have a value greater than or less than the same pixel in image #2. Thus the possiblity of negative values. However I was under the impression that the Iris software can deal with negative numbers when calculating a mean and SD from individual pixel differences. Furthermore I believe that the offset is added after the calculation of these statistics is performed. I have tried this with two images, both with and without adding an offset. First subtracting image #2 from image #1 and then the reverse. The mean, median and SD (after subtracting the offset) were identical in value but opposite in sign.QuoteHope that helped to understand the chosen procedure.
Cheers,
Bart
It has helped, but still a ways to go
Cheers/Mike
So next question: How does one estimate read noise based on black-frame images from a Nikon D700, which apparently does not apply a bias voltage.
Cheers/Mike
For the Nikon D3 and other cameras not using an offset, one must measure the read noise by extrapolation as shown by Peter Facey (http://www.brisk.org.uk/photog/d3readn.html). He also gives a link to similar study by Emil Martinec.
Regards,
Bill
So next question: How does one estimate read noise based on black-frame images from a Nikon D700, which apparently does not apply a bias voltage.
Cheers/Mike
Another option: Nikon has an area of 'masked pixels' on the sensor that are shielded from light, and for which a bias voltage is applied; apparently these pixel values are written to the raw file. The package 'libraw' can output these pixel values:
http://www.libraw.org/docs/API-datastruct-eng.html#libraw_masked_t
The first part -- increasing the exposure results in a cleaner image -- is correct. Increasing the ISO for fixed exposure will not add noise to the image, as may be seen in any of the examples I gave, increasing ISO at worst does not change the noise, and in many examples results in less noise.
So here is where I’m unclear about what’s going on under the hood. The exposure is identical in terms of aperture and shutter. ISO is higher. We expect doing so would make it appear brighter. What I would like explained further (for the non scientist) is how and why? The same amount of light (photons) strike the sensor. What is the ISO doing here to provide a better S/N ratio reducing the noise?Perhaps a significant source of noise is located after the analog amplifier, meaning that boosting the signal early on will not affect the additive noise that is added later on, resulting in an improved SNR?
So here is where I’m unclear about what’s going on under the hood. The exposure is identical in terms of aperture and shutter. ISO is higher. We expect doing so would make it appear brighter. What I would like explained further (for the non scientist) is how and why? The same amount of light (photons) strike the sensor. What is the ISO doing here to provide a better S/N ratio reducing the noise?
ISO amplification by itself doesn't alter the SNR, so the ISO amplification does't improve output SNR with respect to Npre. However with Npost the story changes, and the higher the ISO amplification is, we have higher useful signal vs Npost, so we are improving final SNR with respect to Npost.
Excellent, thanks! It explains perfectly why it was mentioned that differing cameras may or may not show the results of the test images you provided (and I saw as well on my Canon).
Yes, see this Pentax K5 test where ISO1600 nearly didn't improve noise vs ISO100 at the same aperture/shutter, so it was best to stay at ISO100 which can be useful to prevent highlight clipping:But does the AE modes optimally? If you set such a camera in P/Tv/Av, stick ISO to 100 and take an image, will it do e.g. 1/100 sec and underexpose badly (something that, according to your image can be easily fixed in pp), or will it bump the exposure time to 1/10 sec, resulting in a blurred image that cannot be salvaged?
But does the AE modes optimally? If you set such a camera in P/Tv/Av, stick ISO to 100 and take an image, will it do e.g. 1/100 sec and underexpose badly (something that, according to your image can be easily fixed in pp), or will it bump the exposure time to 1/10 sec, resulting in a blurred image that cannot be salvaged?
Further, if you set it to manual but keep the auto ISO, will it embed a tag saying "please multiply this image by 10 in raw development", or will it multiply the image digitally in-camera, resulting in any highlights being unnecessarily being blown out?
In the real world, Npre would be basically the photon noise (inherent to light capture) plus the read noise (electronic noise) produced in the early stages, prior to ISO amplification. Npost would be the read noise produced after the ISO amplification, i.e. in the AD converter.
Cameras with a very low read noise (and this consequently means very low Npost) hardly benefit from pushing ISO
(this image must have been linked in LL more than 10 times ;D ).
Regards
These two statements seem somewhat contradictory, unless the second sentence refers only to the read noise produced post amplification. What am I missing?