Pages: 1 [2] 3   Go Down

Author Topic: Connecting Exposure, ISO and Noise  (Read 13756 times)

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Connecting Exposure, ISO and Noise
« Reply #20 on: February 28, 2009, 08:07:19 pm »

Quote from: ejmartin
Here's my test:  1D3 at ISO 100, 100% crop

and at ISO 1600 with the same Tv/Av:
Somehow I have the feeling that your demo has nothing to do with what Marc posted. If I understand you correctly, you are comparing a 4EV underexposed shot @ ISO 100 with one using the same exposure but ISO 1600.

Marc compared two properly exposed shots, one @ ISO 100, the other @ 1600.
Logged
Gabor

marcmccalmont

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1780
Connecting Exposure, ISO and Noise
« Reply #21 on: February 28, 2009, 08:16:45 pm »

quote name='Panopeeper' date='Feb 28 2009, 03:07 PM' post='263679']
Somehow I have the feeling that your demo has nothing to do with what Marc posted. If I understand you correctly, you are comparing a 4EV underexposed shot @ ISO 100 with one using the same exposure but ISO 1600.

Marc compared two properly exposed shots, one @ ISO 100, the other @ 1600.
[/quote]

True but same result!
I thought that noise built up over time would exceed the noise in the analog amplifier at some point, I guess not. This was with dark  frame noise reduction on, to make it a good real world test.
Marc

[attachment=11820:IMG_1601_DxO_raw.jpg][attachment=11819:IMG_1605_DxO_raw.jpg][
« Last Edit: February 28, 2009, 08:23:22 pm by marcmccalmont »
Logged
Marc McCalmont

dalethorn

  • Guest
Connecting Exposure, ISO and Noise
« Reply #22 on: February 28, 2009, 09:12:48 pm »

Quote from: scubarob639
The point you make I understand, however the phrase "All men are created equal" is not in our constitution, and a women's right to vote is not an unalienable right as defined by our Declaration of Independence.

Rob

When the DOI said "all men" was it really referring to all persons, or just to males?  You'd think that the founding fathers, some of the most astute and educated men in history, would have the same understanding of that issue.  From what I've read, they had the same understanding of each other's biases, and that's what they best understood.  Capt. Kirk said "no man" - Capt. Picard said "no one".  So, if we had several alternate or independently-produced versions of the DOI at hand, we just might find a Kirk/Picard dialectic there.  Don't you suppose people are short-changing themselves by reserving their utmost respect for only the "official" version of history?

None of this is intended to be political speech - just an illustration of understanding.  Today, while photographing a Golden Tamarind at the zoo, I got a few decent exposures when the critter was in good light, and a few under-exposures when it moved away from the good light.  Not surprisingly, the under-exposures were much noisier than the good exposures, although all of them were taken at ISO 400.  But that's been my experience with any camera.
Logged

Fine_Art

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1172
Connecting Exposure, ISO and Noise
« Reply #23 on: February 28, 2009, 10:00:23 pm »

Quote from: ejmartin
Yes on #1 -- minimum f-stop for required DOF.
On #2, the slowest exposure time in seconds that can freeze the action etc.

I'll add that around the exposure time the same as shutter/mirror vibration time, blur is created. That duration should be avoided.
« Last Edit: February 28, 2009, 10:01:40 pm by Fine_Art »
Logged

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
Connecting Exposure, ISO and Noise
« Reply #24 on: March 01, 2009, 12:56:07 am »

Quote from: marcmccalmont
quote name='Panopeeper' date='Feb 28 2009, 03:07 PM' post='263679']
Somehow I have the feeling that your demo has nothing to do with what Marc posted. If I understand you correctly, you are comparing a 4EV underexposed shot @ ISO 100 with one using the same exposure but ISO 1600.

Marc compared two properly exposed shots, one @ ISO 100, the other @ 1600.


True but same result!
I thought that noise built up over time would exceed the noise in the analog amplifier at some point, I guess not. This was with dark  frame noise reduction on, to make it a good real world test.
Marc

We didn't read each other's posts carefully enough.  I didn't notice that you were changing the exposure in your test, and you didn't interpret my rule of thumb correctly.  The rule of thumb was

1) Choose the minimum f-stop for the required depth of field.
2) Choose the minimum exposure time required to freeze the action, prevent camera shake, etc.
3) Choose the ISO such that the histogram is pushed to the right as much as possible while keeping the highlights that you want to preserve unclipped.

If you were exposing at 1 min and ISO 100, and got a good histogram and satisfied conditions 1 and 2, then a second exposure at 4 sec and ISO 1600 violates #2 -- since you were able to get a good image at 1 min, that is an lower bound to the exposure time, and 4 sec is faster than optimum.
« Last Edit: March 01, 2009, 01:02:24 am by ejmartin »
Logged
emil

marcmccalmont

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1780
Connecting Exposure, ISO and Noise
« Reply #25 on: March 01, 2009, 01:53:23 am »

Quote from: ejmartin
We didn't read each other's posts carefully enough.  I didn't notice that you were changing the exposure in your test, and you didn't interpret my rule of thumb correctly.  The rule of thumb was

1) Choose the minimum f-stop for the required depth of field.
2) Choose the minimum exposure time required to freeze the action, prevent camera shake, etc.
3) Choose the ISO such that the histogram is pushed to the right as much as possible while keeping the highlights that you want to preserve unclipped.

If you were exposing at 1 min and ISO 100, and got a good histogram and satisfied conditions 1 and 2, then a second exposure at 4 sec and ISO 1600 violates #2 -- since you were able to get a good image at 1 min, that is an lower bound to the exposure time, and 4 sec is faster than optimum.

It was a test, I'm agreeing with you!
Marc
« Last Edit: March 01, 2009, 01:54:09 am by marcmccalmont »
Logged
Marc McCalmont

Dale_Cotton2

  • Full Member
  • ***
  • Offline Offline
  • Posts: 109
    • http://daystarvisions.com
Connecting Exposure, ISO and Noise
« Reply #26 on: March 01, 2009, 07:44:45 am »

Gabor, Guillermo, Emil (the Three Tensors?), et al.: for me the digital camera is an expressive artistic tool. To fully exploit it I need to understand how to use it; to understand how to use it I need to understand at least the rudiments of what's going on under the hood. For this reason I can't thank you three enough for your contributions to the on-line community. IMHO, the posts here from you three are one of the very few reasons this forum shouldn't be shut down by the absurdity police. ;) I also very much appreciate you're maintaining a pH-neutral tone ... which stands in marked contrast to the endless acidity of the prima donna set.
Logged

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
Connecting Exposure, ISO and Noise
« Reply #27 on: March 01, 2009, 08:48:51 pm »

Well I guess I came late to the discussion. After the email I got from Gabor on this topic, hadn't realized it was started over here at LL.

One thing that surprised me about Gabor's fascinating article was the fact that not only did noise crop up regardless of ISO, but different color patches exhibited varied amounts of noise according to their color whether low or high ISO.

Very informative article, Gabor. A bit hard to follow and keep organized in my head on what was affecting what because of the varying change ups between different color patches and ISO setting parameters and all the scrolling back and forth on my 20' wide format LCD I had to do, but I finally figured it out.

I experienced pretty much what you illustrated in a shot taken of a white goose in somewhat dark water in blazing bright mid-afternoon sunlight with my Pentax K100D. Since I stood the chance of loosing the shot shooting manual, since the goose was swimming away, which would've forced me to chimp the histogram adjusting exposure to prevent blowouts, I let the camera's metering choose with Av using Center Weighted Averaging and of course it chose to severely under expose the white goose where the water turned to black when viewed in ACR. It was shot at the camera's base ISO of 200, f/6.3, 1/1600s.

When I lightened the image in ACR so I could see the water, there was quite a bit of noise both chroma and luminous throughout the water going into the regions of the shadows of the white goose, but not enough that couldn't be fixed in ACR. There was plenty of light during the shoot and if I had shot manual I could've ETTR'ed maybe by as much as 1/3 to 2/3 stop increased exposure, but from my experience from shots like this with my Pentax, the dynamic range of that scene on my camera's sensor wouldn't have allowed ETTR to reduce the noise in the water or make it look lighter.

But I did get a nice shot of that goose.

Dale,

What do you mean by implying the Luminous Landscape forum shouldn't be shut down? Huh?! Are their talks of this being considered? Hope not.
« Last Edit: March 01, 2009, 08:51:48 pm by tlooknbill »
Logged

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
Connecting Exposure, ISO and Noise
« Reply #28 on: March 01, 2009, 09:02:05 pm »

Gabor,

Something I considered as an afterthought derived from my goose shooting illustration above.

Have you tested for noise levels adjusting exposures using too much light where you have to underexpose to accomodate a dynamic range that happens to be too large for the camera's sensor?
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Connecting Exposure, ISO and Noise
« Reply #29 on: March 02, 2009, 11:15:27 am »

Thanks you all for the critique as well as for the appreciation. I will rework the entire paper. >I admit I have a big problem with assessing how the "average" reader understands the underlying issues. Most critique I received (in emails as well) suggested, that it is too steep, but I received the opposite as well.

I guess I will put the *conclusions* to the front, and then try to prove them; those, who are not interested on the technical details, do not need to go through them. Then I'll try to explain the technicalities in more detail.

Quote
Have you tested for noise levels adjusting exposures using too much light where you have to underexpose to accomodate a dynamic range that happens to be too large for the camera's sensor?
I don't understand the question. The noise does not depend on how and why you arrived at a certain exposure. Furthermore, it does not depend on the exposure per se, but on the illumination of the selected area (of course that does depend on the exposure). In other words. you may have overexposed the shot hopelessly and still have high-noise areas in it. As long as sensor blooming does not play a role, the overexposed and underexposed areas have nothing to do with each other.
Logged
Gabor

marcmccalmont

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1780
Connecting Exposure, ISO and Noise
« Reply #30 on: March 02, 2009, 02:07:21 pm »

I once read a math (or was it an engineering) book that had the main body on the right side pages and the detailed explanations (deep math) on opposing left side pages. I always thought that was a brilliant presentation when in printed book form. Perhaps there is a similar way for web presentation?
Marc
Logged
Marc McCalmont

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
Connecting Exposure, ISO and Noise
« Reply #31 on: March 02, 2009, 03:55:26 pm »

Panopeeper,

Then why did I get noise in the water and into the shadows on the goose when there was a ton of light? It was broad daylight? You couldn't get any more light than what was in this scene.

It was just underexposed caused by the camera's metering. It ended up being a very dark image with a histogram as scene in ACR similar to a linear file. I've shot similar scenes shooting manual with proper exposure and didn't get noise in dark water.

Could this be caused by shooting a scene whose dynamic range is greater than the sensor's capability? IOW can dynamic range mismatch cause noise levels to increase.
« Last Edit: March 02, 2009, 03:57:04 pm by tlooknbill »
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Connecting Exposure, ISO and Noise
« Reply #32 on: March 03, 2009, 02:34:33 pm »

Quote from: tlooknbill
Then why did I get noise in the water and into the shadows on the goose when there was a ton of light? It was broad daylight? You couldn't get any more light than what was in this scene.
Why don't you just upload your goose in raw format, so that we can see specifics, instead of speculating?
Logged
Gabor

fike

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1413
  • Hiker Photographer
    • trailpixie.net
Connecting Exposure, ISO and Noise
« Reply #33 on: March 03, 2009, 03:14:58 pm »

In some ways, this demonstration is pointing out the obvious, but in a very sophisticated way that had never occurred to me.  

It never really occurred to me to equate an underexposed image that is pushed (in Photoshop, for example) to make a 'good' exposure and a high-ISO image under the same conditions.  If I understand this correctly an underexposed image that is pushed to be lighter in Photoshop will have similar noise attributes as a high-ISO image with the same resulting exposure range.    The only difference is what is doing the gain increase--the camera or Photoshop.  The limiting factor remains the number of photons that are read at the sensor, whether post-processing does the gain increase or the camera does it with an ISO, it really doesn't matter.  Higher ISOs provide a more agressive gain table, in a way.  Eh??

Do I have that right?

Now that I think I have a more sophisticated understanding of this, I need to figure out how this changes my work in the field and at the computer.  I am having trouble figuring out how this can help me in practice.  

All I can think of is that it reinforces the well-known lesson that I need to find ways to get more light to the sensor, whether with longer exposures or wider apertures.

Can I draw conclusions about how to do better noise reduction?  All this seems to do on that front is to reinforce the fact that noise reduction reduces and smears detail, because that detail wasn't all that certain to begin with.
Logged
Fike, Trailpixie, or Marc Shaffer

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Connecting Exposure, ISO and Noise
« Reply #34 on: March 03, 2009, 04:55:04 pm »

Quote from: fike
It never really occurred to me to equate an underexposed image that is pushed (in Photoshop, for example) to make a 'good' exposure and a high-ISO image under the same conditions.  If I understand this correctly an underexposed image that is pushed to be lighter in Photoshop will have similar noise attributes as a high-ISO image with the same resulting exposure range.    The only difference is what is doing the gain increase--the camera or Photoshop.  The limiting factor remains the number of photons that are read at the sensor, whether post-processing does the gain increase or the camera does it with an ISO, it really doesn't matter.  Higher ISOs provide a more agressive gain table, in a way.  Eh??

Do I have that right?
Yes or no, depending on the actual camera and ISO.

Emil's advice above is very good, as generous consideration. I find it generally usefult to explain something from different angle of view for a better understanding. I use now the Canon 5DMkII for the explanation.

Down below are the noise graphs at the different *real* ISOs, i.e. ignoring those, which are only numerical derivatives, as they are never useful for the raw data. The horizontal lines represent a given level of noise; the intersection with the graph of an ISO value shows, how great the dynamic range can be with that level of noise. The difference in EV between two ISO settings at a given level of noise shows, how much you gain (or lose) when selecting this or that ISO.

For example if you say you are accepting up to 40% noise (see the examples for how 40% looks), then see that it intersects the ISO 3200 graph at -7.5 EV and the ISO 1600 graph at -8.5 EV. The difference between these is one full stop (like with a fake ISO). This means, that if you choose ISO 3200 over 1600 (and use one stop lower exposure), then you lose exactly one stop of the dynamic range. In other words, if you reduce the exposure but leave the ISO at 1600 (underexpose ISO 1600 by 1 EV), then the shadows will look equally good - or equally bad - but you don't run the risk of clipping one stop more highlights than with 1600 (because the increased ISO cuts off the top EV of the highlights, independently of the effect on the shadows).

In cleartext: don't use ISO 3200 and above with the 5D2, except if the embedded JPEG (the preview) is important, or if you are recording raw + JPEG.

Next step: ISO 1600 vs 800. The difference between those is roughly 0.7 EV, i.e. that is, what you lose when shooting with ISO 1600 vs. ISO 800 (where ISO 800 would be shot with one stop higher exposure). Turn it around: with ISO 1600 you expose by one stop less, but lose only 0.7 EV. The economics is not great, but if you can't efford the higher exposure and the potential of highlight loss is not a concern, then choose ISO 1600 over the underexposed ISO 800.

Now, ISO 800 vs 400: the difference is 0.5 EV. That's not a bad tradeoff for one stop lower exposure.

ISO 400 vs 200: roughly 0.25 EV. This means, that if the DR of the scenery is not very large, you can reduce the exposure practically without penalty.

ISO 200 vs 100: the difference is tiny. Use ISO 100 only if you really want to max out the dynamic range.

ISO 100 vs 50: the graphs conform, what we know already: there is no difference between these; ISO 50 is simply an overexposed ISO 100, there is no reason to use it; in fact, there is good reason not to use it: avoiding overexposure.

Now, all the above was for the case that you are not free to choose the exposure. If you can efford that, then use the lowest ISO (but not the overexposing one).

Note, that the differences between the ISO steps are the same with all the relevant noise levels with this camera and many others, i.e. it does not matter which noise level you accept, the considerations remain the same.

Logged
Gabor

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
Connecting Exposure, ISO and Noise
« Reply #35 on: March 03, 2009, 07:39:45 pm »

Quote from: fike
In some ways, this demonstration is pointing out the obvious, but in a very sophisticated way that had never occurred to me.  

It never really occurred to me to equate an underexposed image that is pushed (in Photoshop, for example) to make a 'good' exposure and a high-ISO image under the same conditions.  If I understand this correctly an underexposed image that is pushed to be lighter in Photoshop will have similar noise attributes as a high-ISO image with the same resulting exposure range.    The only difference is what is doing the gain increase--the camera or Photoshop.  The limiting factor remains the number of photons that are read at the sensor, whether post-processing does the gain increase or the camera does it with an ISO, it really doesn't matter.  Higher ISOs provide a more agressive gain table, in a way.  Eh??

Do I have that right?

Yes gain can be done either in the camera hardware, in the camera firmware (as is the case with Hi ISO extensions in Canons, where ISO boost beyond the highest standard ISO is done digitally by camera firmware), or in the RAW converter by EV compensation.  While it is true that it doesn't matter for the photons captured, they are always the same, it does matter for the noise.  Every electronic component adds a bit of noise to the signal.  There are two possibilities:

1) If that noise is added prior to amplification, then it gets amplified along with the signal, and so S/N stays the same;
2) If it is added after amplification then increasing the amplification increases the S/N since S is boosted but N is not.  

When using analog gain in the camera via a choice of ISO, one gains an advantage because all the noises from electronics that process the signal after amplification (such as the amplifier itself and the A/D converter) come post-amplification and are of type 2.  If one uses a lower ISO and amplifies in the RAW converter by applying EV compensation, then all noises are of type 1) for this amplification, and no S/N benefit accrues.

So a major lesson to take away is that any ISO that is a digital manipulation of a lower ISO by the camera firmware, provides no benefit in S/N while at the same time losing extra stops of highlights because image areas that are not blown out after the analog amplification can be made so by the additonal amplification applied in the firmware.

A cautionary note -- it seems that some converters (ACR seems to be one) the exposure compensation is not done all that accurately, and shadow detail is lost.  See a thread over at DPR running at the moment:

http://forums.dpreview.com/forums/read.asp...essage=31188392

This is not a problem with the RAW file or the principle of pushing an underexposure for ultra-high ISO, it's an issue of the converter.


Quote
Now that I think I have a more sophisticated understanding of this, I need to figure out how this changes my work in the field and at the computer.  I am having trouble figuring out how this can help me in practice.  

All I can think of is that it reinforces the well-known lesson that I need to find ways to get more light to the sensor, whether with longer exposures or wider apertures.

Can I draw conclusions about how to do better noise reduction?  All this seems to do on that front is to reinforce the fact that noise reduction reduces and smears detail, because that detail wasn't all that certain to begin with.

Yes, it's always best to get as many photons as possible.

Noise reduction is a separate discussion regarding the art of post-processing.  One point though -- different RAW converters do better or worse with regard to noise.  It is worth experimenting to see which ones give the best starting point for further processing.


Logged
emil

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
Connecting Exposure, ISO and Noise
« Reply #36 on: March 04, 2009, 01:37:03 am »

Panopeeper,

Here's the link to the goose shot. It is in its original Pentax PEF format. Let me know if you need it converted to DNG if you find you can't open it.

http://www.yousendit.com/download/U0d6a3ZFNXZBNkZjR0E9PQ

Just curious do you find what you understand about noise level behavior to be the same among other brands of camera's or are you only basing your studies and research on the Canon 5D Mark II?

Let me know what you find. And thank you for your time and insightful information.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Connecting Exposure, ISO and Noise
« Reply #37 on: March 04, 2009, 11:52:13 am »

Quote from: tlooknbill
Here's the link to the goose shot. It is in its original Pentax PEF format. Let me know if you need it converted to DNG if you find you can't open it
(No problem with the PEF)

1. The exposure was quite good; you could have exposed higher only 1/3 EV without clipping, see the raw histogram. With +2/3 EV a few spots would have clipped, see the second attachment (magenta indicates green clipping, red shows that the blue too would have clipped).

2. The third attachment is only for orientation, to show where I made the selection.

3. The fourth attachment shows the noise: the red channel was below the eightth stop. You see the vertical strips? That light ewas not enough for the camera. (In cleartext: the DR of the camera is about 8 stops without such noise).

Had you had made the shot with ISO 100, the noise would have been much less.
 
Quote
Just curious do you find what you understand about noise level behavior to be the same among other brands of camera's or are you only basing your studies and research on the Canon 5D Mark II?
The "effectivity" of the ISO steps is highly model dependent.

Logged
Gabor

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
Connecting Exposure, ISO and Noise
« Reply #38 on: March 04, 2009, 03:02:09 pm »

Thanks for the analysis, Gabor.

So the dynamic range of the scene was somewhat beyond the capabilities of my camera. Wish I could've shot at ISO 100 but the Pentax K100D only goes to 200. So noise levels are different between models and brands of camera's based on shooting the same scene.

The Canon 5D Mark II would not show noise shooting a similar scene, correct?

Thanks for your time.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Connecting Exposure, ISO and Noise
« Reply #39 on: March 04, 2009, 07:19:46 pm »

Quote from: tlooknbill
So the dynamic range of the scene was somewhat beyond the capabilities of my camera
That's right. I did not know the K100D does not go lower than ISO 100.

In such case you have no choice but to sacrify something: some bright white spots of the feather, or the darkest areas of the water. I think your shot is good so. It could have been increased by 1/3 EV without compromising the highlights, but that is practically impossible without bracketing. In critial cases I make several shots; I use a "neutral" setting, thus the histogram and the clipping indication show very reliably how exosure *was*, but it does not make the exposure correct, so I have to "circle on". Back home I look at the shots with Rawnalyze, if there was clipping then I see what and how much, look at the noise and select the best version.

Quote
The Canon 5D Mark II would not show noise shooting a similar scene, correct?
Here are two samples (smooth, uniform areas) with ISO 100 and 200, from the same intensity as the area I selected in your shot: around -8.6 EV. Although the 5D2 does show strong pattern noise (like all Canons do, on account of the rather "raw" raw data), that appears more than one stop later.

The next two captures show the noise with ISO 100 at -9.84 EV and -11 EV.
Logged
Gabor
Pages: 1 [2] 3   Go Up