Pages: [1]   Go Down

Author Topic: Long Exposure Hot Pixels: D700 & 5D MKII  (Read 13597 times)

scottp

  • Newbie
  • *
  • Offline Offline
  • Posts: 2
Long Exposure Hot Pixels: D700 & 5D MKII
« on: May 22, 2009, 12:37:03 pm »

I'm currently looking at the Nikon D700 and the Canon 5D MKII as my first serious digital camera purchase (as a "replacement" for my now sold Pentax 645 Med. Format film cam).

Since I do a fair amount of night-time photography using exposures from a few seconds to several minutes, I was concerned when I read this in Thom Hogan's review of the D700:

"The problem with the D700 is that it will absolutely start producing very visible colored hot pixels on long exposures unless you turn on Long Exp NR."

Does anyone have some insights from experience with night-time, long exposure images on either the D700 or the 5d MarkII?

Does the D700's "Long Exp NR" apply to RAW images and is there any negative side-effects from using this NR?

Thanks for your insights.
Logged

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #1 on: May 22, 2009, 01:04:26 pm »

Quote from: scottp
Does the D700's "Long Exp NR" apply to RAW images and is there any negative side-effects from using this NR?

Yes, and no. It's using a blank exposure (shutter staying closed) to subtract fixed-pattern noise from images. There's really no reason not to do it, other than additional time for the blank exposure during shooting.
Logged

Geoff Wittig

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1023
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #2 on: May 23, 2009, 06:34:59 am »

Quote from: Jonathan Wienke
Yes, and no. It's using a blank exposure (shutter staying closed) to subtract fixed-pattern noise from images. There's really no reason not to do it, other than additional time for the blank exposure during shooting.

"It depends".
Using long exposure noise reduction has two significant drawbacks. It exactly doubles your exposure time, because the camera captures a dark frame immediately upon closing the shutter; so your 30 second exposure is actually 60 seconds. This can be pretty frustrating if you're shooting something like nighttime lightning or the Aurora borealis; I find myself gnashing my teeth waiting for the dark frame exposure to finish as new subject matter taunts me. Second problem is that you're burning through battery life twice as fast. In cold weather, very long shutter speeds suck the life right out of your battery. Long exposure noise reduction doubles your power consumption per exposure, so your battery life dwindles that much quicker.

Okay, and there's a third issue. Whenever I use this I forget to turn it off. So a couple days later I'll pull the camera out to shoot some wildlife photos, and I'll miss a fantastic shot while the friggin' camera is taking its time exposing a dark frame. Doh!
Logged

feppe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2906
  • Oh this shows up in here!
    • Harri Jahkola Photography
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #3 on: May 23, 2009, 06:48:47 am »

Quote from: Jonathan Wienke
Yes, and no. It's using a blank exposure (shutter staying closed) to subtract fixed-pattern noise from images. There's really no reason not to do it, other than additional time for the blank exposure during shooting.

Dark frame is a non-solution for me, like it is for Geoff.

Does this mean 5D MkII requires dark frame to produce acceptable results?

edit: according to this brief test hot pixels don't appear to be a problem until exposures reach 8+ minutes with 5D MkII - and even then it is largely an academic issue since they're too small to show up in print.
« Last Edit: May 23, 2009, 06:54:29 am by feppe »
Logged

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #4 on: May 23, 2009, 07:28:16 am »

Quote from: Geoff Wittig
Okay, and there's a third issue. Whenever I use this I forget to turn it off. So a couple days later I'll pull the camera out to shoot some wildlife photos, and I'll miss a fantastic shot while the friggin' camera is taking its time exposing a dark frame. Doh!

I seriously doubt this. Most cameras automatically turn dark frame subtraction off when the exposure time is less than 1 second or so, and with typical wildlife shutter speeds (faster than 1/20) the dark frame exposure wouldn't slow down shooting even if you're machine-gunning at 8FPS. And with the 1Ds and 1D-II (and I assume newer 1-series bodies as well), you can defer the dark frame exposure delay by pressing the shutter release again while the dark frame exposure is happening. You can shoot until the buffer is full, then the camera will force you to stop shooting while it does the dark frame shot and subtracts it from all the frames in the buffer.
« Last Edit: May 23, 2009, 07:32:29 am by Jonathan Wienke »
Logged

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #5 on: May 23, 2009, 08:58:43 am »

There is another solution, which is to shoot normally, and at some point in your session shoot some frames with the lens cap on and the viewfinder covered, at the same exposure times/ISO's that you used for the images (if there are several Tv's/ISO's, you will need black frames for each).   If it's a long shooting session and temperatures are varying, you may want reference black frames at the beginning and end of the session.  You will be restricted to using a RAW converter that can then import both the image file and the blackframe file and do the subtraction; dcraw and some of the freeware GUI utilities based on it have this capability; I don't think any of the standard commercial ones do.  The advantage is that one only has to shoot a small number of blackframes for the whole session, rather than wait for the camera to do a separate blackframe for each image.  The disadvantage is that you can't use the RAW converter of your choice.  I am hopeful that perfectRAW (currently in a pre-alpha state, hopefully becoming available soon) will have this capability.
Logged
emil

free1000

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 465
    • http://www.foliobook.mobi
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #6 on: May 23, 2009, 11:03:23 am »

Quote from: feppe
Dark frame is a non-solution for me, like it is for Geoff.

Does this mean 5D MkII requires dark frame to produce acceptable results?

edit: according to this brief test hot pixels don't appear to be a problem until exposures reach 8+ minutes with 5D MkII - and even then it is largely an academic issue since they're too small to show up in print.

The 5D2 allows you to turn off long exposure noise reduction (dark frame) with Custom Function II-I, though I have only shot long with it turned on so far.

I imagine that if there was not a benefit, then manufacturers wouldn't add this feature. ie: If an exposure at 1 stop less, exposed for the same time (double the main exposure is the same as main exposure+dark frame exposure) was not more noisy, there would be no purpose in introducing dark frames. So if you are doing digital, its probably an advantage not a negative. Try using a medium format back and you will find that dark frame subtraction is essential rather than being optional.

I need to do some testing, if I can stay awake long enough I might do some tonight. Moon is in last quarter so its pretty dark right now, so a good time to go for some star trails maybe.
 
So far I have shot at  800 ISO, 30 sec exposures on the 5DII with dark frame reduction.

http://bit.ly/ft6Sd

I ran it through Imagenomic to control the noise.  I have yet to do tests at ISO 200 and 400 with longer exposure times.  With all this in mind, I'm still shooting more rapidly than I would have been on film, with less noise in the final results. And with the knowledge that if I've travelled 4000 miles for a one night chance to get the shot that its in the can.


« Last Edit: May 23, 2009, 11:10:23 am by free1000 »
Logged
@foliobook
Foliobook professional photo

feppe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2906
  • Oh this shows up in here!
    • Harri Jahkola Photography
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #7 on: May 23, 2009, 02:55:22 pm »

Quote from: free1000
I imagine that if there was not a benefit, then manufacturers wouldn't add this feature. ie: If an exposure at 1 stop less, exposed for the same time (double the main exposure is the same as main exposure+dark frame exposure) was not more noisy, there would be no purpose in introducing dark frames. So if you are doing digital, its probably an advantage not a negative. Try using a medium format back and you will find that dark frame subtraction is essential rather than being optional.

And that's exactly why I'm not even considering MFDBs, even if I could afford one.

You give too much credit to Canon: you're talking about the same people who insisted on having a dedicated direct print button on their prosumer cameras for generations  And don't get me started on MLU - only in the latest generation or two did they make it easily accessible through customizable menu.

I have pretty extensive experience in nighttime photography with 30D and 450D, and have never used dark frame reduction. In my experience it's good for pixel-peepers, but unnecessary for most real-world situations. On the tough noise-critical shots with massive DR (think nighttime cityscapes) I bracket to get rid of noise, which has the added benefit of actually producing new data for Tufusing, instead of just getting rid of bad data.

The issue the OP and I was concerned was whether 5D MkII requires dark frame to function properly in low light. Apparently it's not needed until very long exposures, although I found only one test by quick googling. Therefore this would be a non-issue for me, and I probably wouldn't see a need to use it - which is to be expected from a semi-pro camera.

edit: non-issue for me because the longest exposures I routinely do are 30 secs, and only rarely go to 180 secs.
« Last Edit: May 23, 2009, 02:56:59 pm by feppe »
Logged

free1000

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 465
    • http://www.foliobook.mobi
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #8 on: May 24, 2009, 07:22:51 am »

Quote
In my experience it's good for pixel-peepers, but unnecessary for most real-world situations. On the tough noise-critical shots with massive DR (think nighttime cityscapes) I bracket to get rid of noise, which has the added benefit of actually producing new data for Tufusing, instead of just getting rid of bad data.

Can you explain? What do you mean by a real world situation rather than pixel peeping?  

With the bracketing, are you talking about 3 brackets and using Tufuse, or are more brackets required.  

Maybe I should look at that approach.
« Last Edit: May 24, 2009, 07:25:36 am by free1000 »
Logged
@foliobook
Foliobook professional photo

feppe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2906
  • Oh this shows up in here!
    • Harri Jahkola Photography
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #9 on: May 24, 2009, 07:36:45 am »

Quote from: free1000
Can you explain? What do you mean by a real world situation rather than pixel peeping?  

With the bracketing, are you talking about 3 brackets and using Tufuse, or are more brackets required.  

Maybe I should look at that approach.

Real world meaning printed or web, not looked at 100% or larger magnification, or nose-length on large prints for the slightest flaw.

I usually bracket 3 shots (-2, 0, +2), but I've found out two brackets at four or so stops apart is plenty. I did a three-bracket exposure of this shot, but ended up using only 2 since I didn't get any extra info from the third bracket.

For HDR (if that's your thing) I think you need more than that, though.

slide

  • Newbie
  • *
  • Offline Offline
  • Posts: 9
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #10 on: May 24, 2009, 02:41:33 pm »

I've done long exposures with my 5d2 and not seen any issues with hot pixels. I use no noise reduction in camera. As to the comment that some poster claims too much credit to Canon due to his dislike of a direct print button, what's that got to do with it? We're talking noise and not if a camera has a button you don't like to use. Such comments add more noise than the sum of all the hot pixels ever put into all pix by all the Canons or Nikons on the planet. If you don't like Canon, don't buy them.

Hot pixels in long exposures seems related to ambient temperature. All my long exposures have been below 20 C so I have no idea if this will be a problem later this year when I'll be shooting at 30 C. If so, I will probably handle it in PP
« Last Edit: May 24, 2009, 02:41:57 pm by slide »
Logged

mattcrawford

  • Newbie
  • *
  • Offline Offline
  • Posts: 18
    • http://
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #11 on: May 24, 2009, 05:59:13 pm »

Just about to buy a 5D MKII and have tested a few at the shop, found a number of obvious hot pixies easily visible at 50% on one and what looked like lots of dust / black spots on another which didn't go anywhere even with a good blow of air. Both bodies were out of sealed boxes and dealer told me they had never had a lens on.
Hot pixies were very visible with a 3 second exposure.

Will be trying some more bodies before buying....
Matt
Logged

scottp

  • Newbie
  • *
  • Offline Offline
  • Posts: 2
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #12 on: May 26, 2009, 01:55:40 am »

Thanks for all the insights. I understand this much better now.

There is one other issue that comes up now: I sometimes shoot exposures of up to 60 seconds hand-held, often moving the camera during part of the exposure. I would think that the automatic, in-camera dark frame approach would not work, technically. An earlier post claimed that the hot pixels only appear with exposures of 8+ minutes (on the MKII) but others see them with shorter durations, yes? Perhaps, my 30 sec. average exposures wouldn't need the dark frame NR.

So, to create the dark frame in post-processing (pp): Do you simply create a black image of the same resolution? If so, why does the in-camera (second exposure) approach need an exposure of the same length as the "first" exposure? Is it gathering ambient light data? Does the shutter actually open for the duration to make the blank frame? Sorry, if I didn't see any specific instructions on creating the blank/black frame in pp in any articles.

I'm probably over-thinking this.
Logged

LoisWakeman

  • Guest
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #13 on: May 26, 2009, 04:47:31 am »

As I understand it (probably not well), the characteristics of the noise depend on the length of the exposure, so for optimal cancelling-out by the dark frame, you need the same conditions each time.
Logged

feppe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2906
  • Oh this shows up in here!
    • Harri Jahkola Photography
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #14 on: May 26, 2009, 04:59:47 am »

Quote from: LoisWakeman
As I understand it (probably not well), the characteristics of the noise depend on the length of the exposure, so for optimal cancelling-out by the dark frame, you need the same conditions each time.

That's right. It also depends on ambient temperature and sensor temp, so it's better to take the dark frame immediately after the shot.

Once again, such concerns are largely academic, and I'd encourage the OP to test whether this is an issue at all for him.

Jeremy Payne

  • Guest
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #15 on: May 26, 2009, 07:39:34 am »

I've heard that in the context of long-exposure night imagery, Canon's approach is better than Nikon's Long Exp NR because Nikon's approach can "eat" stars ... not sure as that's not my thing.
Logged

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #16 on: May 26, 2009, 11:08:46 am »

Quote from: Jeremy Payne
I've heard that in the context of long-exposure night imagery, Canon's approach is better than Nikon's Long Exp NR because Nikon's approach can "eat" stars ... not sure as that's not my thing.
I wonder if the reason for Canon to be better than Nikon for this kind of images is that Canon's black point correspond to a positive value in the RAW file, while Nikon (and most other brands) clip the RAW values to 0 for the black point.

Regarding the dark frame substraction, I was just wondering: to substract a dark frame from a regular frame could be fine for fixed pattern noise such as hot pixels, banding noise,... which would then be (nearly) cancelled. Should also work on only positive noise RAW files (all but Canon actually).

But for the Canon RAW files, that allow for negative noise with respect to the black level, a more sofisticated strategy should be done because a substraction of a negative value would mean increasing noise (5 - (-3) = 8).
I was looking at DCRAW's routine for dark frame substraction:

void CLASS subtract (char *fname)
{
..FILE *fp;
..int dim[3]={0,0,0}, comment=0, number=0, error=0, nd=0, c, row, col;
..ushort *pixel;

..if (!(fp = fopen (fname, "rb"))) {
....perror (fname); return;
..}

..if (fgetc(fp) != 'P' || fgetc(fp) != '5') error = 1;
..while (!error && nd < 3 && (c = fgetc(fp)) != EOF) {
....if (c == '#')  comment = 1;
....if (c == '\n') comment = 0;
....if (comment) continue;
....if (isdigitĀ©) number = 1;
....if (number) {
......if (isdigitĀ©) dim[nd] = dim[nd]*10 + c -'0';
......else if (isspaceĀ©) {
.........number = 0; nd++;
......} else error = 1;
....}
..}

..if (error || nd < 3) {
....fprintf (stderr,_("%s is not a valid PGM file!\n"), fname);
....fclose (fp); return;
..} else if (dim[0] != width || dim[1] != height || dim[2] != 65535) {
....fprintf (stderr,_("%s has the wrong dimensions!\n"), fname);
....fclose (fp); return;
..}

..pixel = (ushort *) calloc (width, sizeof *pixel);
..merror (pixel, "subtract()");
..for (row=0; row < height; row++) {
....fread (pixel, 2, width, fp);
....for (col=0; col < width; col++)
......BAYER(row,col) = MAX (BAYER(row,col) - ntohs(pixel[col]), 0);
..}

..free (pixel);
..black = 0;
}

After all checks, in the last loop it performs a simple substraction between the Bayer values from the RAW file and the values read from the dark frame file, clipping to 0 for negative results. But this subtract() function is called bofre scale_colors(), which is the function that actually substracts the black level in Canon file. So I wonder if dark frame substraction could not be optimum for Canon files in DCRAW.

EDIT: forget all those worries about the Canon dark frame substraction in DCRAW: RAW values are always positive, even for Canon. The only difference is that the black point is not 0 but a positive value in Canon (128, 256, 1024,...). So the dark frame substraction performed by DCRAW is also fine for Canon files.

Regards.
« Last Edit: May 26, 2009, 11:23:47 am by GLuijk »
Logged

free1000

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 465
    • http://www.foliobook.mobi
Long Exposure Hot Pixels: D700 & 5D MKII
« Reply #17 on: May 27, 2009, 05:21:40 am »

A few more thoughts having made some tests last night.

I shot some stars at ISO's  200 up to 6400, without dark frame (Long Ex NR) off and on. Exposures were between 30 secs and 10 minutes. The conditions were not ideal for me as I really wanted more moonlit illumination of the scene, as a result shadows were underexposed so this test was only really significant for the skies.

I was pretty amazed at the performance of the 5D2.  The performance at ISO's up to 3200 was very good. At 6400, there was a bit more visible noise. I was mostly looking at the sky here, which is featureless. The noise was more troublesome on the ground which was mostly grass. I suspect that buildings would render better.

With the dark frame subtraction switched off I found that the results are very good, so perhaps its not necessary most of the time. Yes there are many hot pixels visible, at exposures as short as 1-2 minutes.  However these can be removed by the hot pixel removal in Raw Developer.  With the long ex NR switched on I found some 'black dots' which actually were less easy to remove than the hot pixels.
 
My feeling would be that shooting anywhere from 800-3200 with the NR switched off should give acceptable results for shorter exposure times.

I still haven't done a sufficiently systematic test to draw final conclusions about where the sweet spot is, but on the other hand I don't think I'd be 'burnt' using settings anywhere from 200-3200 ISO with long exposures.

There is an Auto setting for the long exposure NR. I don't know what factors switch it on or off though.

One proviso, temperature was fairly cool, so that probably helped overall. I think the air temp was about 14c or so maybe.

I'm now thinking about putting the camera in a coolbag of some sort if I'm going somewhere hot. I wonder if anyone else has tried this. Probably would need to bag the camera with some silica gel before placing it in the coolbag.
« Last Edit: May 27, 2009, 05:35:13 am by free1000 »
Logged
@foliobook
Foliobook professional photo
Pages: [1]   Go Up