Luminous Landscape Forum
Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: Jim Kasson on November 10, 2013, 12:59:51 pm
-
I’ve been doing some macro photography, and I’m unsatisfied with the sharpness that I’m getting. I’ve been trying to develop a testing regime aimed at determining the amount of blurring caused by camera vibration, unflat field, aberrations, diffraction, and the like. To that end, I’ve created a target which has a fairly broad band of high-spatial-frequency energy. I am photographing the target, processing the images in Lightroom, exporting them as TIFFs, reading them into Matlab, converting them to a linear (gamma = 1) representation, and performing analysis, the critical part of which is measuring the standard deviation (or, if you prefer, root-mean-square noise) of the image.
In order to make the rms noise measurement meaningful with real-world images, I need to compensate for both global and local exposure differences. Since the value of the standard deviation of an image in a linear color space should, all else being equal and ignoring photon noise, be proportional to exposure, I perform the compensation by forming a correcting image by filtering the input image with a large (400x400 pixel) constant-value kernel, and then dividing the input image by the correcting image to form the corrected image.
There’s a problem: the technique overcorrects the images exported from Lightroom. I tried turning off everything I could find in the Develop module, including camera calibration. It made a difference, but didn’t fix things.
So I created two images that differed in exposure by a stop. I measured the ratio of the mean values of the G channels of the two raw images with Rawdigger: it was 2.015. In Lightroom, Using PV 2012, the ratio of the linearized Adobe RGB green channels was 1.688. With PV 2010 and PV 2003, it was 1.681. Using all three channels and converting to monochrome in Matlab produces similar result. There seems to be a tone curve applied by Lightroom that keeps ratios in the raw file from being preserved in linear representations of the converted image.
I tried Iridient Developer, and got different, but still incorrect (in the photogrammetric sense) ratios of about 1.65. The ratio varied with the processing options. I thought the raw channel mixer set to green only might produce the right ratio, but no joy.
Using DCRAW, with the command line incantation “dcraw -v -4 -a -w -j -T -o1 _D437349.NEF” produces sRGB files with the green channel mean ratio of 2.015, the same ratio as that of the raw files –- actually it’s not quite the same ratio (it differs in the fifth decimal place) but I attribute that to the change of color space from camera native to sRGB. DCRAW users will note that I’m white balancing to average; leaving this out makes little difference.
So, while I have a solution for raw conversion that I can use, it’s much less convenient for me than to do the conversions using Lightroom.
So, my question is: How do I set up Lightroom so that the tone ratios of the original raw file are preserved in a linear representation of the output file?
Thanks for any help on this.
Jim
PS. For a little background, my first fumbling steps in this project are covered in this (http://blog.kasson.com/?p=3556)and subsequent posts.
-
That's quite a quandary you've found yourself there, Jim.
Did you make sure to check the parabula statistus murado on the second go round on those linear compounder numbers you came up with? You may need to get Lightroom CSI, a version used by NASA and NORAD.
Is there a reason why you haven't posted image samples showing this sharpness issue with macro shots both here and on your blog?
I mean I use the crappiest lenses with extension tube and 2X teleconverter all (under $100) strapped to the cheapest DSLR I could find and get excellent sharpness editing all those foggy, crappy captures into tack sharp perfection in both Lightroom and ACR. No analysis needed.
Oh, and I can do all those edits faster than I could read and figure out what you were trying to accomplish in your post.
-
That's quite a quandary you've found yourself there, Jim.
Tim, I admit to having a analytical bent. I like doing this stuff, probably more than is good for my photography. Guilty as charged. I'm hoping maybe Eric can help me out, but I suspect that a lot of Lightroom gurus know how to turn off some processing I missed.
Is there a reason why you haven't posted image samples showing this sharpness issue with macro shots both here and on your blog?
I did. Here (http://blog.kasson.com/?p=3536)and here (http://blog.kasson.com/?p=3536).
Oh, and I can do all those edits faster than I could read and figure out what you were trying to accomplish in your post.
I'm sure you can. I've found that I get better results if I spend most of my effort on the capture, and relatively little on the editing, but that's just me.
Thanks (I think).
BTW, I've figured out how to call DCRAW from Matlab and really don't need to know how to make Lightroom do the job anymore, but I'm still curious as to what it's doing with the mystery tone curve (if that's what it is) and why.
Jim
-
Jim, your firehouse images look better than what I can get straight out of my camera with regards to sharpness. I really don't see a sharpness problem with those posted images. They look real and follow the character of the light in how it's suppose to render sharpness appearance.
They do have a sort of natural looking tonal roll off "Low Clarity" for lack of a better descriptive term which brings me to mentioning something I noticed about display calibration variances between two displays I've been using in the past 3 years that have affected my perception of the appearance of clarity in combination with PV2003 vs PV2012 renderings.
Clarity works hand and glove with sharpness appearance and I'ld been using CS3's PV2003 Clarity rendering which behaves much different than PV2012's. But to compound things I also noticed when I went from the Dell 2209WA (subpar quality with regards to linearity/black point appearance), calibrated with old i1Match/i1Display package, and switching to my current LG 27ea63v-p (VERY LINEAR) with much deeper blacks and slightly punchier contrast calibrating with ColorMunki Display, PV2003's Clarity & Parametric Curve behavior noticeably improved working on the LG.
For some reason the LG induced me to re-edit quite a few low contrast scenes edited on the Dell in PV2003 that used to look over processed (HDCartooned?) where now the PV2003 Clarity slider can be cranked up even more than I could on the Dell now with the LG. When I switch to viewing these images edited on the Dell/PV2003 using Lightroom 4's PV2012 they look even less cartoonish but now the Clarity slider applies a much broader tonal roll off over PV2003's "HighPass" fat halo look to contrasted edges. This is most likely what you are seeing in Lightroom in how it renders low light, low contrast scenes in the firehouse images.
So my point is what I think you may be seeing is influenced by variances I've described in my situation that I think instead of applying precision math to sort it out is better mitigated by finding other ways to make your images look sharper using Lightroom's editing tools. I'm guessing your images are most likely over 12MP and with the low contrast/AA filter would most likely require a larger LR sharpening radius while maintaing low contrast. It's easier to make high contrast images look sharper.
I can't tell you how many times I've gone back and re-edited certain shots I've fallen in love with because I lingered on the image for too long to make them look 3D. That Parametric curve editor is addictive especially cranking up mids/highlights on low contrast, dim shots.
Below are two rendering attempts on two separate shots of the same scene shot through a window where one is darker/high contrast, the other is brighter and a bit lower contrast to show how contrast perception affects overall appearance of sharpness. Both have the Sharpness radius set at (2) instead of the normal (1) using PV2003 in CS3 ACR. Clarity is set at 65 on the darker/hi contrast version w/over-cranked parametric curve while the lighter less contrast version has Clarity at 100 w/less parametric midrange boost.
-
Hi,
I think you can go to Process Version 2010 and set all relevant controls to zero. I don't think you can get straight rendition out of PV 2012.
Best regards
Erik
Tim, I admit to having a analytical bent. I like doing this stuff, probably more than is good for my photography. Guilty as charged. I'm hoping maybe Eric can help me out, but I suspect that a lot of Lightroom gurus know how to turn off some processing I missed.
I did. Here (http://blog.kasson.com/?p=3536)and here (http://blog.kasson.com/?p=3536).
I'm sure you can. I've found that I get better results if I spend most of my effort on the capture, and relatively little on the editing, but that's just me.
Thanks (I think).
BTW, I've figured out how to call DCRAW from Matlab and really don't need to know how to make Lightroom do the job anymore, but I'm still curious as to what it's doing with the mystery tone curve (if that's what it is) and why.
Jim
-
Jim, have you examined the quality of pixel edge sawtoothed detail viewing your firehouse images in their current edited state at pixel level to see if the proper sharpen settings are adequate?
My shots posted above using subpar quality equipment and captured through a window are sharp in their current unedited Raw state that produce perfect halo-less sharp edges viewed at 400% after applying only 120 Amount in ACR PV2003.
What do the pixelated edges look like on your images compared to what I get shown below viewed at 400%?
-
It is easy to process raw files in MATLAB using dcraw as a file-parser (decode in "document mode" into 16-bit TIFF, then use imread() in MATLAB).
I believe that Lightroom is very much about "offer users what we think they will be happy with" as opposed to "offer sliders that are mathematically/physically analyzable". And it seems to be a popular software package.
-h
-
It is easy to process raw files in MATLAB using dcraw as a file-parser (decode in "document mode" into 16-bit TIFF, then use imread() in MATLAB).
Good idea. Since yesterday, I've been constructing DCRAW commands as character strings in Matlab and using "system(commandString);" to execute them and then using imread() to get the TIFF file. Your idea of using DCRAW's document mode rather than letting DCRAW do the demosaicing would give me access to each of the four raw planes, and I could analyze each individually, or use Matlab's demosaicing function, and stay in native camera color space.
Jim
-
Jim, your firehouse images look better than what I can get straight out of my camera with regards to sharpness. I really don't see a sharpness problem with those posted images. They look real and follow the character of the light in how it's suppose to render sharpness appearance.
Tim, here is an example image, with no clarity or contrast boosting in Lr, just using the default sharpening. I did tweak Exposure and Shadows a bit. The image was exposed at f/25 for 3 seconds with the 105mm f/2.8G Nikon Macro lens. I don't like to use openings that small because of diffraction, but I needed the DOF, and, as you'll see, I still don't have enough. I'm worried about camera motion, but don't know if is an issue with this image. The camera was a D4, which I used because I like the way it handles, but when I get this series dialed in, I'm going to switch to a D800E.
First, the overall image:
(http://www.kasson.com/ll/_D436523.jpg)
Next, the plane of sharpest focus, at 1:1:
(http://www.kasson.com/ll/_D436523-4.jpg)
It could be a lot sharper, IMHO. I know I can make improvements in post, but I'd rather start with a better image.
And, to boot, the DOF isn't adequate, as seen by the two crops from behind the plane of sharp focus:
(http://www.kasson.com/ll/_D436523-2.jpg)
(http://www.kasson.com/ll/_D436523-3.jpg)
So, I see my challenge as getting the main plane of focus sharper by opening up the lens and maybe (depending on what I find with my testing) doing something to mitigate vibration besides shutter delay and the self-timer. That will make the DOF problem worse, and I'll probably have to bite the bullet and do focus stacking to fix that.
Then there's my reflection in the shiny bits. I'm still thinking about what to do about that.
Thanks for your suggestions. I'm sure they'll help once I get a better capture.
Jim
-
Tim, here is an example image, with no clarity or contrast boosting in Lr, just using the default sharpening. I did tweak Exposure and Shadows a bit. The image was exposed at f/25 for 3 seconds with the 105mm f/2.8G Nikon Macro lens.
Hi Jim,
That will not only give a lot of diffraction, but it will also lose effective resolution (and require long exposure times). An aperture of f/25 will limit resolution to 1/(N*wavelength) cycles/mm, or 1/(25*0.000555)=72 cy/mm for green light. The fire-truck red will be even worse. At 72 cy/mm and higher the MTF will be zero, i.e. no resolution, and all micro-detail will be lost without any chance of recovery. At the Nyquist frequency of the D4 sensor (68.5 cy/mm) the MTF is almost zero, so only the highest contrast (>100:1) detail has any chance to make it in the absence of noise. Add a tiny bit of lens aberration and there will be nothing to recover, and any added defocus will strip even the faint chance of resolving any high spatial frequency.
I don't like to use openings that small because of diffraction, but I needed the DOF, and, as you'll see, I still don't have enough.
Only focus-stacking can achieve what you want. Opening up the lens will significantly reduce the diffraction, and allow shorter exposure times.
I'm worried about camera motion, but don't know if is an issue with this image. The camera was a D4, which I used because I like the way it handles, but when I get this series dialed in, I'm going to switch to a D800E.
The smaller sensels of the D800 will be impacted by diffraction starting at f/5.6 (the D4 at f/8.0), if you want to gain resolution you will be limited to f/18 before total loss of high spatial frequency resolution.
It could be a lot sharper, IMHO. I know I can make improvements in post, but I'd rather start with a better image.
And, to boot, the DOF isn't adequate, as seen by the two crops from behind the plane of sharp focus:
Focus stacking at f/4 (maybe f/5.6 at most) is going to give you optimal results for macro shots at 1:1, but the DOF will be approx. 0.1 mm, so either a lot of focus brackets will be required, or you'll have to settle for a smaller (downsampled) output size, and/or more diffraction.
Then there's my reflection in the shiny bits. I'm still thinking about what to do about that.
The wider aperture used with focus stacking may help, otherwise use some lens blur in post-processing.
Cheers,
Bart
-
Hi,
This page shows the effects of diffraction vs. DoF quite well: http://echophoto.dnsalias.net/ekr/index.php/photoarticles/49-dof-in-digital-pictures?start=1
Personally, I would use a large radius (like 1.6 and deconvolution type sharpening).
Best regards
Erik
Hi Jim,
That will not only give a lot of diffraction, but it will also lose effective resolution (and long exposure times). An aperture of f/25 will limit resolution to 1/(N*wavelength) cycles/mm, or 1/(25*0.000555)=72 cy/mm for green light. The fire-truck red will be even worse. At 72 cy/mm and higher the MTF will be zero, i.e. no resolution, and all micro-detail will be lost without any chance of recovery. At the Nyquist frequency of the D4 sensor (68.5 cy/mm) the MTF is almost zero, so only the highest contrast (>100:1) detail has any chance to make it in the absence of noise. Add a tiny bit of lens aberration and there will be nothing to recover, and any added defocus will strip even the faint chance of resolving any high spatial frequency.
Only focus-stacking can achieve what you want. Opening up the lens will significantly reduce the diffraction, and allow shorter exposure times.
The smaller sensels of the D800 will be impacted by diffraction starting at f/5.6 (the D4 at f/8.0), if you want to gain resolution you will be limited to f/18 before total loss of high spatial frequency resolution.
Focus stacking at f/4 (maybe f/5.6 at most) is going to give you optimal results for macro shots at 1:1, but the DOF will be approx. 0.1 mm, so either a lot of focus brackets will be required, or you'll have to settle for a smaller (downsampled) output size, and/or more diffraction.
The wider aperture may help, otherwise use some lens blur in post-processing.
Cheers,
Bart
-
Tim, here is an example image, with no clarity or contrast boosting in Lr, just using the default sharpening. I did tweak Exposure and Shadows a bit. The image was exposed at f/25 for 3 seconds with the 105mm f/2.8G Nikon Macro lens. I don't like to use openings that small because of diffraction, but I needed the DOF, and, as you'll see, I still don't have enough. I'm worried about camera motion, but don't know if is an issue with this image. The camera was a D4, which I used because I like the way it handles, but when I get this series dialed in, I'm going to switch to a D800E.
First, the overall image:
(http://www.kasson.com/ll/_D436523.jpg)
Next, the plane of sharpest focus, at 1:1:
(http://www.kasson.com/ll/_D436523-4.jpg)
It could be a lot sharper, IMHO. I know I can make improvements in post, but I'd rather start with a better image.
And, to boot, the DOF isn't adequate, as seen by the two crops from behind the plane of sharp focus:
(http://www.kasson.com/ll/_D436523-2.jpg)
(http://www.kasson.com/ll/_D436523-3.jpg)
So, I see my challenge as getting the main plane of focus sharper by opening up the lens and maybe (depending on what I find with my testing) doing something to mitigate vibration besides shutter delay and the self-timer. That will make the DOF problem worse, and I'll probably have to bite the bullet and do focus stacking to fix that.
Then there's my reflection in the shiny bits. I'm still thinking about what to do about that.
Thanks for your suggestions. I'm sure they'll help once I get a better capture.
Jim
Change-over valve on a fire engine's pump panel, right?
Rand
-
Change-over valve on a fire engine's pump panel, right?
Sounds like you know a lot more about it than I do. Here's a view of most of the panel. The control is at the very bottom, just a little right of center. Engine is an early 70s American LaFrance.
(http://www.kasson.com/ll/_D436514.jpg)
Jim
-
This page shows the effects of diffraction vs. DoF quite well: http://echophoto.dnsalias.net/ekr/index.php/photoarticles/49-dof-in-digital-pictures?start=1
Thanks, Erik. And thanks for your post processing ideas. I'm not ready for them yet, but I'll get there.
Jim
-
Only focus-stacking can achieve what you want. Opening up the lens will significantly reduce the diffraction, and allow shorter exposure times.
The smaller sensels of the D800 will be impacted by diffraction starting at f/5.6 (the D4 at f/8.0), if you want to gain resolution you will be limited to f/18 before total loss of high spatial frequency resolution.
Thanks, Bart, for the suggestions and the theory. Now that I've got DCRAW and Matlab playing together, my first (strobe-lit) tests indicate that the sharpest aperture for the Nikon 105mm f/2.8G macro is f/8, and for the Zeiss 100mm f/2 ZF, it's f5.6, at least at the distance I'm using initially. That's very close to what you said independent of the particular lens. I still have more testing to do, with both lenses on both bodies, and I want to see if there is much variation in optimum aperture based on the spatial frequencies that I look at. Then I want to compare sharpness of the two lenses against each other. Then it'll be onto vibration-induced blur with various shutter and mirror release techniques. Then some other lenses: the Sigma 180 mm APO, and maybe the 60mm Micro Nikkor.
Jim
-
Jim,
Please share your findings!
Best regards
Erik
Thanks, Bart, for the suggestions and the theory. Now that I've got DCRAW and Matlab playing together, my first (strobe-lit) tests indicate that the sharpest aperture for the Nikon 105mm f/2.8G macro is f/8, and for the Zeiss 100mm f/2 ZF, it's f5.6, at least at the distance I'm using initially. That's very close to what you said independent of the particular lens. I still have more testing to do, with both lenses on both bodies, and I want to see if there is much variation in optimum aperture based on the spatial frequencies that I look at. Then I want to compare sharpness of the two lenses against each other. Then it'll be onto vibration-induced blur with various shutter and mirror release techniques. Then some other lenses: the Sigma 180 mm APO, and maybe the 60mm Micro Nikkor.
Jim
-
Sounds like you know a lot more about it than I do. Here's a view of most of the panel. The control is at the very bottom, just a little right of center. Engine is an early 70s American LaFrance.
(http://www.kasson.com/ll/_D436514.jpg)
Jim
Jim,
Sorry for the digression on the thread, but I can't resist. Here's a MUCH younger version of me at the wheel of a '73 LaFrance engine (taken circa '82)
(http://rsadams.smugmug.com/photos/i-pHqrtmD/0/S/i-pHqrtmD-S.jpg)
If you're interested, what that valve does is change the pump from pumping in series (for max pressure) to pumping in parallel (max volume/capacity). The pump has two impellers and can be run both ways.
All this to prove, you can take the firefighter out of the firehouse... but....
Rand
-
Then there's my reflection in the shiny bits. I'm still thinking about what to do about that.
The trick to photographing reflective objects with curves and angles is camera position and lighting. Usually multiple exposures are made with different lighting set ups, which are then blended in post. Keeping light off camera will help too with reflections.
Jim Haefner (http://www.jameshaefner.com) has posted some behind the scenes of his car photography on LuLa which demonstrate exactly this method and he's done some stunning car photography. Couldn't any to link to though.
-
I’ve been doing some macro photography, and I’m unsatisfied with the sharpness that I’m getting. I’ve been trying to develop a testing regime aimed at determining the amount of blurring caused by camera vibration, unflat field, aberrations, diffraction, and the like. To that end, I’ve created a target which has a fairly broad band of high-spatial-frequency energy. I am photographing the target, processing the images in Lightroom, exporting them as TIFFs, reading them into Matlab, converting them to a linear (gamma = 1) representation, and performing analysis, the critical part of which is measuring the standard deviation (or, if you prefer, root-mean-square noise) of the image.
In order to make the rms noise measurement meaningful with real-world images, I need to compensate for both global and local exposure differences. Since the value of the standard deviation of an image in a linear color space should, all else being equal and ignoring photon noise, be proportional to exposure, I perform the compensation by forming a correcting image by filtering the input image with a large (400x400 pixel) constant-value kernel, and then dividing the input image by the correcting image to form the corrected image.
There’s a problem: the technique overcorrects the images exported from Lightroom. I tried turning off everything I could find in the Develop module, including camera calibration. It made a difference, but didn’t fix things.
So I created two images that differed in exposure by a stop. I measured the ratio of the mean values of the G channels of the two raw images with Rawdigger: it was 2.015. In Lightroom, Using PV 2012, the ratio of the linearized Adobe RGB green channels was 1.688. With PV 2010 and PV 2003, it was 1.681. Using all three channels and converting to monochrome in Matlab produces similar result. There seems to be a tone curve applied by Lightroom that keeps ratios in the raw file from being preserved in linear representations of the converted image.
I tried Iridient Developer, and got different, but still incorrect (in the photogrammetric sense) ratios of about 1.65. The ratio varied with the processing options. I thought the raw channel mixer set to green only might produce the right ratio, but no joy.
Using DCRAW, with the command line incantation “dcraw -v -4 -a -w -j -T -o1 _D437349.NEF” produces sRGB files with the green channel mean ratio of 2.015, the same ratio as that of the raw files –- actually it’s not quite the same ratio (it differs in the fifth decimal place) but I attribute that to the change of color space from camera native to sRGB. DCRAW users will note that I’m white balancing to average; leaving this out makes little difference.
So, while I have a solution for raw conversion that I can use, it’s much less convenient for me than to do the conversions using Lightroom.
So, my question is: How do I set up Lightroom so that the tone ratios of the original raw file are preserved in a linear representation of the output file?
Thanks for any help on this.
Jim
PS. For a little background, my first fumbling steps in this project are covered in this (http://blog.kasson.com/?p=3556)and subsequent posts.
Hello,
I took a Johnny-come-lately perusal of this thread. You got some great info from Brat and Eric. I want to address your desire to compute the local noise in the image: Perhaps you can succeed in the technique you are trying but i think there is a more straightforward analysis that will be helpful. First, the noise in the sensels is independent, from sensel to sensel, if you can work on the raw values. The key is not to convolve adjacent sensel values. Simply record about 20 identical images and compute the noise at each voxel as the sample standard deviation using Matlab. Hope this helps.
-
Jim,
Sorry for the digression on the thread, but I can't resist. Here's a MUCH younger version of me at the wheel of a '73 LaFrance engine (taken circa '82)
Rand
So that was You doubling for Burt!
-
Here's a MUCH younger version of me at the wheel of a '73 LaFrance engine (taken circa '82)
Rand,
Then you probably were born about the time this engine was manufactured (photographed in Sunnyside, Prince Edward Island, with an M240 and the 90mm Summicron).
(http://www.kasson.com/ll/L1002133.jpg)
Jim
-
Hello,
I took a Johnny-come-lately perusal of this thread. You got some great info from Brat and Eric. I want to address your desire to compute the local noise in the image: Perhaps you can succeed in the technique you are trying but i think there is a more straightforward analysis that will be helpful. First, the noise in the sensels is independent, from sensel to sensel, if you can work on the raw values. The key is not to convolve adjacent sensel values. Simply record about 20 identical images and compute the noise at each voxel as the sample standard deviation using Matlab. Hope this helps.
That would work for some important kinds of noise, but not for banding.
-h
-
Rand,
Then you probably were born about the time this engine was manufactured (photographed in Sunnyside, Prince Edward Island, with an M240 and the 90mm Summicron).
(http://www.kasson.com/ll/L1002133.jpg)
Jim
Jim,
Beautiful ... Thanks for sharing.
Rand
-
So that was You doubling for Burt!
LOL At this stage of my life I'm more often confused with Wilfred Brimley. ;D
Rand
-
I’ve been doing some macro photography, and I’m unsatisfied with the sharpness that I’m getting. I’ve been trying to develop a testing regime aimed at determining the amount of blurring caused by camera vibration, unflat field, aberrations, diffraction, and the like. To that end, I’ve created a target which has a fairly broad band of high-spatial-frequency energy. I am photographing the target, processing the images in Lightroom, exporting them as TIFFs, reading them into Matlab, converting them to a linear (gamma = 1) representation, and performing analysis, the critical part of which is measuring the standard deviation (or, if you prefer, root-mean-square noise) of the image.
There’s a problem: the technique overcorrects the images exported from Lightroom. I tried turning off everything I could find in the Develop module, including camera calibration. It made a difference, but didn’t fix things.
So I created two images that differed in exposure by a stop. I measured the ratio of the mean values of the G channels of the two raw images with Rawdigger: it was 2.015. In Lightroom, Using PV 2012, the ratio of the linearized Adobe RGB green channels was 1.688. With PV 2010 and PV 2003, it was 1.681. Using all three channels and converting to monochrome in Matlab produces similar result. There seems to be a tone curve applied by Lightroom that keeps ratios in the raw file from being preserved in linear representations of the converted image.
I tried Iridient Developer, and got different, but still incorrect (in the photogrammetric sense) ratios of about 1.65. The ratio varied with the processing options. I thought the raw channel mixer set to green only might produce the right ratio, but no joy.
Using DCRAW, with the command line incantation “dcraw -v -4 -a -w -j -T -o1 _D437349.NEF” produces sRGB files with the green channel mean ratio of 2.015, the same ratio as that of the raw files –- actually it’s not quite the same ratio (it differs in the fifth decimal place) but I attribute that to the change of color space from camera native to sRGB. DCRAW users will note that I’m white balancing to average; leaving this out makes little difference.
Jim,
In my testing, I have found it is very difficult to obtain linear results with PV2012. However, if one uses PV2010 and sets the tone curve to linear (sliders on main page = 0 and point curve to linear) one can obtain approximately linear results by rendering into ProPhotoRGB and then converting the image to a custom space using ProPhoto primaries and a gamma of 1.0.
Here are my results with the Nikon D800e using ACR 8.2 and PV2010 with the Adobe Camera Standard profile. I photographed a Stouffer step wedge using the above technique and analyzed the results with Imatest. The results are shown graphically. The resulting gamma is approximately 1.0 and halving the exposure (the steps are 0.1 OD or 1/3 f/stop) results in half the pixel value.
Regards,
Bill
-
That would work for some important kinds of noise, but not for banding.
-h
h,
The noise that OP referred to is random. My impression is that banding is a systematic error, not random.
-
... if one uses PV2010 and sets the tone curve to linear (sliders on main page = 0 and point curve to linear) one can obtain approximately linear results by rendering into ProPhotoRGB and then converting the image to a custom space using ProPhoto primaries and a gamma of 1.0.
Bill, I'll go back to Lr and try that. But for at least the next week or so, I'm focused (ahem!) on solving my sharpness testing problem with DCRAW so I can get back to making actual photographs.
I did go back and look at my Lr settings, and, thanks to you, I found the setting I had missed: the point curve, which was set to the default of Med Contrast. So what was happening is that the more ETTR images (and the brighter parts of all the images) were being compressed (slope less than one) while the parts of the image nearer middle grey were being expanded (slope greater than one).
So, problem solved, subject to my verification.
Thanks!
Jim
-
I want to address your desire to compute the local noise in the image: Perhaps you can succeed in the technique you are trying but i think there is a more straightforward analysis that will be helpful. First, the noise in the sensels is independent, from sensel to sensel, if you can work on the raw values. The key is not to convolve adjacent sensel values. Simply record about 20 identical images and compute the noise at each voxel as the sample standard deviation using Matlab. Hope this helps.
I'm afraid that I've caused confusion by using the term "rms noise" in the original post, and I apologize for that. What I meant by that term is the same algorithm used to calculate the standard deviation. Basically, it's the rms value of the deviations from the mean. Engineers often refer to that calculation as rms noise, even when the thing causing the deviations isn't, strictly speaking, stochastic. That's the way I was using the term, and it was sloppy, and it gave you and others the wrong impression. Another way to think of the calculation is to use a term more often used in one-dimensional signal processing: the rms value of the ac component.
In the images that I am processing, most of the variation from the mean value is signal: that is, it is caused by the target, as interpreted by the imaging system. I am well away from the read noise, so banding is not an issue. However, there is one source of real noise: photon/shot noise. There is another source of what can be thought of as noise: pixel response non-uniformity. The first can be averaged out as you suggest. The second cannot. I am operating in a portion of the dynamic range of the camera where PRNU is at least as large as photon noise.
You got me thinking about this, so I did some testing. I averaged the ten exposure-corrected images in each set, and compared the standard deviation of the averaged image to the average standard deviation of the ten individual images. The errors ranged from 0.05% to 0.14% in the eight-set aperture series I used for this experiment.
I could come up with a way to reduce the effect of the photon noise by computing the gain factor from sensor electrons to ADC counts (camera ISO / Unity Gain ISO), computing the photon noise standard deviation as the square root of the average sensor electrons count, and subtracting that in quadrature from the standard deviation I've already computed. I'm resisting that, because it seems unnecessary, and because it will make the whole procedure more complicated to explain.
I could also compute a correcting image for each camera by making a bunch of images of a flat field, applying lighting correction, and averaging to get a pixel response map which I could use to correct for PRNU. I am resisting that for the same reasons as above.
Thanks for your interest, and, again, I apologize for the confusion.
Jim
-
So, while I have a solution for raw conversion that I can use, it’s much less convenient for me than to do the conversions using Lightroom.
So, my question is: How do I set up Lightroom so that the tone ratios of the original raw file are preserved in a linear representation of the output file?
Hi Jim, I would think there is so many places in the application between capture to export where this can be influenced that this question can be only answered by somebody from the LR team, like Eric. That having said, I am trying to figure out how you want to compensate for the blur and if there aren't tools already available in LR and PS that do it the same way. LR can do deconvolution sharpening. LR can do a sort of high radius/ low amount (your 400x400 matrix?). Photoshop can do several types of deblur, including motion and lens. Photoshop and plugins can do DOF stacking.
-
In the images that I am processing, most of the variation from the mean value is signal: that is, it is caused by the target, as interpreted by the imaging system. I am well away from the read noise, so banding is not an issue. However, there is one source of real noise: photon/shot noise. There is another source of what can be thought of as noise: pixel response non-uniformity. The first can be averaged out as you suggest. The second cannot. I am operating in a portion of the dynamic range of the camera where PRNU is at least as large as photon noise.
If you have a sufficiently "smooth" target and no temporal (signal) variation, fire N shots, then (ideally) all sensels within a color plane should be the same. The samples can be seen as a 3-d array with variation ("noise") in the vertical, horisontal and temporal axes.
Zero-mean noise added to a "smooth" signal (in time and/or space) can be reduced by averaging. But is all "noise" zero-mean? All signals are not all that "smooth".
How can this variation be visualised and interpreted? What does it tell us? Is it used by the big guys presenting sensor analysis?
If you "dissect" the noise response of a given camera at various temperatures and settings into temporal and spatial components... Is it possible to improve noise reduction (even for single-image input)?
-h
-
Check out Helicon Remote, it automates focus stacking from a Laptop, Desktop or Android devise.
I use it with my D7000 and Nexus 7 tablet.
If you need 100 shots to get the needed stack, no problem with Helicon remote.
Do a search, they have a video.