Luminous Landscape Forum
Raw & Post Processing, Printing => Digital Image Processing => Topic started by: Hening Bettermann on March 20, 2015, 03:52:09 pm
-
Hi!
It dawns to me that most of my images, which are shot with focus stacking in mind, will be impossible to stack due to moving foliage. In images with moderate depth, a workaround might be to take the image with the most important parts in focus and treat the unsharp parts with deconvolution (beyond global capture sharpening). But areas of different defocus would of course require different degrees of deconvolution.
Can anybody point to a software or a workaround, that would make it possible to extract pixels of equal degrees of blur from 1 source image and export these as separate images? These could then be imported as layers into one final image.
Any ideas to the rescue?
-
I HDR from one image. I will be interested to see focus stacks from one image.
-
Hi!
It dawns to me that most of my images, which are shot with focus stacking in mind, will be impossible to stack due to moving foliage.
Hi Hening,
I'm trying to envision what you are worried about.
When in a frame that's supposed to be in focus, there is motion blur, then that's how it is. When there are two frames, slices of bracketed focus, and the supposed to be in-focus-frame has motion blur, but the supposed to be out-of-focus-frame is relatively better focused because there happened to be no motion, then that part of the image will be used. The result is an overall better focused image.
You always have the possibility to address defocus or motion blur in a single image by selective deconvolution. A common application is to improve corner sharpness with a radially masked layer where the deconvolution is targeted at the corners, and the mask gradually hides the part (the center of the frame) that gets over-sharpened. You can also address very local motion blur, but then you may need more layers if the motion is in different directions.
In addition, Focus Stacking software like Helicon Focus has a retouching capability which allows to manually pick detail from any of the input images. So if you envision motion blur issues, then take two shots with the same focus, and let the stacking software pick the least blurred parts automatically, or if that fails do it manually.
Can anybody point to a software or a workaround, that would make it possible to extract pixels of equal degrees of blur from 1 source image and export these as separate images? These could then be imported as layers into one final image.
To select defocused regions, you would need to use software that allows to isolate (de-)focused areas. This hints at spatial frequency separation, something that can e.g. be done with ImageJ with it's FFT Bandpass filter. Open the image and convert it to an "Image type 32-bit" version with the menu choices. Then use "Process FFT Bandpass Filter...", and use the setting from the attached image to select the focused areas. Then convert the result to "Image Type 16-bit" and save it as a TIFF or a PNG file, which then can be used in Photoshop to create a mask.
You can create a mask from the above selection and a layer filled with 50% gray, in Difference mode. Then use a Levels adjustment and a bit of blur to get the general mask, perfected with some local painting in/out.
Cheers,
Bart
-
Hi Bart,
thank you very much for your reply, and sorry if I was not detailed enough in the description of my problem.
Motion blur in a single image is not my problem. Moving foliage will be rendered unsharp, and in many cases, I could swallow that, as long as the branches are sharp. But I have no single images, I have shot all my images with focus- and/or exposure- and/or super resolution stacking in mind.
Typically, these moving leaves are scattered over different parts of the image, so a radial mask will not do the trick.
I have and use Helicon Focus of course. It renders moving foliage not unsharp, but as ghost images, and that is undigestable. Manual retouching in HF would in many cases be a job from here to eternity. - BTW, I was not aware of that HF would pick the best image from multiple frames of the SAME focus slice, I'll have to try that. In a number of cases, I have multiple frames, originally intended for super resolution stacking (something I have given up due to artifacts - in STATIONARY parts of the image.)
But what I really had hoped for is something like your recommendation of ImageJ - WOW! I'll try this out ASAP. I may have to get back with more detailed questions on my way... If I can make this work, my plan is then in the future to shoot moving foliage in only 1 exposure triplet, maybe in f/16 rather than f/8 which is the standard now. This may work in cases of MODERATE depth in the subject.
add-on
You write "use the settings from the attached image to select the FOCUSSED areas." - Will it be possible, by changing the parameters, to generate different output images for different degrees of (de)focus?
Thanks again! - Hening
-
I have and use Helicon Focus of course. It renders moving foliage not unsharp, but as ghost images, and that is undigestable. Manual retouching in HF would in many cases be a job from here to eternity. - BTW, I was not aware of that HF would pick the best image from multiple frames of the SAME focus slice, I'll have to try that.
Yes, and you can use that also in another way. You can blur parts of all but one image that will be assembled to a single image, and only the image that was not blurred will be used. Of course, the common way of combining images for a single slice is by using Median averaging. That will usually get rid of ghosts and only retain stationary objects. It preserves sharp edges that are most frequently present in all images. It's the same method one can use to remove tourists from images of monuments, just take a number of images with intervals (so people move to different positions between images), and only the stationary monument will remain.
You write "use the settings from the attached image to select the FOCUSSED areas." - Will it be possible, by changing the parameters, to generate different output images for different degrees of (de)focus?
Yes, but there is some smooth overlap, so it won't be just that easy. I chose the 1 to 0 pixel region because that's easy, nothing is smaller than 0 pixels. But when you'd select, say, the 2 to 1 pixel region, it is not a hard boundary with the 1 to 0 region but rather one with a peak with smooth edge transitions to the frequency regions above and below. It might be easier to e.g. select a 2 to 0 pixel region and subtract the resulting mask from the 1 to 0 region from it, to make sure that the overlap connections are clean.
Cheers,
Bart
-
Thanks again, Bart, I'll have a lot to learn and try, but it looks like there might be hope.
-
ImageJ 1.49 and the Mac 10.9.5 memory management don't seem to understand each other. I did not find this mentioned under known bugs. Would anybody know a trick?
-
ImageJ 1.49 and the Mac 10.9.5 memory management don't seem to understand each other. I did not find this mentioned under known bugs. Would anybody know a trick?
Don't know about the Mac memory management specifics, but on Java under Windows there is a setting in an ImageJ.cfg file that allows to boost the amount of memory allocated when starting ImageJ. I have it set to "-Xmx6400m" which allocated 6GB of RAM, which seems enough for most purposes, although with some wild experiments I can occasionally still get out-of-memory errors. Also, clicking on the status bar of the ImageJ application panel will trigger a garbage collection memory cleanup, freeing some of ImageJ's temporarily used memory.
Cheers,
Bart
-
You can set the memory limit in the Edit>Options>Memory&Threads... menu. Restart the program and continue.
I've been looking for something like this. Helicon and Zerene are great, but sometimes you just want to get into the weeds. Thanks, Bart!
-Matt
-
Thanks for your both' comments.
>You can set the memory limit in the Edit>Options>Memory&Threads... menu. Restart the program and continue.
This is what I have done, and allocated 12 of 16 available GB of RAM. But as my screen shot should show, ImageJ claims to have used all of them, and that's not enough; while the Mac Activity Monitor says only 7 or 8 GB are used. Have yet to try the trick with the garbage cleanup.
Best regards - Hening.
Edit:
I have tried it now, and it does not seem to work. According to the status bar, ImageJ has used 300 of 1200 MB available memory, but I still get the error report that it has used 1185 and is out of memory.
-
I have tried it now, and it does not seem to work. According to the status bar, ImageJ has used 300 of 1200 MB available memory, but I still get the error report that it has used 1185 and is out of memory.
How large is the image (dimensions)?
Cheers,
Bart
-
It's a 16 bit TIF, 3745 x 5626 pixels, 126,5 MB; 241 MB after conversion to 32 bit in ImageJ.
-
It's a 16 bit TIF, 3745 x 5626 pixels, 126,5 MB; 241 MB after conversion to 32 bit in ImageJ.
That shouldn't be a problem, memory wise. You can display a memory usage indicator (Plugin>Utilities>Monitor Memory...). Try clicking on the ImageJ status bar for memory garbage collection, during the activity that generates the out of memory error. You may also want to convert the image from a 3 layer or channel composite image to an RGB image (Image>Color>Stack to RGB), for a different reason. A three layer composite image is not treated the same as an RGB image, also for Fourier conversions.
Cheers,
Bart
-
Ooops - I wondered why ImageJ displayed the red channel only/on top; so one has to stack it to RGB first... After I did that, the size has gone down from 126 to 80 MB - despite the conversion to 32 bit. It must have lost information?? Regardless, the Status bar and the
Monitor Memory window now show 545/546 MB of memory usage, but when trying the Bandpass filter, I still get the out-of-memory error, claiming that 1185 MB have been used. Clicking in the Status bar does not change it. I can not click in the Status bar DURING the the execution of the Bandpass command - the out-of-memory alert appears before I can do it.
The Mac Activity Monitor now shows a memory usage of 1.14 GB for ImageJ. So that is close to the error message.
Now I try to stack to RGB first, then convert to 32 bit. That results in a black&white image.
Then I try to stack to RGB without converting to 32 bit first, that results in the same 80 MB color image. But now I get no error message when performing the FFT! Now, after conversion to Image > Type > 16 bit, I get a b&w image (40 MB) which looks like a complete image, just in b&w.
I repeat the procdure with the region 16-0 pixels. That also looks like a complete image in b&w. I save both to disk as TIFs.
When opened in Preview or PhotoLine, they are both completely black...
Uff...
-
On a completely different thought, to me focus stacking does not work. If you change the focal point then you also change the focal length. That is how a lens focusses. So the image "moves" even if the subject hasn't.
Get a good lens and plenty of light and shoot at f22 in one click.
-
On a completely different thought, to me focus stacking does not work. If you change the focal point then you also change the focal length. That is how a lens focusses. So the image "moves" even if the subject hasn't.
Hi Bob,
Focus stacking does work, and very well at that, because the stacking software will compensate for the changing magnification that results from refocusing. Images are resampled (up or down) to have perfect registration at the intersection of the focus slices. BTW, the focal length (usually) stays the same, but you probably meant focus distance induced magnification.
Get a good lens and plenty of light and shoot at f22 in one click.
Most images will be resolution limited due to diffraction at f/22. Resolution limited does mean just that, not just do the images get lower in overall contrast, but it also won't reach its resolution potential (which is usually somewhere near f/4 to f/5.6, depending on design) because resolution is lost beyond restoration.
Cheers,
Bart
-
Well, focus stacking does work in Helicon Focus, as long as the subject does not move. HF aligns the images. The slices I hope to extract using ImageJ do not move, so it should be possible to use them as masks and fuse the resulting partial images without further alignment.
Good light! - Hening.
edit: I wrote this in reply to Bob before I had read Barts more detailed answer.
-
Most images will be resolution limited due to diffraction at f/22.
Sorry, just realised you must be talking about 35mm cameras.
The f64 Group if they started now would have to call themselves The f5.6 Group (:-)
-
Sorry, just realised you must be talking about 35mm cameras.
No, not really. Diffraction doesn't care about the sensor it falls on. It's the required magnification for output that determines if it affects visible resolution.
The f64 Group if they started now would have to call themselves The f5.6 Group (:-)
The film sizes that were common for the f/64 group didn't require a lot of magnification to print, so the huge diffraction pattern of f/64, some 88+ microns when focused at 10 metres (33 feet) distance, stayed under the visual acuity limits at contact print size, and were becoming only just visible at 2x to 3x magnification.
Cheers,
Bart
-
Diffraction doesn't care about the sensor it falls on. It's the required magnification for output that determines if it affects visible resolution.
Cheers,
Bart
Exactly. The bigger the camera the less magnification is required. To produce an A4 print from 35mm you have to magnify roughly 8 times in each direction. Actually 77x.
To produce an A4 print from medium format you only have to magnify about 6 times in each direction, or about 30x. Enormous difference.
To produce an A4 from an 8x10 you don't magnify at all.
Conversely if it was a cropped camera you would need to magnify 184x.
The lens would need to be from the Hubble to be diffraction clean. So you need to resort to less than ideal ways to overcome a basic problem.
-
After some fiddling around, I have found out that even if I convert the stacked RGB to 8 bit grayscale before trying the Bandpass filter, I get the Out of Memory message. I had overlooked that, because the alert window is covered by any new image window which is created.
If I use 8bit gray images, and export as tifs, I get a grayscale image, not just black. But regardless which parameters I use, the resulting images are all the same. Meaning the Bandpass Filter operation is not performed at all, due to memory problems.
So I think it is a problem of memory management, maybe specific for the Mac. When I get back home, I'll try it on Windows.
Regardless, Bart, I don't understand the conversions you recommend. Why the conversion from a 16 bit RGB TIF to a 32 bit grayscale image? Since these will be used as masks, even 8 bit may be enough?
-
After some fiddling around, I have found out that even if I convert the stacked RGB to 8 bit grayscale before trying the Bandpass filter, I get the Out of Memory message. I had overlooked that, because the alert window is covered by any new image window which is created.
Sorry, I can't help with that Mac or machine specific issue.
Regardless, Bart, I don't understand the conversions you recommend. Why the conversion from a 16 bit RGB TIF to a 32 bit grayscale image? Since these will be used as masks, even 8 bit may be enough?
It's more of a safety measure, because Fourier Transforms can produce artifacts of their own if not done with enough numerical precision. So 16-bit/channel integer values as input may be okay, but I want to be sure that the subsequent processing is done at the highest precision that ImageJ can deliver. Since your goal is to produce masks, even 8-b/ch input may be adequate, as long as you don't intend to do other calculations in Fourier space (divisions by zero, e.g. for deconvolution, cause trouble). Band-pass filtering should probably be safe if implemented as multiplication.
For the generation of selections/masks, there are also lots of other tools available in ImageJ (e.g. Filters>Variance...), but that's getting a bit too involved in Digital Signal Processing of images for a quick explanation.
Maybe you are making things a bit too difficult for yourself. A good HDR application will automatically take care of Ghosting artifacts, or offer manual intervention in the process of stacking the exposure brackets.
Cheers,
Bart
-
Hi Bart,
thank you for your reply.
Of course you can not do anything about the Mac.
I understand now the logic behind the conversion to 32 bit.
> For the generation of selections/masks, there are also lots of other tools available in ImageJ (e.g. Filters>Variance...), but that's getting a bit too involved in Digital Signal Processing of images for a quick explanation.
Hm- where can I read more about it? The ImageJ Help > Documentation only says
"29.11.9 Variance…Highlights edges in the image by replacing each pixel with the neighborhood variance."
> A good HDR application will automatically take care of Ghosting artifacts, or offer manual intervention in the process of stacking the exposure brackets.
Lumariver, my HDR app, takes care of ghosting artifacts to a good degree, if not 100%. But what is at stake here is the *focus* stacking, which I will try to work around.
> Maybe you are making things a bit too difficult for yourself.
Well - my intention is of course the opposite. Assume that a major part of the foliage in this image is moving, then try to manually retouch it in Helicon...
Thanks again - Hening.
-
The RAM mystery is solved at last:
As the attached screen shot shows, I had not allocated 12 GB of RAM, but just 1.2... This is what happens if I have to deal with numbers - just not my thing... Sorry for bothering you with this.
So now I can run the bandpass filter. But the result looks like a luminance mask, not a sharpness selection. Is there a way to achieve what I want?
-
For the generation of selections/masks, there are also lots of other tools available in ImageJ (e.g. Filters>Variance...), but that's getting a bit too involved in Digital Signal Processing of images for a quick explanation.
Hm- where can I read more about it? The ImageJ Help > Documentation only says
"29.11.9 Variance…Highlights edges in the image by replacing each pixel with the neighborhood variance."
Well, if there is a lot of detail, then the local variance will be high. So that should give an indication of where the focus is high and the detail is fine and of high contrast. But it would be one step in a series towards creating a mask, depending on image content. Thresholding of images, and differences between images may need to be done to get the detail out, but it still won't solve the issue of detail displacement between slices. It would only offer a means to select detail in a single image. This is a separate thing.
A good HDR application will automatically take care of Ghosting artifacts, or offer manual intervention in the process of stacking the exposure brackets.
Lumariver, my HDR app, takes care of ghosting artifacts to a good degree, if not 100%. But what is at stake here is the *focus* stacking, which I will try to work around.
Yes, I understand. But the Focus stacking algorithms are based on one image being (locally) better focused than all others. When there are multiple candidates for the same image feature, then ghosting is likely to show up. Non-stationary subjects are not the prime candidates for focus-stacking, if you want to avoid a lot of manual intervention.
Occasionally it may help to divide the stacking procedure into multiple runs. One could e.g. try stacking the odd and the even images from a sequence separately, thus skipping one each time, and thereby create a better separation between the level of focus of each slice. Then stack the two resulting stacked images. The best strategy depends a lot on the actual images, and pre-processing the images into spatial frequency band masked versions may also be an option, but it's not all that easy.
The cleaner the slices can separate focused layers of the scene by distance, the easier the stacking should be, but it doesn't help if each slice's detail has moved ... Maybe shooting with a slower shutterspeed will blur the movement enough for the branches and grass to blend, while the stationary parts of the image benefit from the enhanced DOF resolution.
Cheers,
Bart
-
The RAM mystery is solved at last:
Good!
So now I can run the bandpass filter. But the result looks like a luminance mask, not a sharpness selection. Is there a way to achieve what I want?
It's just a first step. You can now e.g. calculate a difference with a uniform gray layer, so the dark and light edges will all turn light, you can then threshold it, and blur it, and contrast adjust it, to fine-tune the selection mask.
Cheers,
Bart
-
Hi Bart
thank you for your extensive answer.
> it still won't solve the issue of detail displacement between slices. It would only offer a means to select detail in a single image. This is a separate thing.
And this is exactly what I have in mind. My idea is to work *around* the focus stacking, using the one image with the most important parts in focus, and then use "graduated deconvolution" on the other parts.
> It's just a first step. You can now e.g. calculate a difference with a uniform gray layer, so the dark and light edges will all turn light, you can then threshold it, and blur it, and contrast adjust it, to fine-tune the selection mask.
This sounds like I would need a more detailed recipe, which does not seem to exist.
Thanks again! - Hening.
-
There may be hope for a solution. Not for images of which you have only one frame. Rather for those which are shot with focus stacking in mind, but suffer from moving foliage or the like. And the depth of the subject must not be too large; in this case, the sharpening of the extreme slices would lead to artefacts.
Helicon Focus lets you export the depth map as gray images. These could be used as masks. However, they are based on the source images as aligned in HF, so they will not fit the genuine source image. HF also allows you to export the aligned source images, but they are reduced to 8 bit color depth. So if HF just would offer an option to export the original source images in an aligned state, but otherwise untreated, it might work.
I have tried to persuade Stas Yatzenko from HF, and he has promised to put this on their to-do list, without a date of course. So their may be hope ahead...
Good light!