"Focus stacking" from 1 image??

[Hening Bettermann]

Hi!

It dawns to me that most of my images, which are shot with focus stacking in mind, will be impossible to stack due to moving foliage. In images with moderate depth, a workaround might be to take the image with the most important parts in focus and treat the unsharp parts with deconvolution (beyond global capture sharpening). But areas of different defocus would of course require different degrees of deconvolution.

Can anybody point to a software or a workaround, that would make it possible to extract pixels of equal degrees of blur from 1 source image and export these as separate images? These could then be imported as layers into one final image.

Any ideas to the rescue?

[Iluvmycam]

I HDR from one image. I will be interested to see focus stacks from one image.

[Bart_van_der_Wolf]

Hi!

It dawns to me that most of my images, which are shot with focus stacking in mind, will be impossible to stack due to moving foliage.

Hi Hening,

I'm trying to envision what you are worried about.

When in a frame that's supposed to be in focus, there is motion blur, then that's how it is. When there are two frames, slices of bracketed focus, and the supposed to be in-focus-frame has motion blur, but the supposed to be out-of-focus-frame is relatively better focused because there happened to be no motion, then that part of the image will be used. The result is an overall better focused image.

You always have the possibility to address defocus or motion blur in a single image by selective deconvolution. A common application is to improve corner sharpness with a radially masked layer where the deconvolution is targeted at the corners, and the mask gradually hides the part (the center of the frame) that gets over-sharpened. You can also address very local motion blur, but then you may need more layers if the motion is in different directions.

In addition, Focus Stacking software like Helicon Focus has a retouching capability which allows to manually pick detail from any of the input images. So if you envision motion blur issues, then take two shots with the same focus, and let the stacking software pick the least blurred parts automatically, or if that fails do it manually.

Can anybody point to a software or a workaround, that would make it possible to extract pixels of equal degrees of blur from 1 source image and export these as separate images? These could then be imported as layers into one final image.

To select defocused regions, you would need to use software that allows to isolate (de-)focused areas. This hints at spatial frequency separation, something that can e.g. be done with ImageJ with it's FFT Bandpass filter. Open the image and convert it to an "Image type 32-bit" version with the menu choices. Then use "Process FFT Bandpass Filter...", and use the setting from the attached image to select the focused areas. Then convert the result to "Image Type 16-bit" and save it as a TIFF or a PNG file, which then can be used in Photoshop to create a mask.

You can create a mask from the above selection and a layer filled with 50% gray, in Difference mode. Then use a Levels adjustment and a bit of blur to get the general mask, perfected with some local painting in/out.

Cheers,
Bart

[Hening Bettermann]

Hi Bart,

thank you very much for your reply, and sorry if I was not detailed enough in the description of my problem.

Motion blur in a single image is not my problem. Moving foliage will be rendered unsharp, and in many cases, I could swallow that, as long as the branches are sharp. But I have no single images, I have shot all my images with focus- and/or exposure- and/or super resolution stacking in mind.

Typically, these moving leaves are scattered over different parts of the image, so a radial mask will not do the trick.

I have and use Helicon Focus of course. It renders moving foliage not unsharp, but as ghost images, and that is undigestable. Manual retouching in HF would in many cases be a job from here to eternity. - BTW, I was not aware of that HF would pick the best image from multiple frames of the SAME focus slice, I'll have to try that. In a number of cases, I have multiple frames, originally intended for super resolution stacking (something I have given up due to artifacts - in STATIONARY parts of the image.)

But what I really had hoped for is something like your recommendation of ImageJ - WOW! I'll try this out ASAP. I may have to get back with more detailed questions on my way... If I can make this work, my plan is then in the future to shoot moving foliage in only 1 exposure triplet, maybe in f/16 rather than f/8 which is the standard now. This may work in cases of MODERATE depth in the subject.

add-on

You write "use the settings from the attached image to select the FOCUSSED areas." - Will it be possible, by changing the parameters, to generate different output images for different degrees of (de)focus?

Thanks again! - Hening



 

[Bart_van_der_Wolf]

I have and use Helicon Focus of course. It renders moving foliage not unsharp, but as ghost images, and that is undigestable. Manual retouching in HF would in many cases be a job from here to eternity. - BTW, I was not aware of that HF would pick the best image from multiple frames of the SAME focus slice, I'll have to try that.

Yes, and you can use that also in another way. You can blur parts of all but one image that will be assembled to a single image, and only the image that was not blurred will be used. Of course, the common way of combining images for a single slice is by using Median averaging. That will usually get rid of ghosts and only retain stationary objects. It preserves sharp edges that are most frequently present in all images. It's the same method one can use to remove tourists from images of monuments, just take a number of images with intervals (so people move to different positions between images), and only the stationary monument will remain.

You write "use the settings from the attached image to select the FOCUSSED areas." - Will it be possible, by changing the parameters, to generate different output images for different degrees of (de)focus?

Yes, but there is some smooth overlap, so it won't be just that easy. I chose the 1 to 0 pixel region because that's easy, nothing is smaller than 0 pixels. But when you'd select, say, the 2 to 1 pixel region, it is not a hard boundary with the 1 to 0 region but rather one with a peak with smooth edge transitions to the frequency regions above and below. It might be easier to e.g. select a 2 to 0 pixel region and subtract the resulting mask from the 1 to 0 region from it, to make sure that the overlap connections are clean.

Cheers,
Bart

[Hening Bettermann]

Thanks again, Bart, I'll have a lot to learn and try, but it looks like there might be hope.

[Hening Bettermann]

ImageJ 1.49 and the Mac 10.9.5 memory management don't seem to understand each other. I did not find this mentioned under known bugs. Would anybody know a trick?

[Bart_van_der_Wolf]

ImageJ 1.49 and the Mac 10.9.5 memory management don't seem to understand each other. I did not find this mentioned under known bugs. Would anybody know a trick?

Don't know about the Mac memory management specifics, but on Java under Windows there is a setting in an ImageJ.cfg file that allows to boost the amount of memory allocated when starting ImageJ. I have it set to "-Xmx6400m" which allocated 6GB of RAM, which seems enough for most purposes, although with some wild experiments I can occasionally still get out-of-memory errors. Also, clicking on the status bar of the ImageJ application panel will trigger a garbage collection memory cleanup, freeing some of ImageJ's temporarily used memory.

Cheers,
Bart

[mgrayson]

You can set the memory limit in the Edit>Options>Memory&Threads... menu. Restart the program and continue.

I've been looking for something like this. Helicon and Zerene are great, but sometimes you just want to get into the weeds. Thanks, Bart!

-Matt

[Hening Bettermann]

Thanks for your both' comments.
>You can set the memory limit in the Edit>Options>Memory&Threads... menu. Restart the program and continue.
This is what I have done, and allocated 12 of 16 available GB of RAM. But as my screen shot should show, ImageJ claims to have used all of them, and that's not enough; while the Mac Activity Monitor says only 7 or 8 GB are used. Have yet to try the trick with the garbage cleanup.
Best regards - Hening.
Edit:
I have tried it now, and it does not seem to work. According to the status bar, ImageJ has used 300 of 1200 MB available memory, but I still get the error report that it has used 1185 and is out of memory.

[Bart_van_der_Wolf]

I have tried it now, and it does not seem to work. According to the status bar, ImageJ has used 300 of 1200 MB available memory, but I still get the error report that it has used 1185 and is out of memory.

How large is the image (dimensions)?

Cheers,
Bart

[Hening Bettermann]

It's a 16 bit TIF, 3745 x 5626 pixels, 126,5 MB; 241 MB after conversion to 32 bit in ImageJ.

[Bart_van_der_Wolf]

It's a 16 bit TIF, 3745 x 5626 pixels, 126,5 MB; 241 MB after conversion to 32 bit in ImageJ.

That shouldn't be a problem, memory wise. You can display a memory usage indicator (Plugin>Utilities>Monitor Memory...). Try clicking on the ImageJ status bar for memory garbage collection, during the activity that generates the out of memory error. You may also want to convert the image from a 3 layer or channel composite image to an RGB image (Image>Color>Stack to RGB), for a different reason. A three layer composite image is not treated the same as an RGB image, also for Fourier conversions.

Cheers,
Bart

[Hening Bettermann]

Ooops - I wondered why ImageJ displayed the red channel only/on top; so one has to stack it to RGB first... After I did that, the size has gone down from 126 to 80 MB - despite the conversion to 32 bit. It must have lost information?? Regardless, the Status bar and the
Monitor Memory window now show 545/546 MB of memory usage, but when trying the Bandpass filter, I still get the out-of-memory error, claiming that 1185 MB have been used. Clicking in the Status bar does not change it. I can not click in the Status bar DURING the the execution of the Bandpass command - the out-of-memory alert appears before I can do it.
The Mac Activity Monitor now shows a memory usage of 1.14 GB for ImageJ. So that is close to the error message.

Now I try to stack to RGB first, then convert to 32 bit. That results in a black&white image.

Then I try to stack to RGB without converting to 32 bit first, that results in the same 80 MB color image. But now I get no error message when performing the FFT! Now, after conversion to Image > Type > 16 bit, I get a b&w image (40 MB) which looks like a complete image, just in b&w.

I repeat the procdure with the region 16-0 pixels. That also looks like a complete image in b&w. I save both to disk as TIFs.

When opened in Preview or PhotoLine, they are both completely black...

Uff...

[BobShaw]

On a completely different thought, to me focus stacking does not work. If you change the focal point then you also change the focal length. That is how a lens focusses. So the image "moves" even if the subject hasn't.
Get a good lens and plenty of light and shoot at f22 in one click.

[Bart_van_der_Wolf]

On a completely different thought, to me focus stacking does not work. If you change the focal point then you also change the focal length. That is how a lens focusses. So the image "moves" even if the subject hasn't.

Hi Bob,

Focus stacking does work, and very well at that, because the stacking software will compensate for the changing magnification that results from refocusing. Images are resampled (up or down) to have perfect registration at the intersection of the focus slices. BTW, the focal length (usually) stays the same, but you probably meant focus distance induced magnification.
 

Get a good lens and plenty of light and shoot at f22 in one click.

Most images will be resolution limited due to diffraction at f/22. Resolution limited does mean just that, not just do the images get lower in overall contrast, but it also won't reach its resolution potential (which is usually somewhere near f/4 to f/5.6, depending on design) because resolution is lost beyond restoration.

Cheers,
Bart

[Hening Bettermann]

Well, focus stacking does work in Helicon Focus, as long as the subject does not move. HF aligns the images. The slices I hope to extract using ImageJ do not move, so it should be possible to use them as masks and fuse the resulting partial images without further alignment.
Good light! - Hening.

edit: I wrote this in reply to Bob before I had read Barts more detailed answer.

[BobShaw]

Most images will be resolution limited due to diffraction at f/22.

Sorry, just realised you must be talking about 35mm cameras.
The f64 Group if they started now would have to call themselves The f5.6 Group (:-)

[Bart_van_der_Wolf]

Sorry, just realised you must be talking about 35mm cameras.

No, not really. Diffraction doesn't care about the sensor it falls on. It's the required magnification for output that determines if it affects visible resolution.

The f64 Group if they started now would have to call themselves The f5.6 Group (:-)

The film sizes that were common for the f/64 group didn't require a lot of magnification to print, so the huge diffraction pattern of f/64, some 88+ microns when focused at 10 metres (33 feet) distance, stayed under the visual acuity limits at contact print size, and were becoming only just visible at 2x to 3x magnification.

Cheers,
Bart

[BobShaw]

Diffraction doesn't care about the sensor it falls on. It's the required magnification for output that determines if it affects visible resolution.
Cheers,
Bart

Exactly. The bigger the camera the less magnification is required. To produce an A4 print from 35mm you have to magnify roughly 8 times in each direction. Actually 77x.
To produce an A4 print from medium format you only have to magnify about 6 times in each direction, or about 30x. Enormous difference.
To produce an A4 from an 8x10 you don't magnify at all.
Conversely if it was a cropped camera you would need to magnify 184x.
The lens would need to be from the Hubble to be diffraction clean. So you need to resort to less than ideal ways to overcome a basic problem.