Image Deblurring using Inertial Measurement Sensors

[Guillermo Luijk]

Recenrly Bill Janes started a thread about image deconvolution to eliminat AA filter blurring. This seems to be a test from MS to achieve similar results in deblurring images shot at too low shutter speeds. Software image stabilization?

Image Deblurring using Inertial Measurement Sensors.

Abstract

We present a deblurring algorithm that uses a hardware attachment coupled with a natural image prior to deblur images from consumer cameras. Our approach uses a combination of inexpensive gyroscopes and accelerometers in an energy optimization framework to estimate a blur function from the camera’s acceleration and angular velocity during an exposure. We solve for the camera motion at a high sampling rate during an exposure and infer the latent image using a joint optimization. Our method is completely automatic, handles per-pixel, spatially-varying blur, and out-performs the current leading image-based methods. Our experiments show that it handles large kernels – up to at least 100 pixels, with a typical size of 30 pixels. We also present a method to perform “ground-truth” measurements of camera motion blur. We use this method to validate our hardware and deconvolution approach. To the best of our knowledge, this is the first work that uses 6 DOF inertial sensors for dense, per-pixel spatially-varying image deblurring and the first work to gather dense ground-truth measurements for camera-shake blur.



Regards

[BernardLanguillier]

Recenrly Bill Janes started a thread about image deconvolution to eliminat AA filter blurring. This seems to be a test from MS to achieve similar results in deblurring images shot at too low shutter speeds. Software image stabilization?

Yep, read that.

Very interesting indeed. I guess that we will see very soon cameras embedding more motion sensors. The Hassy H4D might be closest to being able to do this actually.

I wouldn't be surprised if some brands delayed their next generation high end body to include such capabilities.

Cheers,
Bernard

[ejmartin]

As I understand it, the method amounts to recording the motion data for use in a later deconvolution, rather than simply having image stabilization hardware.  Seems like a rather Rube Goldberg way of going about things.  If I may make an analogy, which would you rather have the camera do: (1) autofocus, or (2) record in metadata the distance data for correct focus, for later use by a software deblurring algorithm?  

[NikoJorj]

If I may make an analogy, which would you rather have the camera do: (1) autofocus, or (2) record in metadata the distance data for correct focus, for later use by a software deblurring algorithm?  

Well, that depends on what is the most efficient...  

Should the mention of "a typical kernel size of 30 pixels" mean that the method can deblur a movement blur 30 pixels wide? If true, I'd say that it would be (much) more efficient than traditionnal stabilisation systems (which shoudln't be usable in conjunction with the deblur method, should they?), and that Rube Goldberg rules.

[Bart_van_der_Wolf]

Should the mention of "a typical kernel size of 30 pixels" mean that the method can deblur a movement blur 30 pixels wide? If true, I'd say that it would be (much) more efficient than traditionnal stabilisation systems (which shoudln't be usable in conjunction with the deblur method, should they?), and that Rube Goldberg rules.

Hi Nicolas,

There is a limit to what can still be recovered after a 30 pixel motion blur. Of course, it will do better if there is enough convolved signal in neighboring pixels to use, because it will have more signal to work with, but a lot of signal will have become too low to be of use.

A camera movement based correction will still not allow to correct subject motion, something that regular image restoration procedures can do (also within limits).

Cheers,
Bart

[Guillermo Luijk]

Today I received the Konica to M4/3 adaptor for the Olympus E-P1, and did a quick stabilization test shooting with the Hexanon AR 40mm f/1.8 at 1/10. That is 3 stops slower than the general rule 1 / focal_length_eq.

The improvement is very good:



As soon as the battery is full I plan to do a sharpness and noise comparision:

1/10 + IS
vs
1/80 + 3 extra ISO stops

To find out how IS can be seen as a noise reduction.

Salu2

[hjulenissen]

Software/accelerometer-gyro/deconvolution based image "stabilization" should have a disadvantage in that deconvolution is a process that is sensitive to noise and deep spectral nulls.

On the other hand, it could add very little camera cost, and be able to correct rotation in the image sensor plane (no lense stabilization will ever do this, and I dont think that in-camera stabilization is able to rotate the sensor either).


A simple camera motion blur along pixel rows or columns should be a spatial 1-d rectangular PSF, giving a sin(x)/x frequence response. Its inverse involves infinite gain at zero-crossings, as well as high overall gain at higher frequencies.

-h