Motion from BlurShengyang Dai and Ying Wu

EECS Department, Northwestern University

NORTHWESTERNUNIVERSITY

http://vision.middlebury.edu/flow/Baker, Scharstein, Lewis, Roth,

Black, Szeliski, ICCV’07(courtesy to Simon Baker)

Motion from blur

Optical flow estimation

?

Discrete samples

over time

Integral over time

LiteratureTask Input Extra info

Q. Shan, W. Xiong, and J. Jia, CVPR 07 Rotational motion blur

Single image

User interaction

A. Levin, NIPS 06

Multiple / local invariant linear motion estimation and

segmentation

Known single blur direction

S. Cho, Y. Matsushita, and S. Lee, ICCV 07 Two images

XL. Bar, B. Berkels, M. Rumpf, and G. Sapiro,

ICCV 07Sequence

Space-variant linear motion estimation from blurred image(s)

Our workTask Input Extra info

Q. Shan, W. Xiong, and J. Jia, CVPR 07 Rotational motion blur

Single image

User interaction

A. Levin, NIPS 06

Multiple / local invariant linear motion estimation and

segmentation

Known single blur direction

S. Cho, Y. Matsushita, and S. Lee, ICCV 07 Two images

XL. Bar, B. Berkels, M. Rumpf, and G. Sapiro,

ICCV 07Sequence

Our work

1. Global parametric form motion from blur (e.g., affine / rotational motion)

2. Multiple / local motion estima-tion and segmentation from blur

3. Non-parametric motion from blur

Single image X

Space-variant linear motion estimation from blurred image(s)

)2

()2

(b

pb

pbpb III

+

-b

pb bI

P

pbI

Motion blur constraint

=( ) ·

)2

()2

(b

pb

pbpb III

Motion Blur vs. Optic Flow

)2

()2

(b

pb

pbb III

tII m

• Lucas and Kanade, 81• Horn and Schunck, 81• Barron, Fleet, Beauchemin, IJCV 94• Black and Anandan, CVIU 96• Baker and Matthews, IJCV 04• Baker, Scharstein, Lewis, Roth, Black,

Szeliski, ICCV 07• ……

?

channel image representation

Spectral matting Levin, Rav-Acha, and Lischinski, CVPR 07

Unsupervised image layer decomposition

(courtesy to Anat Levin)0 1

Alpha motion blur constraint

)2

()2

(b

pb

pbb III )2

()2

(b

pb

pbb 1 bb0

1

b b

0 bb 0 b

• Use to replace the original image

• Assumptions: most pixels have 0 / 1 alpha values

• Observation: mostly when

,b II ,b

0 bwhen

motion blurred imagebI

1 bb

Alpha motion blur constraint

1

0

b1 bb

Alpha motion blur constraint

b

Alpha channel of the blurred imageb

Motion Blur vs. Optic Flow

1 bb )2

()2

(b

pb

pbb III

tII m

• Lucas and Kanade, 81• Horn and Schunck, 81• Barron, Fleet, Beauchemin, IJCV 94• Black and Anandan, CVIU 96• Baker and Matthews, IJCV 04• Baker, Scharstein, Lewis, Roth, Black,

Szeliski, ICCV 07• ……

?

Hough transform

b

1 bb

Techniques for optical flow

• Lucas-Kanade• Horn-Schunck• Robust estimation• RANSAC• Multiple resolution

• Parametric• Piecewise smooth• Segmentation• LK meets HS• ……

Techniques for motion from blur

• Lucas-Kanade• Horn-Schunck• Robust estimation• RANSAC• Multiple resolution

• Parametric• Piecewise smooth• Segmentation• LK meets HS• ……

Tasks

Global affine motion blur

Global rotational motion blur

Multiple blur model estimation and segmentation

Non-parametric motion field estimation

Experiments – affine motion

Experiments – rotational motion

Experiments – segmentation

Segmentation result

Experiments – non-parametric motion

Applications

Ground truth image

Synthesized affine blurred image

(PSNR: 21.66dB)

Deblurred result (PSNR: 24.75dB)

Image deblurringModified Richardson-Lucy iteration

average estimation error: 0.43average motion vector: 15.43

Affine blurred image Our deblur result with• space-variant blur• modified RL iteration

Matlab deconvlucy with • space-invariant blur• original RL iteration

time Input

Deblurred

Blur synthesis

Motion synthesis

Blur / motion synthesis

Blur synthesis

Input Output

Motion synthesis

Input Output

Motion synthesis

Summary• Contributions

– A local linear motion blur constraint– Connection between motion blur and optic flow– Space-variant motion estimation from blur– Applications on deblurring and blur/motion synthesis

• Limitations– May not hold for heavily textured region– Rely on robust matting

• Future work– Integrating more algorithms from optic flow

Motion from BlurShengyang Dai and Ying Wu

EECS Department, Northwestern Universityb

1 bb

Thanks! Questions?