2019-04-10

Motion Magnification

ISL

안재원

Liu, Ce, et al. "Motion magnification." ACM transactions on graphics (TOG) 24.3 (2005): 519-526.

Motion Magnification

Video Registration

Layer-based motion analysis

2019-04-10

2

Video RegistrationLayer-based

motion analysisRendervideo

Motion Magnification2019-04-10

3

Intro

Motion Magnification2019-04-10

4

Intro

Big motion`

Small motion Magnification Big motion

Video Registration

Layer-based motion analysis

Rendervideo

- Motion magnification flow

Video Registration2019-04-10

5

Reference frame

Harris corner detector

?

? ? ??

?? ??

??

?

- SSD를 이용해 각 Feature를 matching한다.

? ( )2

- Reference frame과 각 Frame(k)간의 Global affine matrix를 구한다.

?

?

𝐴𝑘

Video Registration2019-04-10

6

Reference frame

?

??

?

22

2/1expPr k

Ty

nk

x

nk

T

nknkknk vvyxA n

Ty

nk

x

nk

T

nknkkk vvyxAn

2

11

nkkn PrPr ii

n PrmaxPr

kA~

- 각 Feature가 Stable feature일 확률을 계산.

- 다음의 조건을 만족하는 Feature는 Stable feature.

Layer-based motion analysis2019-04-10

7

Feature point tracking

Feature point tracking Trajectory clusteringDense optical flow

interpolationLayer segmentation

time

- SSD Feature matching의 문제

Stable feature

time

Layer-based motion analysis2019-04-10

8

Feature point tracking(Expectation-maximization algorithm)

- E-step

time

ni

Ki

nk

niKi

nknk

d

d

SSD

SSD

11min2min2

expPr

K

k

nk

K

k

nknnk

n

qpBqpB

s

qpqp

2

2

2

2

1

2

22

Pr2

Pr),(),(

2exp),(

- 𝑑𝑛𝑘

Layer-based motion analysis2019-04-10

9

Feature point tracking(Expectation-maximization algorithm)

- M-step

timeOutliers

- Removal and Interpolation(Minimizing the second derivative energy)

ni

Ki

nk

niKi

nknk

d

d

SSD

SSD

11min2min2

expPr

Layer-based motion analysis2019-04-10

10

Trajectory clustering

- 동일한 대상은 유사한 움직임을 보인다.

k

y

mk

x

mk

k

y

nk

x

nk

k

y

mk

x

mk

y

nk

x

nk

mn

vvvv

jvvjvv

2222

,

ZELNIK-MANOR, L., AND IRANI, M. “Degeneracies, dependencies and their implications in multi-body and multi-sequence factorizations.”

Layer-based motion analysis2019-04-10

11

Dense optical flow interpolation

Bicubic interpolation

MotionGroup 1(Green)

MotionGroup 2(Red)

?

?

Layer-based motion analysis2019-04-10

12

Layer segmentation

- Motion likelihood

- Color likelihood

- Spatial connectivity

주어진 Motion을 따라 움직였을 때 유사한 픽셀이 나오는가를 확인.

Gaussian mixture model을 사용해 어떤 layer에 속하는지 분류

인접한 픽셀간의 연결성 확인

BOYKOV, Y., VEKSLER, O., AND ZABIH, R. “Fast approximate energy minimization via graph cuts”

Layer-based motion analysis2019-04-10

13

Layer segmentation

Q & A

2019-04-10