Statistical Consistency of the Normalized Eight-Point...

Statistical Consistency of the NormalizedEight-Point Algorithm

W. Chojnacki and M. J. Brooks

School of Computer ScienceThe University of Adelaide

ICIAP 2007

1 W. Chojnacki and M. J. Brooks Consistency of the Normalized Eight-Point Algorithm

Introduction

Outline

1 Introduction

2 Prerequisites

3 Extended Statistical Model

4 Data Normalization

5 Parameter Estimation

6 Bias

7 Approximate Consistency

8 Conclusion


Introduction

Key Points

Hartley’s normalized eight-point algorithm can be justifiedstatistically

Data normalization =⇒ increase of the consistency ofestimates as the number of data points increases

Uses a statistical model for data distribution


Introduction

Related Work

Longuet-Higgins (Nature, 1981)

Hartley (PAMI, 1997)

Muhlich and Mester (ECCV, 1998)

Chojnacki, Brooks, van den Hengel, Gawley (ICIAP, PAMI,2003)


Prerequisites

Outline

1 Introduction

2 Prerequisites




6 Bias


8 Conclusion


Prerequisites

Fundamental Matrix

m = [mx, my, 1]T image point

(m, m′) pair of corresponding points

F = [fij] 3× 3 fundamental matrix

Fundamental matrixCaptures

Relative orientation of the camerasInternal geometry of the cameras

Satisfies

Epipolar constraintm′TFm = 0

Rank-2 constraintdet F = 0


Prerequisites

Algebraic Least Squares I

{(mn, m′n)}N

n=1 — data points

ALS cost function

JALS(F) = ∑Nn=1(m′T

n Fmn)2

‖F‖2F

‖F‖F = (∑i,j

f 2ij )

1/2 Frobenius norm

ALS estimate

FALS = arg minF 6=0

JALS(F)

Ignores the rank-2 constraint


Prerequisites

Algebraic Least Squares II

x = [mx, my, m′x, m′

y]T joint image point

based on (m, m′)

u(x) = vec(mm′T) carrier

θ = vec(FT) parameter vector

m′TFm = θTu(x)


Prerequisites

Algebraic Least Squares III

xn = [mx,n, my,n, m′x,n, m′

y,n]T joint image pointbased on (mn, m′

n)

A =N

∑n=1

u(xn)u(xn)T moment matrix

JALS(θ) =θTAθ

‖θ‖2 ‖θ‖ = (θ21 + · · ·+ θ2

9)1/2

vec(FTALS) = θALS is an eigenvector of A associated with the

smallest eigenvalue


Prerequisites

Normalized Algebraic Least Squares I

ALS estimates are highly sensitive to noise

Hartley’s modification

Normalize input data prior to running ALS

mn = Tmn m′n = T ′m′

n

T, T ′ 3× 3 data-dependent affine matrices

Apply ALS to the normalized data {(mn, m′n)}N

n=1 and thenback-transform the result


Prerequisites

Normalized Algebraic Least Squares II

Pre-NALS estimate of F

FALS ALS estimate based on {(mn, m′n)}N

n=1

NALS estimate of F

FNALS = T ′TFALST


Prerequisites

Normalized Algebraic Least Squares III

Computing Pre-NALS Estimate

xn = [mx,n, my,n, m′x,n, m′

y,n]T joint image pointbased on (mn, m′

n)

A =N

∑n=1

u(xn)u(xn)T moment matrixbased on {xn}N

n=1

θALS = vec(FTALS) is an eigenvector of A associated with the

smallest eigenvalue


Prerequisites

Reasons for ImprovementExisting Explanations

Hartley — numerical explanation

the smallest eigenvector of A = ∑Nn=1 u(xn)u(xn)T is less

sensitive to small perturbations of the matrix entries than thesmallest eigenvector of A = ∑N

n=1 u(xn)u(xn)T

Chojnacki et al. — statistical explanation

NALS estimate is a minimiser of a cost function, JNALSthe summands of JNALS are more balanced in terms of spreadthan the summands of JALS

uses a statistical model for data distribution


Extended Statistical Model

Outline

1 Introduction

2 Prerequisites




6 Bias


8 Conclusion



Reasons for ImprovementNew Statistical Explanation

Data normalization improves the consistency of estimates

Requires an extended statistical model for data distribution

Inspired by work of Muhlich and Mester on enhancing totalleast squares estimation methods via equilibration



Structure of Image Points

mn = nn + ∆mn m′n = n′n + ∆m′

n

nn = c + ∆nn n′n = c′ + ∆n′nn = 1, 2, . . .

nn, n′n “true” locations of ideal data points

∆mn, ∆m′n noise

c, c′ nonrandom “centroids”

∆nn, ∆n′n ideal data-point structural perturbations



Statistical Properties

∆mn, ∆m′n, ∆nn, ∆n′n are mutually independent

Zero means

E[∆mn] = E[∆m′n] = E[∆nn] = E[∆n′n] = 0

Anisotropic noise

E[∆mn∆mTn ] = diag(σ2

x , σ2y , 0)

E[∆m′n∆m′T

n ] = diag(σ′2x , σ′2y , 0)

Anisotropic structural perturbations

E[∆nn∆nTn ] = diag(τ2

x , τ2y , 0)

E[∆n′n∆n′Tn ] = diag(τ′2x , τ′2y , 0)


Data Normalization

Outline

1 Introduction

2 Prerequisites




6 Bias


8 Conclusion


Data Normalization

Empirical Means and Deviations

Means

mN =1N

N

∑n=1

mn m′N =

1N

N

∑n=1

m′n

Standard deviations

sx,N =

(1N

N

∑n=1

(mx,n −mx,N)2

)1/2

sy,N =

(1N

N

∑n=1

(my,n −my,N)2

)1/2

s′x,N =

(1N

N

∑n=1

(m′x,n −m′

x,N)2

)1/2

s′y,N =

(1N

N

∑n=1

(m′y,n −m′

y,N)2

)1/2


Data Normalization

Affine Matrices I

Image-based random affine matrices

TN =

s−1x,N 0 −s−1

x,Nmx,N

0 s−1y,N −s−1

y,Nmy,N

0 0 1

T′

N =

s′−1x,N 0 −s′−1

x,Nm′x,N

0 s′−1y,N −s′−1

y,Nm′y,N

0 0 1


Data Normalization

Affine Matrices II

Limit nonrandom affine matrices

limN→∞

TN = T

limN→∞

T′N = T ′

almost sure convergence(“except on an event ofprobability zero”)

T =

(σ2x + τ2

x )−1/2 0 −(σ2x + τ2

x )−1/2cx0 (σ2

y + τ2y )−1/2 −(σ2

y + τ2y )−1/2cy

0 0 1

T ′ =

(σ′2x + τ′2x )−1/2 0 −(σ′2x + τ′2x )−1/2c′x0 (σ′2y + τ′2y )−1/2 −(σ′2y + τ′2y )−1/2c′y0 0 1


Data Normalization

Normalized Image Points

Normalized noisy image points derived from mn and m′n

mn,N = TNmn

m′n,N = T′

Nm′n

n = 1, . . . , N

Normalized “true” image points

nn = Tnn

n′n = T ′n′nn = 1, 2, . . .


Data Normalization

Joint Image Points

Joint image points based on (mn, m′n) and (nn, n′n)

xn = [mn,x, mn,y, m′n,x, m′

n,y]T

yn = [nn,x, nn,y, n′n,x, n′n,y]T

Joint image points based on (mn,N, m′n,N) and (nn, n′n)

xn,N = [mn,N,x, mn,N,y, m′n,N,x, m′

n,N,y]T

yn = [nn,x, nn,y, n′n,x, n′n,y]T


Parameter Estimation

Outline

1 Introduction

2 Prerequisites




6 Bias


8 Conclusion



Moment matrices

Pre-NALS moment matrices

AN =1N

N

∑n=1

u(xn,N)u(xn,N)T

BN =1N

N

∑n=1

u(yn)u(yn)T

Normalizing factor N−1

Ensures statistical stabilityDoes not effect the eigenvectors



Eigenpairs

Eigenvectors and their corresponding eigenvalues

aj,N aj,N the jth eigenpair of AN

bj,N bj,N the jth eigenpair of BNj = 1, . . . , 9

All eigenvectors are

NormalizedArranged in descending order



Random Estimates

a9,N = vec(FTALS,N) b9,N = vec(FT

N) pre-NALS

FNALS,N = T′TNFALS,NTN FN = T ′TFNT NALS

FNALS,N is a typical estimate based on N noisycorrespondences

FN is a reference value against which to compare FNALS,N

If the {(nn, n′n)}Nn=1 were genuine correspondences bound by a

fundamental matrix FN, then FN would be a natural “true”value with which to gauge FNALS,NNo such FN exists, however.


Bias

Outline

1 Introduction

2 Prerequisites




6 Bias


8 Conclusion


Bias

Bias in NALS estimates

AN = {nn, n′n}Nn=1 first N “ideal” data points

E[FNALS,N|AN] the conditional expectationof FNALS,N given AN

DN = ‖E[FNALS,N|AN]− FN‖ bias in the FNALS,N estimates

DN captures the bias conditional on any particulararrangement of the “true” data points {nn, n′n}N

n=1.


Bias

Bias in Pre-NALS Estimates

Bias in the pre-NALS estimate

dN = ‖E[FALS,N|AN]− FN‖

Equivalent definition

dN = ‖E[a9,N|AN]− b9,N‖

Asymptotic behavior of DN is fully controlled by asymptoticbehavior of dN

limN→∞

DN = 0 if and only if limN→∞

dN = 0


Bias

Approximate Bias in Pre-NALS Estimates

dN is difficult to evaluate

No simple expression for a9,N

Use instead the approximate formula for aj,N

aj,N = bj,N +9

∑k=1k 6=j

bTk,N(AN − BN)bj,N

bj,N − bk,Nbk,N 1 ≤ j ≤ 9

Replace dN by

d∗N =

∥∥∥∥∥ 8

∑k=1

bTk,N(E[AN|AN]− BN)b9,N

b9,N − bk,Nbk,N

∥∥∥∥∥


Approximate Consistency

Outline

1 Introduction

2 Prerequisites




6 Bias


8 Conclusion




Main result

limN→∞

d∗N = 0 a.s.

Two main ingredients of proof

1

g = limN→∞

(b8,N − b9,N)−1 < ∞ a.s.

2

limn→∞

E[AN|AN] = I9 a.s.

Proof details


Conclusion

Outline

1 Introduction

2 Prerequisites




6 Bias


8 Conclusion


Conclusion

Summary

Hartley’s famous normalized eight-point algorithm

Has a novel statistical interpretationCan be placed more clearly within the spectrum of methodsavailable


Conclusion

Thank you!

The End


Appendix

Outline

9 Appendix


Appendix

Appendix Outline

9 Appendix


Appendix


d∗N =

∥∥∥∥∥ 8

∑k=1

bTk,N(E[AN|AN]− BN)b9,N

b9,N − bk,Nbk,N

∥∥∥∥∥d∗N ≤

8

∑k=1

|bTk,N(E[AN|AN]− BN)b9,N|

b9,N − bk,N‖bk,N‖

d∗N ≤8

∑k=1


b9,N − bk,N


Appendix


lim supN→∞

d∗N ≤ lim supN→∞

max1≤j≤8

1bj,N − b9,N

× lim supN→∞

8

∑k=1



Appendix


lim supN→∞

d∗N ≤ g lim supN→∞

8

∑k=1



Appendix


lim supN→∞

d∗N ≤ g lim supN→∞

8

∑k=1

|bTk,N(I9 − BN)b9,N|

bTk,N(I9 − BN)b9,N = (1− b9,N)bT

k,Nb9,N = 0

Return


Statistical Consistency of the Normalized Eight-Point...

Documents

Transcript of Statistical Consistency of the Normalized Eight-Point...