Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech...
-
Upload
gyles-mcbride -
Category
Documents
-
view
214 -
download
1
Transcript of Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech...
![Page 1: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/1.jpg)
Center for Machine PerceptionDepartment of Cybernetics, Faculty of Electrical Engineering
Czech Technical University in Prague
MAKING MINIMAL SOLVERS FAST
Zuzana Kukelova, Martin Bujnak, Tomas Pajdla
![Page 2: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/2.jpg)
2/24Motivation
Recognition & Tracking
Placeholder text
Placeholder text
3D Reconstruction
Augmented reality
Robotics
Require fast solvers
![Page 3: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/3.jpg)
3/24Motivation
Systems of polynomial equations
Camera geometry problems - calibration, relative/absolute pose estimation
![Page 4: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/4.jpg)
4/24
PropertiesProperties RequirementsRequirements
all “non-degenerate” instances of the problem result in systems of equations of the same form - consisting of the same polynomials and differing only in coefficients
all “non-degenerate” instances of the problem result in systems of equations of the same form - consisting of the same polynomials and differing only in coefficients
Contaminated input measurements => problems have to be solved for many different inputs (RANSAC)
High or even real-time performance
Contaminated input measurements => problems have to be solved for many different inputs (RANSAC)
High or even real-time performance
Solve many instances of the same problem, of the same system of polynomial equations only with different coefficients, very fast
Co
nc
lus
ion
Co
nc
lus
ion
Camera geometry problems
![Page 5: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/5.jpg)
5/24Complicated systems5-point relative pose problem
f 2 = 4:68x +6:62y+2:25z + 1:81+ 6:68y2 ¡ 0:806z2 +12:6xy+ 4:03xz +6:99yz +1:01x3+2:04y3 ¡ 0:873z3+6:13xyz +5:53x2y+1:94x2z +6:95xy2 ¡1:13xz2+4:27y2z ¡ 0:130yz2+4:53x2
f 3 = 0:866x+2:28y+3:10z+3:50y2+3:04z2+4:15xy+6:07xz+8:09yz ¡0:076x3+1:34y3+0:497z3+8:03xyz+3:05x2y+1:88x2z+4:69xy2+3:40xz2+5:24y2z +4:21yz2+0:402+0:199x2
f 10 = 5:35x +4:46y+ 3:06z + 3:85y2 +1:48+ 2:57z2 +9:21xy+ 8:05xz +7:16yz +1:41x3+0:991y3+1:09z3+8:06xyz +4:06x2y+4:06x2z +3:83xy2+3:50xz2+3:68y2z+2:60yz2+5:72x2
f 1 = 0:058x +0:331y ¡ 0:671z + 0:698y2 ¡ 0:813z2 ¡ 0:057xy ¡ 0:036xz +0:653yz + 0:534x3 ¡ 0:167y3 ¡ 0:323z3 + 0:633xyz ¡ 0:282x2y + 0:770x2z ¡0:974xy2 ¡ 0:146xz2+0:584y2z +0:239yz2 ¡ 0:170+0:593x2
: : :
![Page 6: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/6.jpg)
6/24General methodsExisting software – well known general Grőbner basis or resultant algorithms
Reliablebut
SLOW!
![Page 7: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/7.jpg)
7/24Specific methods• Design special algorithms to achieve numerical robustness and
computational efficiency
• Not general
• Fast !!!
• Intersection ellipse-line• No need to use general algorithms• Closed form solution
![Page 8: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/8.jpg)
8/24
Online solver• Efficient solver for
solving systems of polynomial equations of one form
• Not general but fast
• This is what the user call
Offline phase• Study the problem • Solve it in some finite
prime field• Design a specific
efficient solver• Needs to be
performed only once
Specific methods
![Page 9: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/9.jpg)
9/24Grőbner basis specific methods
Parameters identification• Number of solutions, basis• Computations in finite prime field
- fast, stable
Elimination template design• Which polynomials
should be multiplied with which monomials and eliminated to obtain the solution
G-J elimintation of the found template• With concrete
coefficients
Eigenvalue computations• Computation of
eigenvalues and eigenvactors of the action matrix created from the eliminated template
![Page 10: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/10.jpg)
10/24Offline phase
25min15min
Offline phase & Elimination template – hard to understand and implement
Requires knowledge from algebraic geometry
1+1=?
ideal
mul
tiplic
atio
n m
atrix
Quotient ring
xy2+7y+..
field
![Page 11: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/11.jpg)
11/24Automatic generator
Final “online” solver
List of known coeffs
List of unknowns
Equations
Offline phase
![Page 12: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/12.jpg)
12/24How to make the final solver fast
Reduce the size of the elimination template• Kukelova Z., Bujnak M.,
Pajdla T., Automatic Generator of Minimal Problem Solvers, ECCV 2008, Marseille, France, October 12-18, 2008.
Sparse G-J elimintation of the found template
The slowest part • Idea: Exchange it for
computation of roots of a single-variable polynomial using Sturm-sequences
• Bujnak M., Kukelova Z., Pajdla T., Making Minimal Solvers Fast, CVPR 2012.
![Page 13: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/13.jpg)
13/24Grőbner bases of the ideal• System of input polynomial equations
• The ideal generated by is a set of all polynomials that can be generated as polynomial combinations of
f 1; : : : ; fm
where are arbitrary polynomials from
I = fP m
i=1 f i hi : hi 2 C [x1; :::;xn]g
f 1; : : : ; fm
hi
I
𝒇 𝟏
𝒇 𝟐
𝒇 𝒎
+ …
I
![Page 14: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/14.jpg)
14/24Grőbner bases of the ideal
𝒇 𝟏
𝒇 𝟐
𝒇 𝒎
+ …
I𝒈𝟏
𝒈𝟐
𝒈𝒎
+ + …
I
• An ideal can be generated by many different sets of generators which all share the same solutions
¿
• Grőbner bases - special bases• Useful in solving system of polynomial equations
![Page 15: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/15.jpg)
15/24
Lexicographic GBLexicographic GB Graded Reverse Lex GBGraded Reverse Lex GB
Contains polynomial in one
variable Very expensive to compute Not feasible for computer
vision problems
Contains polynomial in one
variable Very expensive to compute Not feasible for computer
vision problems
Doesn’t contain a single variable polynomial
Special multiplication matrix – eigenvalues & eigenvectors give solutions
Easier to compute Used to solve CV problems
Doesn’t contain a single variable polynomial
Special multiplication matrix – eigenvalues & eigenvectors give solutions
Easier to compute Used to solve CV problems
Grőbner bases of the ideal
g1 (x1)
g2 (x1;x2)
: : :
gn (x1; : : : ;xn)
q1 (x1; : : : ;xn)
q2 (x1; : : : ;xn)
: : :
ql (x1; : : :;xn)
![Page 16: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/16.jpg)
16/24
• We have the Graded Reverse Lexicographic GB and multiplication (action) matrix
1. FGLM conversion algorithmW
e h
av
eW
e h
av
e
• We want the variable polynomial from the Lexicographic GB
We
wa
nt
We
wa
nt
• Well known FGLM algorithm for converting grevlex GB to Lexicographic GB
• Needs polynomial division• Doesn’t bring a speed up over eigenvalue computations
So
luti
on
So
luti
on
q1 (x1; : : : ;xn)
q2 (x1; : : : ;xn)
: : :
ql (x1; : : :;xn)
g1 (x1)
![Page 17: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/17.jpg)
17/24
• Replace time consuming polynomial division performed in standard FGLM algorithm with efficient matrix-vector multiplication using the action matrix
New Matrix FGLM algorithmT
ric
kT
ric
k
• New Matrix FGLM algorithm• To obtain a single variable polynomial for a system with solutions
• matrix-vector multiplications• Solve system of linear equations in unknowns
• Solve single variable polynomial using efficient Sturm-sequences
Alg
ori
thm
Alg
ori
thm
• Significant speedup over the eigenvalue computation• Only feasible solutions – positive feasible focal lengths, depths or radial
distortion coefficientsPo
sit
ive
sP
os
itiv
es
![Page 18: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/18.jpg)
18/24
• We have a multiplication (action) matrix for multiplying with some variable • Eigenvalues of this action matrix give solutions to
2. Characteristic polynomial methodW
e h
av
eW
e h
av
e
• We want a single variable polynomial
We
wa
nt
We
wa
nt
• Characteristic polynomial of the action matrix • Krylov’s method• Faddeev-Leverrier method• Danilevsky method
So
luti
on
So
luti
on
g(x)
![Page 19: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/19.jpg)
19/242. Characteristic polynomial method
• Faddeev-Leverrier method• Well-known method, compute traces of matrices• Suffers from a large numerical instabilityF
ad
de
ev
Fa
dd
ee
v
• Danilevsky method• Transforms input matrix to its companion matrix by s − 1 similarity
transformations• very efficient and numerically stable
So
luti
on
So
luti
on
• Krylov’s method• Based on Cayley-Hamilton theorem• Equivalent to the proposed matrix FGLM algorithm• Computes - for some matrices may be unstable
Kry
lov
Kry
lov
![Page 20: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/20.jpg)
20/24
• 5-point relative pose problem
• Nister – SOTA “closed form solution”• GB+eig – standard Grőbner basis solution with eigenvalue
computation
• New “single variable polynomial”solutions – • ~5x speedup over GB+eig• Comparable to the closed form solution
Speedup over the eigenvalue method
Danilevsky
• 14.2μs
mFGML
• 13.7μs
Faddeev
• 17.2μs
GB+eig
• 61.2μs
Nister
• 10.6μs
![Page 21: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/21.jpg)
21/24
• 6-point focal length relative pose problem
• New “single variable polynomial”solutions • ~8x speedup over GB+eig
Speedup over the eigenvalue method
Danilevsky
• 22.6μs
mFGML
• 21.3μs
GB+eig
• 176.3μs
![Page 22: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/22.jpg)
22/24
• P4P+f absolute pose problem
• New “single variable polynomial”solutions – • ~3x speedup over GB+eig
Speedup over the eigenvalue method
Danilevsky
• 47.4μs
mFGML
• 46.2μs
Faddeev
• 51.2μs
Sparse GB
• 82.9μs
GB+eig
• 127.4μs
![Page 23: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/23.jpg)
23/24
• Many problems in computer vision and other fields require fast solvers of systems of polynomial equations
• General methods – not feasible• Specific solvers – not general but fast
Conclusion
• Automatic generator of Grőbner basis solvers
• New methods for speeding up Grőbner basis solvers• Matrix FGLM method + Sturm sequences• Characteristic polynomial method + Sturm sequences
• Significant speed up over existing GB solvers
![Page 24: Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c90ed/html5/thumbnails/24.jpg)
24/24http://cmp.felk.cvut.cz/minimal
THANK YOU FOR YOUR ATTENTION