Complexity of direct methods n 1/2 n 1/3 2D3D Space (fill): O(n log n)O(n 4/3 ) Time (flops): O(n...
-
Upload
byron-gaines -
Category
Documents
-
view
217 -
download
2
Transcript of Complexity of direct methods n 1/2 n 1/3 2D3D Space (fill): O(n log n)O(n 4/3 ) Time (flops): O(n...
Complexity of direct methods
n1/2 n1/3
2D 3D
Space (fill): O(n log n) O(n 4/3 )
Time (flops): O(n 3/2 ) O(n 2 )
Time and space to solve any problem on any well-shaped finite element mesh
Complexity of linear solvers
2D 3DDense Cholesky: O(n3 ) O(n3 )
Sparse Cholesky: O(n1.5 ) O(n2 )
CG, exact arithmetic: O(n2 ) O(n2 )
CG, no precond: O(n1.5 ) O(n1.33 )
CG, modified IC0: O(n1.25 ) O(n1.17 )
CG, support trees: O(n1.20 ) -> O(n1+ ) O(n1.75 ) -> O(n1+ )
Multigrid: O(n) O(n)
n1/2n1/3
Time to solve model problem (Poisson’s equation) on regular mesh
Hierarchy of matrix classes (all real)
• General nonsymmetric
• Diagonalizable
• Normal
• Symmetric indefinite
• Symmetric positive (semi)definite = Factor width n
• Factor width k
. . .
• Factor width 4
• Factor width 3
• Symmetric diagonally dominant = Factor width 2
• Generalized Laplacian = Symmetric diagonally dominant M-matrix
• Graph Laplacian
Definitions
• The Laplacian matrix of an n-vertex undirected graph G is the n-by-n symmetric matrix A with
• aij = -1 if i ≠ j and (i, j) is an edge of G
• aij = 0 if i ≠ j and (i, j) is not an edge of G
• aii = the number of edges incident on vertex i
• Theorem: The Laplacian matrix of G is symmetric, singular, and positive semidefinite. The multiplicity of 0 as an eigenvalue is equal to the number of connected components of G.
• A generalized Laplacian matrix (more accurately, a symmetric weakly diagonally dominant M-matrix) is an n-by-n symmetric matrix A with
• aij ≤ 0 if i ≠ j
• aii ≥ Σ |aij| where the sum is over j ≠ i
Edge-vertex factorization of generalized Laplacians
• A generalized Laplacian matrix A can be factored as A = UUT, where U has:• a row for each vertex• a column for each edge, with two nonzeros of equal
magnitude and opposite sign• a column for each excess-weight vertex, with one nonzero
A U UT= ×
vertices
vertices
vertices
edges(2 nzs/col)
excess-weight
vertices(1 nz/col)
vertices
Support Graph Preconditioning
+: New analytic tools, some new preconditioners
+: Can use existing direct-methods software
-: Current theory and techniques limited
CFIM: Complete factorization of incomplete matrix
• Define a preconditioner B for matrix A
• Explicitly compute the factorization B = LU
• Choose nonzero structure of B to make factoring cheap
(using combinatorial tools from direct methods)
• Prove bounds on condition number using both
algebraic and combinatorial tools
Spanning Tree Preconditioner [Vaidya]
• A is generalized Laplacian (symmetric diagonally dominant with negative off-diagonal nzs)
• B is the gen Laplacian of a maximum-weight spanning tree for A (with diagonal modified to preserve row sums)
Form B: costs O(n log n) or less time (graph algorithms for MST)
Factorize B = RTR: costs O(n) space and O(n) time (sparse Cholesky)
Apply B-1: costs O(n) time per iteration
G(A) G(B)
Combinatorial analysis: cost of preconditioning
• A is generalized Laplacian (symmetric diagonally dominant with negative off-diagonal nzs)
• B is the gen Laplacian of a maximum-weight spanning tree for A (with diagonal modified to preserve row sums)
• Form B: costs O(n log n) time or less (graph algorithms for MST)
• Factorize B = RTR: costs O(n) space and O(n) time (sparse Cholesky)
• Apply B-1: costs O(n) time per iteration (two triangular solves)
G(A) G(B)
Numerical analysis: quality of preconditioner
• support each edge of A by a path in B
• dilation(A edge) = length of supporting path in B
• congestion(B edge) = # of supported A edges
• p = max congestion, q = max dilation
• condition number κ(B-1A) bounded by p·q (at most O(n2))
G(A) G(B)
Spanning Tree Preconditioner [Vaidya]
• can improve congestion and dilation by adding a few
strategically chosen edges to B
• cost of factor+solve is O(n1.75), or O(n1.2) if A is planar
• in experiments by Chen & Toledo, often better than
drop-tolerance MIC for 2D problems, but not for 3D.
G(A) G(B)
Numerical analysis: Support numbers
Intuition from networks of electrical resistors:
• graph = circuit; edge = resistor; weight = 1/resistance = conductance
• How much must you amplify B to provide as much conductance as A?
• How big does t need to be for tB – A to be positive semidefinite?
• What is the largest eigenvalue of B-1A ?
The support of B for A is
σ(A, B) = min { τ : xT(tB – A)x 0 for all x and all t τ }
• If A and B are SPD then σ(A, B) = max{λ : Ax = λBx} = λmax(A, B)
• Theorem: If A and B are SPD then κ(B-1A) = σ(A, B) · σ(B, A)
Old analysis, splitting into paths and edges
• Split A = A1+ A2 + ··· + Ak and B = B1+ B2 + ··· + Bk
• such that Ai and Bi are positive semidefinite
• Typically they correspond to pieces of the graphs of A and B
(edge, path, small subgraph)
• Theorem: σ(A, B) maxi {σ(Ai , Bi)}
• Lemma: σ(edge, path) (worst weight ratio) · (path length)
• In the MST case:
• Ai is an edge and Bi is a path, to give σ(A, B) p·q
• Bi is an edge and Ai is the same edge, to give σ(B, A) 1
Edge-vertex factorization of generalized Laplacians
• A generalized Laplacian matrix A can be factored as A = UUT, where U has:• a row for each vertex• a column for each edge, with two nonzeros of equal
magnitude and opposite sign• a column for each excess-weight vertex, with one nonzero
A U UT= ×
vertices
vertices
vertices
edges(2 nzs/col)
excess-weight
vertices(1 nz/col)
vertices
New analysis: Algebraic Embedding Lemma vv[Boman/Hendrickson]
Lemma: If V·W=U, then σ(U·UT, V·VT) ||W||22
(with equality for some choice of W)
Proof:
• take t ||W||22 = λmax(W·WT) = max y0 { yTW·WTy / yTy }
• then yT (tI - W·WT) y 0 for all y
• letting y = VTx gives xT (tV·VT - U·UT) x 0 for all x
• recall σ(A, B) = min{τ : xT(tB – A)x 0 for all x, all t τ}
• thus σ(U·UT, V·VT) ||W||22
A B
-a2
-b2
-a2 -c2
-b2
[ ]a2 +b2 -a2 -b2
-a2 a2 +c2 -c2
-b2 -c2 b2 +c2 [ ]a2 +b2 -a2 -b2
-a2 a2 -b2 b2
[ ] a b
-a c -b -c
[ ] a b
-a c -b
U V
=VVT=UUT
A B
-a2
-b2
-a2 -c2
-b2
[ ]a2 +b2 -a2 -b2
-a2 a2 +c2 -c2
-b2 -c2 b2 +c2 [ ]a2 +b2 -a2 -b2
-a2 a2 -b2 b2
[ ] a b
-a c -b -c
[ ] a b
-a c -b
U V
=VVT=UUT
[ ] 1 -c/a
1 c/b/b
W
= x
A edges
B e
dges
A B
-a2
-b2
-a2 -c2
-b2
[ ]a2 +b2 -a2 -b2
-a2 a2 +c2 -c2
-b2 -c2 b2 +c2 [ ]a2 +b2 -a2 -b2
-a2 a2 -b2 b2
[ ] a b
-a c -b -c
[ ] a b
-a c -b
U V
=VVT=UUT
σ(A, B) ||W||22 ||W|| x ||W||1
= (max row sum) x (max col sum)
(max congestion) x (max dilation)
[ ] 1 -c/a
1 c/b/b
W
= x
A edges
B e
dges
• Using another matrix norm inequality [Boman]:
||W||22 ||W||F2 = sum(wij
2) = sum of (weighted) dilations,
and [Alon, Karp, Peleg, West] construct spanning trees with
average weighted dilation exp(O((log n loglog n)1/2)) = o(n ).
This gives condition number O(n1+) and solution time O(n1.5+),compared to Vaidya’s O(n1.75) with augmented MST.
• Is there a graph construction that minimizes ||W||22 directly?
• [Spielman, Teng]: complicated recursive partitioning construction
with solution time O(n1+) for all generalized Laplacians! (Uses yet another matrix norm inequality.)
Extensions, remarks, open problems I
Extensions, remarks, open problems II
• Make spanning tree methods more effective in 3D?• Vaidya gives O(n1.75) in general, O(n1.2) in 2D.• Issue: 2D uses bounded excluded minors, not just separators.
• Support theory methods for more general matrices?• [Boman, Chen, Hendrickson, Toledo]: different matroid for all
symmetric diagonally dominant matrices (= factor width 2).• Matrices of bounded factor width? Factor width 3?• All SPD matrices?
• Is there a version that’s useful in practice?• Maybe for non-geometric graph Laplacians?• [Koutis, Miller, Peng 2010] simplifies Spielman/Teng a lot.• [Kelner et al. 2013] : random Kaczmarz projections in the
dual space – even simpler, good O() theorems, but not yet fast enough in practice.
Support-graph analysis of modified incomplete Cholesky
• B has positive (dotted) edges that cancel fill• B has same row sums as A
Strategy: Use the negative edges of B to support both the negative edges of A and the positive edges of B.
-1 -1 -1
-1 -1 -1
-1 -1 -1
-1 -1 -1
-1-1 -1-1
-1-1 -1-1
-1-1 -1-1
A
A = 2D model Poisson problem
.5 .5 .5
.5 .5 .5
.5 .5 .5
-1 -1 -1
-1 -1 -1
-1 -1 -1
-1 -1 -1
-1-1 -1-1
-1-1 -1-1
-1-1 -1-1
B
B = MIC preconditioner for A
Supporting positive edges of B
• Every dotted (positive) edge in B is supported by two paths in B
• Each solid edge of B supports one or two dotted edges
• Tune fractions to support each dotted edge exactly
• 1/(2n – 2) of each solid edge is left over to support an edge of A
Analysis of MIC: Summary
• Each edge of A is supported by the leftover 1/(2n – 2) fraction of the same edge of B.
• Therefore σ(A, B) 2n – 2
• Easy to show σ(B, A) 1
• For this 2D model problem, condition number is O(n1/2)
• Similar argument in 3D gives condition number O(n1/3) or O(n2/3) (depending on boundary conditions)