7/29/2019 Sparse Day 1 LU FACTOR

1/17

Sparse Matr ix Methods

Day 1: Overview

Matlab and examples

Data structures

Ax=b Sparse matrices and graphs

Fill-reducing matrix permutations

Matching and block triangular form

Day 2:Direct methods

Day 3: Iterative methods

D

7/29/2019 Sparse Day 1 LU FACTOR

2/17

(Demo in Matlab)

Matlab sprand

spy

sparse, full

matrix arithmetic and indexing

examples of sparse matrices from different

applications (from UF site)

7/29/2019 Sparse Day 1 LU FACTOR

3/17

Matlab sparse matr ices: Design pr inc ip les

All operations should give the same results forsparse and full matrices (almost all)

Sparse matrices are never created automatically,but once created they propagate

Performance is important -- but usability, simplicity,completeness, and robustness are more important

Storage for a sparse matrix should be O(nonzeros)

Time for a sparse operation should be O(flops)(as nearly as possible)

7/29/2019 Sparse Day 1 LU FACTOR

4/17

Data structu res

Full:

2-dimensional array ofreal or complex numbers

(nrows*ncols) memory

31 0 53

0 59 0

41 26 0

31 41 59 26 53

1 3 2 3 1

Sparse:

compressed columnstorage

about (1.5*nzs + .5*ncols)memory

D

7/29/2019 Sparse Day 1 LU FACTOR

5/17

(Demos in Matlab)

A \ b Cholesky factorization and orderings

7/29/2019 Sparse Day 1 LU FACTOR

6/17

Solving l inear equat ion s

x = A \ b;

Is A square?

no => use QR to solve least squares problem

Is A triangular or permuted triangular?yes => sparse triangular solve

Is A symmetric with positive diagonal elements?

yes => attempt Cholesky after symmetric minimum degree

Otherwise

=> use LU on A(:, colamd(A))

7/29/2019 Sparse Day 1 LU FACTOR

7/17

Matr ix facto r izat ion s in Matlab

Cholesky: R = chol(A);

simple left-looking column algorithm

Nonsymmetric LU:

[L,U,P] = lu(A);

left-looking GPMOD, depth-first search, symmetric pruning

Orthogonal:

[Q,R] = qr(A);

George-Heath algorithm: row-wise Givens rotations

7/29/2019 Sparse Day 1 LU FACTOR

8/17

Graphs and Sparse Matr ices: Cholesky factor izat ion

10

13

2

4

5

6

7

8

9

10

13

2

4

5

6

7

8

9

G(A) G+(A)[chordal]

Symmetric Gaussian elimination:

for j = 1 to n

add edges between js

higher-numbered neighbors

Fill:new nonzeros in factor

7/29/2019 Sparse Day 1 LU FACTOR

9/17

Elim inat ion Tree

10

13

2

4

5

6

7

8

9

Cholesky factor G+(A) T(A)

10

1

3

2

45

6

7

8

9

T(A) : parent(j) = min { i > j : (i,j) inG+(A) }

T describes dependencies among columns of factor

Can compute T from G(A) in almost linear time

Can compute G+(A) easily from T

D

7/29/2019 Sparse Day 1 LU FACTOR

10/17

(Demos in Matlab)

matrix and graph elimination tree

orderings in detail

7/29/2019 Sparse Day 1 LU FACTOR

11/17

Fil l -reduc ing m atr ix permutat ions

Minimum degree:

Eliminate row/col with fewest nzs, add fill, repeat Theory: can be suboptimal even on 2D model problem

Practice: often wins for medium-sized problems

Nested dissection:

Find a separator, number it last, proceed recursively Theory: approx optimal separators => approx optimal fill and flop count

Practice: often wins for very large problems

Banded orderings (Reverse Cuthill-McKee, Sloan, . . .):

Try to keep all nonzeros close to the diagonal

Theory, practice: often wins for long, thin problems

Best modern general-purpose orderings are ND/MD hybrids.

7/29/2019 Sparse Day 1 LU FACTOR

12/17

Fil l -reduc ing permutat ions in Matlab

Nonsymmetric approximate minimum degree:

p = colamd(A); column permutation: lu(A(:,p)) often sparser than lu(A)

also for QR factorization

Symmetric approximate minimum degree:

p = symamd(A);

symmetric permutation: chol(A(p,p)) often sparser than chol(A)

Reverse Cuthill-McKee

p = symrcm(A);

A(p,p) often has smaller bandwidth than A

similar to Sparspak RCM

D

7/29/2019 Sparse Day 1 LU FACTOR

13/17

(Demos in Matlab)

nonsymmetric LU dmperm, dmspy, components

7/29/2019 Sparse Day 1 LU FACTOR

14/17

Matching and block tr iangular form

Dulmage-Mendelsohn decomposition:

Bipartite matching followed by strongly connected components

Square, full rank A:

[p, q, r] = dmperm(A);

A(p,q) has nonzero diagonal and is in block upper triangular form also, strongly connected components of a directed graph

also, connected components of an undirected graph

Arbitrary A:

[p, q, r, s] = dmperm(A); maximum-size matching in a bipartite graph

minimum-size vertex cover in a bipartite graph

decomposition into strong Hall blocks

7/29/2019 Sparse Day 1 LU FACTOR

15/17

Complexi ty of di rect methods

2D 3D

Space (fill): O(n log n) O(n 4/3 )

Time (flops): O(n 3/2 ) O(n 2 )

n1/2n1/3

7/29/2019 Sparse Day 1 LU FACTOR

16/17

The Landscape of Sparse Ax=b Solvers

Pivoting

LU

GMRES,

QMR,

Cholesky Conjugate

gradient

DirectA = LU

Iterativey = Ay

Non-

symmetric

Symmetric

positive

definite

More Robust Less Storage

More Robust

More General

D

7/29/2019 Sparse Day 1 LU FACTOR

17/17

(Demos in Matlab)

conjugate gradients