Inexact inverse iteration and tuned preconditioning

OutlineMotivation

Inexact Inverse IterationHermitian problems and preconditioning

Nonhermitian problems and preconditioning

Melina Freitag

Department of Mathematical SciencesUniversity of Bath

22nd GAMM-Seminar Leipzig onLarge Scale Eigenvalue Computations

19th January 2006

Joint work with: Alastair Spence, Eero Vainikko

Melina Freitag Inexact inverse iteration and tuned preconditioning

OutlineMotivation

1 Motivation

2 Inexact Inverse Iteration

3 Hermitian problems and preconditioning

4 Nonhermitian problems and preconditioning

OutlineMotivation

Problem and Inverse Iteration

Find an eigenvalue and eigenvector of s.p.d. A:

Ax = λx,

Inverse Iteration:(A − σI)y = x

A large, sparse.

λ λ λλ 4 λn

Inverse iteration with preconditioned iterative solves

OutlineMotivation

Convergence ratesMINRES

Inexact Inverse Iteration

for i = 1 to . . . do

choose τ (i), σ(i)

solve‖(A − σ(i)I)y(i) − x(i)‖ ≤ τ (i),

Rescale x(i+1) =y(i)

‖y(i)‖,

Update λ(i+1) = x(i+1)T

Ax(i+1),possibly: update the shift σ(i)

Test: eigenvalue residual r(i+1) = (A − λ(i+1)I)x(i+1).end for

OutlineMotivation

Error indicator

Error indicator (orthogonal decomposition for symmetric A, Parlett)

(i) (i)

θ (i)

Q x = O (sin ) measure for the error

Eigenvalue residual

C| sin θ(i)| ≤ ‖r(i)‖ ≤ C ′| sin θ(i)|

Note: similar for nonsymmetric case if generalised sin θ is used.

OutlineMotivation

Error indicator

Error indicator (orthogonal decomposition for symmetric A, Parlett)

(i) (i)

θ (i)

Q x = O (sin ) measure for the error

Eigenvalue residual

C| sin θ(i)| ≤ ‖r(i)‖ ≤ C ′| sin θ(i)|

Note: similar for nonsymmetric case if generalised sin θ is used.

OutlineMotivation

Convergence rates of inexact inverse iteration

Error in θ (eigenvector)

tan θ(i+1) ≤|λ1 − σ(i)|

|λ2 − σ(i)|T (i)

Exact solves: T (i) = tan θ(i)

Inexact solves T (i) =| sin θ(i) + C1τ

| cos θ(i) − C2τ(i)|

Choice of τ

Choice 1: τ (i) = C‖r(i)‖ = O(sin θ(i)) ⇒ T (i) = O(tan θ(i))

Choice 2: τ (i) = constant ⇒ T (i) = O(1)

OutlineMotivation

Error in θ (eigenvector)

tan θ(i+1) ≤|λ1 − σ(i)|

|λ2 − σ(i)|T (i)

Exact solves: T (i) = tan θ(i)

Inexact solves T (i) =| sin θ(i) + C1τ

| cos θ(i) − C2τ(i)|

Choice of τ

Choice 1: τ (i) = C‖r(i)‖ = O(sin θ(i)) ⇒ T (i) = O(tan θ(i))

Choice 2: τ (i) = constant ⇒ T (i) = O(1)

OutlineMotivation

Decreasing tolerance τ (i) ≤ C‖r(i)‖ = O(sin θ(i))

1 For decreasing tolerance τ (i) ≤ C‖r(i)‖ = O(sin θ(i)) the inexactmethod recovers the rate of convergence achieved by exact solves.

2 Fixed shift σ: linear convergence. [see Golub/Ye 2000,Berns-Muller/Graham/Spence 2005]

3 Rayleigh quotient shift σ(i) = ρ(x(i)) = x(i)T

x(i)Tx(i)

: cubic

convergence for A = A∗. [see Smit/Paardekooper 1999,Berns-Muller/Graham/Spence 2005]

OutlineMotivation

MINRES (A − σI)y = x when A is symmetric

Solving a linear system (A − σI)y = x

MINRES:

‖x − (A − σI)y‖ ≤ 2

κ − 1

κ + 1

)k−1

‖x‖.

If ‖x − (A − σI)y‖ ≤ τ then

k ≥ 1 + κ

log 2 + log‖x‖

where κ is the condition number of A − σI .

OutlineMotivation

MINRES (A − σI)y = x when A is symmetric

Solving a linear system (A − σI)y = x

MINRES:

‖x − (A − σI)y‖ ≤ 2

κ − 1

κ + 1

)k−1

‖x‖.

If ‖x − (A − σI)y‖ ≤ τ then

k ≥ 1 + κ

log 2 + log‖x‖

where κ is the condition number of A − σI .

OutlineMotivation

Unpreconditioned solves with MINRES

Convergence rates for solves with MINRES for simple eigenvalue

If A is positive definite and has a simple eigenvalue then

‖x(i) − (A − σI)y(i)k‖2 ≤ 2

|λ1 − λn|

|λ1 − σ|

κ1 − 1

κ1 + 1

)k−1

‖Qx(i)‖2.

where Q is the orthogonal projection onto span{x2, . . . , xn} and κ1 is

the reduced condition number κ1 =maxi=2,...,n |λi−σ|mini=2,...,n |λi−σ| .

Number of inner solves for each i for ‖x(i) − (A − σ(i)I)y(i)‖ ≤ τ (i)

k(i) ≥ C1 + C2 log

‖Qx(i)‖2

|λ1 − σ|τ (i)

OutlineMotivation

‖x(i) − (A − σI)y(i)k‖2 ≤ 2

|λ1 − λn|

|λ1 − σ|

κ1 − 1

κ1 + 1

)k−1

‖Qx(i)‖2.

k(i) ≥ C1 + C2 log

‖Qx(i)‖2

|λ1 − σ|τ (i)

OutlineMotivation

‖x(i) − (A − σI)y(i)k‖2 ≤ 2

|λ1 − λn|

|λ1 − σ|

κ1 − 1

κ1 + 1

)k−1

| sin θ(i)|.

k(i) ≥ C1 + C2 log

| sin θ(i)|

|λ1 − σ|τ (i)

OutlineMotivation

Number of inner solves for each i

k(i) ≥ C1 + C2 log

| sin θ(i)|

|λ1 − σ|τ (i)

Example

For fixed shift σ, and τ (i) ≤ C‖r(i)‖ = O(sin θ(i)) the number of innersolves k(i) for each i does not increase with i

OutlineMotivation

PreconditioningTuning the preconditionerNumerical ResultsPerturbation theory

Preconditioning

Incomplete Cholesky preconditioning

A = LLT + E

symmetric preconditioning of (A − σI)y(i) = x(i):

L−1(A − σI)L−T y(i) = L−1x(i), y(i) = L−T y(i)

Remarks

1 changes number of inner iterations

k(i) ≥ C1 + C2 log

‖L−1‖

|λ1 − σ|τ (i)

2 k(i) increases with i for τ (i) ≤ C‖r(i)‖.

OutlineMotivation

Preconditioning

A = LLT + E

Remarks

k(i) ≥ C1 + C2 log

‖L−1‖

|λ1 − σ|τ (i)

OutlineMotivation

Preconditioning

A = LLT + E

Remarks

k(i) ≥ C1 + C2 log

‖L−1‖

|λ1 − σ|τ (i)

OutlineMotivation

Derivation

1 modify L → L

L−1(A − σI)L−T y(i) = L

−1x(i), y(i) = L−T y(i)

2 minor extra computation cost for L

3 ”nice” RHS L−1x(i) (same behaviour as unpreconditioned solves,

e.g. for fixed shifts k(i) does not increase with i)

Condition

MINRES theory indicates that L−1x(i) should be close to

eigenvector of L−1(A − σI)L−T

Holds if LLT x(i) = Ax(i).

OutlineMotivation

Derivation

1 modify L → L

L−1(A − σI)L−T y(i) = L

−1x(i), y(i) = L−T y(i)

Condition

OutlineMotivation

Derivation

1 modify L → L

L−1(A − σI)L−T y(i) = L

−1x(i), y(i) = L−T y(i)

Condition

OutlineMotivation

Derivation

1 modify L → L

L−1(A − σI)L−T y(i) = L

−1x(i), y(i) = L−T y(i)

Condition

OutlineMotivation

Derivation

1 modify L → L

L−1(A − σI)L−T y(i) = L

−1x(i), y(i) = L−T y(i)

Condition

OutlineMotivation

Choice of L

Justification of LLT x(i) = Ax(i)

If x(i) = x1 then LLT x1 = λ1x1

L−1(A − σI)L−T

L−1x1 =

λ1 − σ

λ1L−1x1

Theorem

With e(i) = Ax(i) − LLT x(i) (known) and L chosen such that

L = L + α(i)e(i)e(i)T

with α(i) root of quadratic function we get LLT x(i) = Ax(i).

L is a rank-one update of L.

OutlineMotivation

Choice of L

Justification of LLT x(i) = Ax(i)

If x(i) = x1 then LLT x1 = λ1x1

L−1(A − σI)L−T

L−1x1 =

λ1 − σ

λ1L−1x1

Theorem

With e(i) = Ax(i) − LLT x(i) (known) and L chosen such that

L = L + α(i)e(i)e(i)T

with α(i) root of quadratic function we get LLT x(i) = Ax(i).

L is a rank-one update of L.

OutlineMotivation

Convergence rates

The tuned preconditioner

1 retains outer convergence rates

2 provides cheap inner solves

k(i) ≥ C1 + C2 log

γ| sin θ(i)|

|λ1 − σ|τ (i)

3 only a single extra back substitution with L per outer iterationneeded

OutlineMotivation

Example

SPD matrix from the Matrix Market library (nos5: 3 story buildingwith attached tower)

seek eigenvalue near fixed shift σ = 100

A ≈ LLT , incomplete Cholesky factorisation (drop tol. = 0.1)

compare standard and tuned preconditioner

OutlineMotivation

Fixed shift solves

Preconditioning with standard incomplete Cholesky

0 5 10 15 20 25 30 3510

outer iterations

Standard preconditioner

Tuned preconditioner

total number of inner iterations using standard preconditioner: 2026

total number of inner iterations using tuned preconditioner: 779

OutlineMotivation

Comparison of LLT with LLT

L−1(A − σI)L−T eigenvalues µi

L−1(A − σI)L−T eigenvalues ξi

−4 −3 −2 −1 0 1 2 3 4 5

f2(ξ)

f1(ξ)

µ1 µ

Clustering properties preserved

κ1L≤ κ1

L≤ κ1

L(1 + γvT v) [see F/Spence 2005]

OutlineMotivation

Comparison of LLT with LLT

L−1(A − σI)L−T eigenvalues µi

L−1(A − σI)L−T eigenvalues ξi

−4 −3 −2 −1 0 1 2 3 4 5

f2(ξ)

f1(ξ)

µ1 µ

Clustering properties preserved

κ1L≤ κ1

L≤ κ1

L(1 + γvT v) [see F/Spence 2005]

OutlineMotivation

Right preconditioningTuning the preconditionerNumerical Results

Nonhermitian generalised eigenproblems

Extention toAx = λMx

Application: Stability theory for linearised Navier-Stokes equations

Inverse iteration with preconditioned GMRES

Tuning for nonsymmetric problems

OutlineMotivation

Standard right preconditioning for (A−σ(i)M)y(i) = Mx(i)

GMRES (see Berns-Muller/Spence 2005)

Let P be a preconditioner then

(A − σ(i)M)P−1y(i) = Mx(i), with y(i) = P−1y(i)

k(i) ≥ C1 + C2 log

‖QMx(i)‖

|λ1 − σ(i)|τ (i)

where ‖QMx(i)‖ = O(1) hence k(i) increases with i

How can we get ‖QMx(i)‖ = O(sin θ(i))?

To obtain ‖QMx(i)‖ = O(sin θ(i)) Mx(i) should be close to eigenvectorof (A − σ(i)M)P−1

OutlineMotivation

Standard right preconditioning for (A−σ(i)M)y(i) = Mx(i)

GMRES (see Berns-Muller/Spence 2005)

Let P be a preconditioner then

(A − σ(i)M)P−1y(i) = Mx(i), with y(i) = P−1y(i)

k(i) ≥ C1 + C2 log

‖QMx(i)‖

|λ1 − σ(i)|τ (i)

where ‖QMx(i)‖ = O(1) hence k(i) increases with i

How can we get ‖QMx(i)‖ = O(sin θ(i))?

To obtain ‖QMx(i)‖ = O(sin θ(i)) Mx(i) should be close to eigenvectorof (A − σ(i)M)P−1

OutlineMotivation

Tuning the preconditioner

Right preconditioned system (A − σ(i)M)P−1y(i) = Mx(i)

Preconditioned iterates should behave as in unpreconditioned case

In practice, choose

Px(i) = Ax(i)

Then‖QMx(i)‖ = O(| sin θ(i)|).

which recovers the same behaviour as for unpreconditioned standardeigenvalue problem

Implementation of Px(i) = Ax(i)

Evaluate u(i) = Ax(i) − Px(i)

Rank-one update: P = P + u(i)x(i)H

satisfies Px(i) = Ax(i)

Use Sherman-Morrison - one extra solve per outer iteration

OutlineMotivation

In practice, choose

Px(i) = Ax(i)

OutlineMotivation

In practice, choose

Px(i) = Ax(i)

OutlineMotivation

In practice, choose

Px(i) = Ax(i)

OutlineMotivation

In practice, choose

Px(i) = Ax(i)

OutlineMotivation

Numerical Example

Flow past a circular cylinder (incompressible Navier-Stokes)

Re = 25, λ ≈ ±10i

Mixed FEM: Q2 − Q1 elements

Elman preconditioner: 2-level additive Schwarz

≈ 54000 degrees of freedom

OutlineMotivation

Example

Figure: Fixed Shift

0 5 10 15 20 25100

Untuned preconditioner, 10975 iterations

Figure: Rayleigh Quotient Shift

1 2 3 4 5 6 7 8100

OutlineMotivation

Example

Figure: Fixed Shift

0 5 10 15 20 25100

Tuned preconditioner, 5800 iterations

Figure: Rayleigh Quotient Shift

1 2 3 4 5 6 7 8100

Tuned preconditioner, 2667 iterations

OutlineMotivation

M. A. Freitag and A. Spence, Convergence rates for inexactinverse iteration with application to preconditioned iterative solves,2005.Submitted to BIT.

, A tuned preconditioner for inexact inverse iteration applied toHermitian eigenvalue problems, 2005.Submitted to IMA J. Numer. Anal.

Inexact inverse iteration and tuned preconditioning

Documents

Transcript of Inexact inverse iteration and tuned preconditioning