Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers...

Numerical behavior of inexact saddle point solvers

Pavel Jiranek1,2, Miroslav Rozloznık1,2

Faculty of Mechatronics and Interdisciplinary Engineering Studies,Technical University of Liberec,

Czech Republic1

and

Institute of Computer Science,Czech Academy of Sciences, Prague,

Czech Republic2

9th IMACS International Symposiumon Iterative Methods in Scientific Computings

March 17–20, 2008, Lille, France

1 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers

Saddle point problems

We consider a saddle point problem with the symmetric 2× 2 block form„A BBT 0

«„xy

«=

„f0

«.

A is a square n× n nonsingular (symmetric positive definite) matrix,

B is a rectangular n×m matrix of (full column) rank m.

Applications: mixed finite element approximations, weighted least squares,constrained optimization etc. [Benzi, Golub, and Liesen, 2005].


inexact solutions of inner systems + rounding errors→ inexact saddle point solver


Schur complement reduction method

Compute y as a solution of the Schur complement system

BTA−1By = BTA−1f,

compute x as a solution of

Ax = f −By.

Systems with A are solved inexactly, the computed solution u of Au = b isinterpreted an exact solution of a perturbed system

(A+ ∆A)u = b+ ∆b, ‖∆A‖ ≤ τ‖A‖, ‖∆b‖ ≤ τ‖b‖, τκ(A)� 1.


Iterative solution of the Schur complement system

choose y0, solve Ax0 = f −By0

compute αk and p(y)k

yk+1 = yk + αkp(y)k˛

˛˛˛

solve Ap(x)k = −Bp(y)

k

back-substitution:

A: xk+1 = xk + αkp(x)k ,

B: solve Axk+1 = f −Byk+1,

C: solve Auk = f −Axk −Byk+1,

xk+1 = xk + uk.

9>>>>>>>>>=>>>>>>>>>;inneriteration

r(y)k+1 = r

(y)k − αkB

T p(x)k

9>>>>>>>>>>>>>>>>>>>>>=>>>>>>>>>>>>>>>>>>>>>;

outeriteration


Measure of the limiting accuracy

The limiting (maximum attainable) accuracy is measured by the ultimate(asymptotic) values of:

1 the Schur complement residual: BTA−1f −BTA−1Byk;

2 the residuals in the saddle point system: f −Axk −Byk and −BTxk;

3 the forward errors: x− xk and y − yk.

Numerical example:

A = tridiag(1, 4, 1) ∈ R100×100, B = rand(100, 20), f = rand(100, 1),

κ(A) = ‖A‖ · ‖A−1‖ = 7.1695 · 0.4603 ≈ 3.3001,

κ(B) = ‖B‖ · ‖B†‖ = 5.9990 · 0.4998 ≈ 2.9983.


Accuracy in the outer iteration process

BT (A+ ∆A)−1By = BT (A+ ∆A)−1f,

‖BTA−1f −BTA−1By‖ ≤ τκ(A)

1− τκ(A)‖A−1‖‖B‖2‖y‖.

‖ −BTA−1f +BTA−1Byk − r(y)k ‖ ≤

O(τ)κ(A)

1− τκ(A)‖A−1‖‖B‖(‖f‖+ ‖B‖Yk).

0 50 100 150 200 250 30010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u)

τ = 10−2

τ = 10−6

τ = 10−10

iteration number k

rela

tive

resi

dual

nor

ms

||BTA

−1 f−

BTA

−1 B

y k||/||B

TA

−1 f−

BTA

−1 B

y 0||, ||

r(y)

k||/

||r(y

)0

||


Accuracy in the saddle point system

−BTA−1f +BTA−1Byk = −BTxk −BTA−1(f −Axk −Byk)

‖f −Axk −Byk‖ ≤O(α1)κ(A)

1− τκ(A)(‖f‖+ ‖B‖Yk),

‖ −BTxk − r(y)k ‖ ≤

O(α2)κ(A)

1− τκ(A)‖A−1‖‖B‖(‖f‖+ ‖B‖Yk),

Yk ≡ max{‖yi‖ | i = 0, 1, . . . , k}.

Back-substitution scheme α1 α2

A: Generic update

xk+1 = xk + αkp(x)k

τ u

B: Direct substitutionxk+1 = A−1(f −Byk+1)

τ τ

C: Corrected dir. subst.xk+1 = xk +A−1(f −Axk −Byk+1)

u τ

}additionalsystem with A


Generic update: xk+1 = xk + αkp(x)k

0 50 100 150 200 250 30010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u)

τ = 10−2

τ = 10−6

τ = 10−10

iteration number k

resi

dual

nor

m ||

f−A

x k−B

y k||

0 50 100 150 200 250 30010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u), τ = 10−10, τ=10−6, τ=10−2

iteration number k

rela

tive

resi

dual

nor

ms

||−B

Tx k||/

||−B

Tx 0||,

||r k(y

) ||/||r

0(y) ||


Direct substitution: xk+1 = A−1(f −Byk+1)

0 50 100 150 200 250 30010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u)

τ = 10−2

τ = 10−6

τ = 10−10

iteration number k

resi

dual

nor

m ||

f−A

x k−B

y k||

0 50 100 150 200 250 30010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u)

τ = 10−2

τ = 10−6

τ = 10−10

iteration number k

rela

tive

resi

dual

nor

ms

||−B

Tx k||/

||−B

Tx 0||,

||r k(y

) ||/||r

0(y) ||


Corrected direct substitution: xk+1 = xk +A−1(f −Axk −Byk+1)

0 50 100 150 200 250 30010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u), τ = 10−10, τ=10−6, τ=10−2

iteration number k

resi

dual

nor

m ||

f−A

x k−B

y k||

0 50 100 150 200 250 30010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u)

τ = 10−2

τ = 10−6

τ = 10−10

iteration number k

rela

tive

resi

dual

nor

ms

||−B

Tx k||/

||−B

Tx 0||,

||r k(y

) ||/||r

0(y) ||


Forward error of computed approximate solution

‖x− xk‖ ≤ γ1‖f −Axk −Byk‖+ γ2‖ −BTxk‖,

‖y − yk‖ ≤ γ2‖f −Axk −Byk‖+ γ3‖ −BTxk‖,

γ1 = σ−1min(A), γ2 = σ−1

min(B), γ3 = σ−1min(BTA−1B).

0 50 100 150 200 250 30010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u)

τ = 10−2

τ = 10−6

τ = 10−10

iteration number k

rela

tive

erro

r no

rms

||x−

x k|| A/||

x−x 0|| A

, ||y

−y k|| B

TA

−1 B

/||y−

y 0|| BTA

−1 B


Conclusions

All bounds of the limiting accuracy depend on the maximum norm ofcomputed iterates, cf. [Greenbaum, 1997].

The accuracy measured by the residuals of the saddle point problemdepends on the choice of the back-substitution scheme [J, R, 2008].

Care must be taken when solving nonsymmetric systems [J, R, 2008b].

0 50 100 150 200 250 300 350 400 450

10−15

10−10

10−5

100

105

iteration number k

rela

tive

resi

dual

nor

ms

||BTA

−1 f−

BTA

−1 B

y k||/||B

TA

−1 f||

and

||r(y

)k

||/||B

TA

−1 f||

The residuals in the outer iteration process and the forward errors ofcomputed approximations are proportional to the backward error insolution of inner systems.


Thank you for your attention.

http://www.cs.cas.cz/∼miro

P. Jiranek and M. Rozloznık. Maximum attainable accuracy of inexact saddlepoint solvers. SIAM J. Matrix Anal. Appl., 29(4):1297–1321, 2008.

P. Jiranek and M. Rozloznık. Limiting accuracy of segregated solution methodsfor nonsymmetric saddle point problems. J. Comput. Appl. Math., 215:28–37,2008.


Null-space projection method

compute x ∈ N(BT ) as a solution of the projected system

(I −Π)A(I −Π)x = (I −Π)f,

compute y as a solution of the least squares problem

By ≈ f −Ax,

Π is the orthogonal projector onto R(B).

The least squares with B are solved inexactly, i.e. the computed solution v ofBv ≈ c is an exact solution of a perturbed least squares problem

(B + ∆B)v ≈ c+ ∆c, ‖∆B‖ ≤ τ‖B‖, ‖∆c‖ ≤ τ‖c‖, τκ(B)� 1.


Iterative solution of the null-space projected system

choose x0, solve By0 ≈ f −Ax0

compute αk and p(x)k ∈ N(BT )

xk+1 = xk + αkp(x)k˛

˛˛˛

solve Bp(y)k ≈ r(x)

k − αkAp(x)k

back-substitution:

A: yk+1 = yk + p(y)k ,

B: solve Byk+1 ≈ f −Axk+1,

C: solve Bvk ≈ f −Axk+1 −Byk,

yk+1 = yk + vk.

9>>>>>>>>>=>>>>>>>>>;inneriteration

r(x)k+1 = r

(x)k − αkAp

(x)k −Bp(y)

k

9>>>>>>>>>>>>>>>>>>>>>=>>>>>>>>>>>>>>>>>>>>>;

outeriteration


Accuracy in the saddle point system

‖f −Axk −Byk − r(x)k ‖ ≤

O(α3)κ(B)

1− τκ(B)(‖f‖+ ‖A‖Xk),

‖ −BTxk‖ ≤O(τ)κ(B)

1− τκ(B)‖B‖Xk,

Xk ≡ max{‖xi‖ | i = 0, 1, . . . , k}.

Back-substitution scheme α3

A: Generic update

yk+1 = yk + p(y)k

u

B: Direct substitutionyk+1 = B†(f −Axk+1)

τ

C: Corrected dir. subst.yk+1 = yk +B†(f −Axk+1 −Byk)

u

}additional leastsquare with B


Generic update: yk+1 = yk + p(y)k

0 10 20 30 40 50 60 70 80 90 10010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u), τ = 10−10, τ=10−6, τ=10−2

iteration number

rela

tive

resi

dual

nor

ms

||f−

Ax k−

By k||/

||f−

Ax 0−

By 0||,

||r(x

)k

||/||r

(x)

0||

0 10 20 30 40 50 60 70 80 90 10010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u)

τ = 10−2

τ = 10−6

τ = 10−10

iteration numberre

sidu

al n

orm

||−

BTx k||


Direct substitution: yk+1 = B†(f −Axk+1)

0 10 20 30 40 50 60 70 80 90 10010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u)

τ = 10−2

τ = 10−6

τ = 10−10

iteration number

rela

tive

resi

dual

nor

ms

||f−

Ax k−

By k||/

||f−

Ax 0−

By 0||,

||r(x

)k

||/||r

(x)

0||

0 10 20 30 40 50 60 70 80 90 10010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u)

τ = 10−2

τ = 10−6

τ = 10−10

iteration numberre

sidu

al n

orm

||−

BTx k||


Corrected direct substitution: yk+1 = yk +B†(f −Axk+1 −Byk)

0 10 20 30 40 50 60 70 80 90 10010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u), τ = 10−10, τ=10−6, τ=10−2

iteration number

rela

tive

resi

dual

nor

ms

||f−

Ax k−

By k||/

||f−

Ax 0−

By 0||,

||r(x

)k

||/||r

(x)

0||

0 10 20 30 40 50 60 70 80 90 10010

−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

τ = O(u)

τ = 10−2

τ = 10−6

τ = 10−10

iteration numberre

sidu

al n

orm

||−

BTx k||


References

M. Benzi, G. H. Golub, and J. Liesen. Numerical solution of saddle pointproblems. Acta Numer., 14:1–137, 2005.

A. Greenbaum. Estimating the attainable accuracy of recursively computedresidual methods. SIAM J. Matrix Anal. Appl., 18(3):535–551, 1997.

P. Jiranek and M. Rozloznık. Maximum attainable accuracy of inexact saddlepoint solvers. SIAM J. Matrix Anal. Appl., 29(4):1297–1321, 2008a.

P. Jiranek and M. Rozloznık. Limiting accuracy of segregated solution methodsfor nonsymmetric saddle point problems. J. Comput. Appl. Math., 215:28–37, 2008b.


Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers...

Documents

Transcript of Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers...