Chapter 7.Iterative Solutions of Systems

of Equationsof Equations

EFLUM – ENAC ‐ EPFL

ContentsContents

1 Introduction1. Introduction

2. First example: Scalar Equation

3 i l i f f i3. Iterative solutions of a system of equations: Jacobi iteration method

4. Iterative methods for finite difference equations: Back to problem 6.3

5. The Successive Over Relaxation (SOR)

Introduction: The fixed point iterationIntroduction: The fixed point iterationpp

Previous Method (used on previous class)Uses Gaussian Elimination (or “\” in MatLab)

Introduction: The fixed point iterationIntroduction: The fixed point iterationpp

( )1k kx g x=The main concept:The main concept: ( )1k kx g x+The main concept:The main concept:

Example:Example:

3 6x = 2 6x x+ = 1 32k

kxx + = − +

4X = 4

2.5

3

3.5

4

fxn

X0 = 4

1

1.5

2

Val

ue o

f

X0 = 1

0 1 2 3 4 5 6 7 8 9 100

0.5

Iteration stepX0 = 0

Introduction: The fixed point iterationIntroduction: The fixed point iterationpp

The proof of convergence:The proof of convergence: 1 3kk

xx = − +The proof of convergence:The proof of convergence: 1 32kx + +

( )1 2

1 1 2 3=

+ ( )

Introduction: The fixed point iterationIntroduction: The fixed point iterationpp

First Iteration method:First Iteration method:

3 6x = 2 6x x+ = 1 32k

kxx + = − +

First Iteration method:First Iteration method:

G li d it ti th dG li d it ti th d

2Always ConvergesAlways Converges

3 6x =Generalized iteration method:Generalized iteration method:

(3 ) 6x xα α− + = 13 6

k kx xαα α+

−= − +

Converges?Converges?α α

Introduction: The fixed point iterationIntroduction: The fixed point iterationpp

3 6Generalized iteration method:Generalized iteration method:

(3 ) 63 6x xα−

+3 6x = (3 ) 6x xα α− + =Converges?Converges?

1k kx xα α+ = − +

NoNo YesYes1

X: 1.5Y 1m

ber

0.6

0.8

Y: 1

tera

tion

num

0.2

0.4

ude

of th

e it

1 2 3 4 5 6 7 8 9 100

0.2

value of the splitting parameter α

mag

nitu

X: 3Y: 0

Introduction: The fixed point iterationIntroduction: The fixed point iterationpp

Generalized iteration method:Generalized iteration method: 3 6α−1k kx x

α α+ = − +How fast does it Converges?How fast does it Converges?

The smallerthis value is

The fastest is 4

6

8 α = 1.42α = 1.5α = 2.0α = 2.5α = 3.0

4 0

the convergence

0

2

valu

e of

x n α = 4.0α = 5.0

0 1 2 3 4 5 6 7 8 9 10-4

-2

Iteration Step

Iterative solution of a system of equationsIterative solution of a system of equationsy qy q

Jacobi interaction Jacobi interaction approachapproach

Consider the problem

D: Diagonal elements of A

L: Lower elements of A+U: Upper elements of A

L+U: Matrix B

andand 1Q <

Iterative solution of a system of equationsIterative solution of a system of equationsy qy q

Some notes about the vector normSome notes about the vector normThe vector norm calculation

S t b t th t iS t b t th t iSome notes about the matrix normSome notes about the matrix norm

The matrix norm calculation

Iterative solution of a system of equationsIterative solution of a system of equationsy qy q

Back to problem 6.3 (1Back to problem 6.3 (1°° case case –– last class)last class)p (p ( ))

Back to Laplace Equation (Last class example)

M_diag=sparse(1:21,1:21,-4,21,21);L_diag1=sparse(2:21,1:20,1,21,21);L_diag2=sparse(8:21,1:14,1,21,21);L_diag1(8,7)=0; L_diag1(15,14)=0;A M di +L di 1+L di 2+L di 1'+L di 2'A=M_diag+L_diag1+L_diag2+L_diag1'+L_diag2';b=zeros(21,1); b(7)=-100; b(14)=-100; b(21)=-100;

convcrit=1e9;

L=L_diag1+L_diag2U=L‘;

iteration matrix is Q=-Dinv*LnUconvcrit 1e9;h_old=ones(21,1);kount=0;while convcrit>1e-3 % loop ends when fractional

kount=kount+1; % change in h < 10-3;

LnU=L+U;Dinv=inv(M_diag) %D-1

h=Q*h_old+Dinv*b;convcrit=max(abs(h-h_old)./h);h_old=h;

end

Iterative solution of a system of equationsIterative solution of a system of equationsy qy qconvcrit=1e9;h_old=ones(21,1);kount=0;kount=0;while convcrit>1e-3 % loop ends when fractional

kount=kount+1; % change in h < 10-3

h=Q*h_old+Dinv*b;convcrit=max(abs(h-h_old)./h);h_old=h;

end

Successive over relaxation methodSuccessive over relaxation method

BeforeBefore NowNow

Jacobi iteration approachJacobi iteration approach

ρ(Q) = abs(max(eig(Q)))ρ(Q) ( ( g(Q)))Successive Order Relaxation MethodSuccessive Order Relaxation Method

(SOR)(SOR)

S(ω): Iteration matrix.ρ(Q): Magnitude of the largest eigenvalue of the

Jacobi iteration matrix.ωopt: Iteration parameter, chosen to accelerate

convergence

Iterative solution of a system of equationsIterative solution of a system of equationsy qy q

it 1 9

wopt=2/(1+sqrt(1-(normest(Q))^2));y=inv(D*(1/wopt)+L);S=-y*(U+(1-(1/wopt))*D);

it 1 9convcrit=1e9;h_old=ones(21,1);kount=0;while convcrit > 1e-3

kount=kount+1;

convcrit=1e9;h_old=ones(21,1);kount=0;while convcrit > 1e-3

kount=kount+1;h=Q*h_old+Dinv*b;convcrit=max(abs(h-h_old)./h);h_old=h;

end

h=S*h_old+y*b;convcrit=max(abs(h-h_old)./h);h_old=h;

end

FastFast☺☺Slow Slow