Lecture 8 - Iterative systems of Equations
description
Transcript of Lecture 8 - Iterative systems of Equations
Lecture 8 - Iterative systems of Lecture 8 - Iterative systems of EquationsEquations
CVEN 302
June 19, 2002
Lecture’s GoalsLecture’s Goals
• Iterative Techniques– Jacobian method– Gauss-Siedel method– Relaxation technique
Iterative TechniquesIterative Techniques
• The method of solving simultaneous linear algebraic equations using Gaussian Elimination and the Gauss-Jordan Method. These techniques are known as direct methods. Problems can arise from round-off errors and zero on the diagonal.
• One means of obtaining an approximate solution to the equations is to use an “educated guess”.
Iterative MethodsIterative Methods
• We will look at three iterative methods:– Jacobi Method– Gauss-Seidel Method– Successive over Relaxation (SOR)
Convergence RestrictionsConvergence Restrictions
• There are two conditions for the iterative method to converge.– Necessary that 1 coefficient in each equation is
dominate.– The sufficient condition is that the diagonal is
dominate.
Jacobi IterationJacobi Iteration• If the diagonal is dominant, the matrix can be
rewritten in the following form
Jacobi IterationJacobi Iteration• The technique can be rewritten in a shorthand
fashion, where D is the diagonal, A” is the matrix without the diagonal and c is the right-hand side of the equations.
Jacobi IterationJacobi Iteration
• The technique solves for the entire set of x values for each iteration.
• The problem does not update the values until an iteration is completed.
Example (Jacobi Iteration)Example (Jacobi Iteration)
4X1 + 2X2 = 2
2X1 + 10X2 + 4X3 = 6
4X2 + 5X3 = 5
Solution: (X1 , X2 , X3 ) = (0.41379, 0.17241, 0.86206)
Jacobi ExampleJacobi Example
3
2
1
3
2
1
3
2
1
3
2
1
3
2
1
040
402
020
5
6
2
500
0100
004
5
6
2
040
402
020
500
0100
004
5
6
2
540
4102
024
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Jacobi ExampleJacobi Example
3
2
1
3
2
1
3
2
1
3
2
1
040
402
020
2.000
01.00
0025.0
5
6
2
2.000
01.00
0025.0
500
0100
004
2.000
01.00
0025.0
2.000
01.00
0025.01
500
0100
004
040
402
020
5
6
2
500
0100
004
X
X
X
X
X
X
X
X
X
X
X
X
Jacobi ExampleJacobi Example
• Formulation of the matrix
Jacobi IterationJacobi IterationConvergence Graph
0
0.2
0.4
0.6
0.8
1
0 2 4 6 8 10
Number of Iterations
Fu
nc
tio
na
l V
alu
eX1
X2
X3
Iteration 1 2 3 4 5 6 7
X 1 0.5 0.2 0.45 0.324 0.429 0.376 0.42
X 2 0.6 0.1 0.352 0.142 0.248 0.16 0.204
X 3 1 0.52 0.92 0.718 0.886 0.802 0.872
Jacobi ProgramJacobi Program
• The computer program is setup to do the Jacobi method for any size square matrix:
Jacobi(A,b) • The program can has options for maximum
number of iterations, nmax, and tolerance, tol.
Jacobi(A,b,nmax,tol)
Gauss-Seidel IterationsGauss-Seidel Iterations
• The Gauss-Seidel / Seidel technique is similar to the Jacobi iteration technique with one difference.
• The method updates the results continuously. It uses the new information from the previous iteration to accelerate converge to a solution.
Gauss-Seidel ModelGauss-Seidel Model
• The Gauss-Seidel Algorithm:
• The combined vector is upgraded ever term.
Example (Gauss-Seidel Iteration)Example (Gauss-Seidel Iteration)
4X1 + 2X2 = 2
2X1 + 10X2 + 4X3 = 6
4X2 + 5X3 = 5
Solution: (X1 , X2 , X3 ) = (0.41379, 0.17241, 0.86206)
Gauss-Seidel ExampleGauss-Seidel Example
• Formulation of the matrix problem
Gauss-Seidel IterationGauss-Seidel Iteration
Iteration 1 2 3 4 5 6 7
X 1 0.5 0.25 0.345 0.384 0.401 0.408 0.411
X 2 0.5 0.31 0.231 0.197 0.183 0.177 0.175
X 3 0.6 0.75 0.815 0.842 0.854 0.858 0.858
Convergence Graph
0
0.2
0.4
0.6
0.8
1
0 2 4 6 8 10
Number of Iteration
Fu
nc
tio
na
l V
alu
es
X1
X2
X3
Gauss-Seidel ModelGauss-Seidel Model
• The computer program is setup to do the Gauss-Seidel method for any size square matrix:
Seidel(A,b)
• The program can has options for maximum number of iterations, nmax, and tolerance, tol.
Seidel(A,b,nmax,tol)
Example - Iteration with no Example - Iteration with no diagonal dominationdiagonal domination
3X1 - 3X2 + 5X3 = 4
X1 + 2X2 - 6X3 = 3
2X1 - X2 + 3X3 = 1
Solution: (X1 , X2 , X3 ) = (1.00, -2.00, -1.00)
Using the GS algorithmUsing the GS algorithm
Using the Gauss-Seidel Program with the following A
matrix and b vector.
Solution: (X1 , X2 , X3 ) = (1.00, -2.00, -1.00)
Gauss-Seidel ExampleGauss-Seidel Example
3
2
1
3
2
1
3
2
1
3
2
1
3
2
1
3
2
1
03333.06667.0
305.0
6667.110
3333.0
5.1
3333.1
012
601
530
3333.000
05.00
003333.0
1
3
4
3333.000
05.00
003333.0
012
601
530
1
3
4
300
020
003
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Gauss-Seidel ExampleGauss-Seidel Example
Program will work with the equations:
3
2
1
3
2
1
03333.06667.0
305.0
6667.110
3333.0
5.1
3333.1
X
X
X
X
X
X
Using a Gauss-Seidel IterationUsing a Gauss-Seidel Iteration
Iteration 1 2 3 4 5 6 7
X 1 1.33 2.630 3.412 2.296 -1.375 6.078 -7.620
X 2 0.833 -0.648 -5.113 -10.584 -11.989 -3.698 14.769
X 3 -0.278 -1.635 -3.645 -4.725 -2.746 3.153 10.336
Convergence Graph
-15
-10
-5
0
5
10
15
20
0 2 4 6 8 10
Number of iterations
Fu
ncti
on
al V
alu
e
X1
X2
X3
Successive over RelaxationSuccessive over Relaxation
• The technique is a modification on the Gauss-Seidel method with an additional parameter, that may accelerate the convergence of the iterations.
• The weighting parameter, has two ranges 0 < <1, and 1< <2. If = 1, then the problem is the Gauss-Seidel technique.
SOR MethodSOR Method
• The SOR algorithm is defined as:
• The difference is the weighting parameter,
Weighting ParameterWeighting Parameter
• If the parameter, is under 1, the residuals will be under-relaxed.
• If the parameter, = 1, the residuals are equal to a Gauss-Seidel model.
• If 1< < 2 the residuals will be over-relaxed and will general help accelerate the convergence of the solution.
Example of SORExample of SOR
4X1 + 2X2 = 2
2X1 + 10X2 + 4X3 = 6
4X2 + 5X3 = 5
Solution: (X1 , X2 , X3 ) = (0.41379, 0.17241, 0.86206)
SOR ExampleSOR Example
• Formulation of the SOR Algorithm
Effects of Effects of ParameterParameter
Number of Number of
Iterations Iterations0.7 33 1.25 120.8 27 1.3 140.9 22 1.4 171 17 1.5 22
1.1 13 1.6 301.15 10 1.7 431.175 101.2 10
• Using the SOR program SOR(A,bnmax,tol) with nmax=50 and tol = 0.000001
SummarySummary
• Convergence conditions need to be met in order for iterative techniques to converge
• Jacobi method upgrades the values after each iteration.
• Gauss-Seidel upgrades continuously through method.
• SOR (Successive over Relaxation) uses the residuals to accelerate the convergence.
HomeworkHomework
• Check the Homework webpage