07. Matrix methods

8/7/2019 07. Matrix methods

1/54

Systems of Linear Equations


2/54


Vectors

Matrices

Special matrices Vector vector scalar product

Matrix multiplied by vector

Linear system of equations

Matrix multiplication


3/54


Is there a solution?

If there is a solution, is there only one?

How can we express the solution with

the inverse of the coefficient matrix?

Is it usually a good idea to get thesolution via the inverse?


4/54

The determinant is a number that can be

calculated knowing the elements of the

(cofficient) matrix. The matrix inverse doesnt exist if the

determinant is zero.

A condition for the inverse to exist is that the

determinant is not zero.

Determinant zero; coefficient matrix is singular;

solution does not exist; system is contradictory

OR undetermined; matrix rank is less than

number of equations (or just division by zero,

overflow etc.): all mean the same


5/54

Solution of Systems of

Linear Equations

Ax = b

solution

X = A-1

b (A-1

is the inverse matrix)or, Ix = A-1b (I is identity matrix)


6/54

Types of matrix problems


7/54

Example 1 - Unique solution.Consider the system

2x + 3y = 8

5x - 6y = 30

Geometrically this

corresponds to

two linescrossing in one

point.

-6

-4

-2

0

2

4

0 2 4 6 8 10

Equation 1

Equation 2


8/54

Example 2 - No unique solution.

Consider the system

2x + 3y = 8

4x + 6y = 16

Geometrically this

corresponds to two

coincident lines.Infinite number of

solutions.

-6

-4

-2

0

2

4

0 2 4 6 8 10

Equation 1

Equation 2


9/54

Example 3 - No solution.

Consider the system

2x + 3y = 8

4x + 6y = 30

Geometrically this

corresponds to

two parallel linesnever crossing. -6-4

-

0

2

4

6

0 2 4 6 8 10

Equat on 1

Equat on 2


10/54

Example 4 Ill-conditioned matrix

Consider the system

2x + 3y = 8

4x + 7y = 15Geometrically this

corresponds to two

almost parallel

lines barelycrossing. Sensitive

to round-off error.

-6

-4

-2

0

2

4

0 2 4 6 8 10

qua ion 1

qua ion 2


11/54

Existence and Uniqueness

Statement with Rank The rank of a matrix tells us how many

independent rows (or how many independent

columns) the coefficient matrix has -- it may be

less than the total number of equations.

If rank is less than the number of equations,two cases can happen:

Consistent (if rank of coeff matrix = rank of

augmented matrix) Infinite number ofsolutions

Non-Consistent (if rank of coeff matrix < rank

of augmented matrix) No solution


12/54

Definition: Augmented matrix

A

nnnb

b

a

a

.

.

...

....

....

... 111


13/54

Example 1Consider the system2x + 3y = 8

4x + 6y = 16

Thus the determinant is 2*6 - 3*4 = 0.

And indeed the second equation is 2 times the first.

The solution can be written as x = 4 - 3/2 y for any y.

Geometrically this corresponds to two coincident

lines.

In terms of rank: the coefficient matrix has rank 1,

the augmented matrix has rank 1

16

8

64

32


14/54


15/54

Direct Methods Gauss-Jordan elimination

Gaussian elimination

LU decomposition / factorization

Thomas algorithm

Iterative Methods Gauss-Seidel


16/54

Gaussian-JordanStep 1. Forward elimination:

Choose a pivot element on the main diagonal (start attop left corner)

Eliminate (set to zero) all elements above and belowthe main diagonal

Move to the next column

Repeat until matrix is an Identity matrix


17/54

Gauss-Jordan Elimination

Reduce an augmented matrix to the

identity to solve a system of linear

equations.

Example:

-

!

-

-

1

3

4

x

x

x

113

111

112

Solve

3

2

1


18/54


-

11133111

4112

:matrixAugmented

-

1113

3111

22/12/11

:2by1rowDivide


19/54


-

!

!

52/52/10

12/12/30

22/12/11

:1333

122

rowrowrowand

rowrowrowSet


20/54


-

52/52/10

3/23/110

22/12/11

:2/3by2roivide


21/54


-

3/163/800

3/23/110

3/73/201

:2ro2/13ro3roet

2ro2/11ro1roet


22/54


-

2100

3/23/110

3/73/201

:3/8by3roivide


23/54


-

!

!

2100

0010

1001

:33/211

33/122

rowrowrowSet

rowrowrowSet

-

2

0

1

: xSolution


24/54

Summary

Gauss-Jordan process

loop over rows (i)

scale row such that diagonal is 1.0 subtract a multiple of row i from

every other row in order to fill the

rest of the column with zeroes. move to next row

Final solution appears at the end of

the the augmented matrix.


25/54

Sub GaussJordan(A() As Double) 'Augmented matrix

Dim PivElt As Double, TarElt As DoubleDim n As Integer 'number of equations

Dim PivRow As Integer, TarRow As Integer 'pivot row; target row

Dim i As Integer, j As Integern = UBound(A, 1)

For PivRow = 1 To n 'process every rowPivElt = A(PivRow, PivRow) 'choose pivot element

If PivElt = 0 Then

MsgBox ("Zero pivot element encountered")

Stop

End IfFor j = 1 To n + 1

A(PivRow, j) = A(PivRow, j) / PivElt 'divide whole row

Next

For TarRow = 1 To n 'now replace all other rows

If Not (TarRow = PivRow) ThenTarElt = A(TarRow, PivRow)

For j = 1 To n + 1

A(TarRow, j) = A(TarRow, j) - A(PivRow, j) * TarElt

Next j

End IfNext TarRow

Next PivRow

End Sub


26/54

Gaussian EliminationPivoting:

If the main diagonal pivot element is small orzero, interchange rows (from below) to

maximize the pivot element

The determinant:

After forward elimination, the determinant is theproduct of all the main diagonal elements of the upper

triangular matrix. det(A) = u11u22u33 unn


27/54

Gaus. Elim. forward elimination

Eliminate elements in column 1

Example:

-

!

-

-

1

3

4

x

x

x

113

111

112

Solve

3

2

1


28/54


Eliminate a21 by calculating a ratio:

R = a21/a11

Set row 2 = row 2 row 1

-

!

-

-

1

1

4

113

2/12/30

112

:Result

3

2

1

x

x

x


29/54


Continue until all elements below

main diagonal = 0

Matrix is now upper triangular

-

!

-

-

3/16

1

4

3/800

2/12/30

112

:Result

3

2

1

x

x

x


30/54


31/54

Gaussian eliminationStep 1. Forward elimination:

Choose a pivot element on the main diagonal (start attop left corner)

Eliminate (set to zero) all elements below the maindiagonal

Move to the next column

Repeat until matrix is upper triangular

Step 2. Backward substitution:

Solve the last row explicitly for the last unknown Solve explicitly for the next row up

Repeat until all the rows are solved


32/54

LU Factorization

LU Factorization is a form of

Gaussian Elimination

The A matrix (in the problem Ax=b) is

factored into the product of two

matrices L and U (A=LU)

L is a lower triangular matrix and U isan upper triangular matrix


33/54

LU Factorization

How do we find the entries in L and

U?

there are nine unknowns (3 Ls and

6 Us)

there are nine equations (each

setting a row of L times a column

of U equal to one of the A values)

-

-

-

333231

232221

131211

33

2322

131211

3231

21

aaa

aaa

aaa

U00

UU0

UUU

1LL

01L

001


34/54

LU Factorization

.etc

a

aasoa

a

aasoa

asoa1

aaa

aaa

aaa

00

0

1

01

001

11

31

11

3131311131

11

21

11

21

21211121

11111111

333231

232221

131211

33

2322

131211

3231

21

!!!v

!!!v

!!v

-

!

-

-


35/54

Pivoting

What would happen if a11 were zero?

We would get a divide by zero error

when building L. Algorithms for LU factorization check

for this and swap the order of the

rows in A (pivoting) to avoid thisproblem.


36/54

LU Factorization

Once L and U have been determined

we have:

LUx = b How do we solve for x?

Set Ux=L-1b = y

Therefore Ly = b

Solve Ly=b for y

Solve Ux = y for x


37/54

Forward Substitution We solve Ly=b using a process

called forward substitution.

23213133

33232131

12122

22121

11

11

3

2

1

3

2

1

3231

21

yLyLby

by1yLyL

yLby

by1yLby

by1

b

b

b

y

y

y

1LL

01L

001

vv!

!vvv

v!

!vv

!

!v

-

!

-

-


38/54

Back Substitution

We solve Ux=y using a processcalled back substitution.

33

33

3333

3

2

1

3

2

1

33

2322

131211

yx

yx

yy

y

xx

x

000

!

!v

-

!

-

-


39/54

Back Substitution

11

31321211

1313212111

22

3232

2

2323222

U

xUxUyx

yxUxUxU

U

xUy

x

yxUxU

vv!

!vvv

v!

!vv


40/54

LU Decomposition and

Backsubstitution

Basic:

Call LUDecompose(A, Indx)

Call LUSubstitute(A, Indx, b, x)


41/54

Sub LUDecompose(A() As Double, Indx() As Integer)

' input a(n,n)

' output a(n,n) and indx(n)

Const Tiny As Double = 1E-16

Dim n As Integern = UBound(A, 1)

Dim i As Integer, j As Integer, k As Integer, m As

Integer

Dim dum As Double

For k = 1 To n - 1m = k

For i = k + 1 To n

If (Abs(A(i, k)) > Abs(A(m, k))) Then m = i

Next i

Indx(k) = mdum = A(m, k)

If m > k Then

A(m, k) = A(k, k)

A(k, k) = dumEnd IfIf Abs(dum) > Tiny Then

dum = 1 / dum

Else

MsgBox ("Singular coefficient matrix")

Stop

End IfNext k


42/54

Sub LUSubstitute(A() As Double, Indx() As Integer, _

b() As Double, x() As Double)' input A(n,n) containing the LU decomposition

' input Indx(n) index vector' input b(n) right hand side

' output x(n) solution

Dim n As Integern = UBound(A, 1)Dim i As Integer, j As Integer, k As Integer

Dim dum As DoubleFor i = 1 To n

End sub


43/54

Direct vs Iterative

Gauss elimination, Gauss-Jordan, LU-

decomposition are DIRECT methods

We can predict the number of

multiplications and additions if the

number of equations (n) is known

There are other direct methods for special

matrices. One of them is important for us:

It is the Thomas algorithm forTridiagonal

equations.


44/54

Occurs often in reservoir simulation

Thomas algorithm is efficient

Uses LU factorization

Tri-diagonal matrix


45/54

Tri-diagonal matrix

X1

.

.

.

.

.

.

.

.

.

.

.

.

.

xn

d1

.

.

.

.

.

.

.

.

.

.

.

.

.

dn


46/54

Thomas algorithm save a,b,c

vectors, not aii elementsX1

.

.

.

.

.

.

.

.

.

.

.

.

.

xn

d1

.

.

.

.

.

.

.

.

.

.

.

.

.

dn

ai

cibi


47/54

Thomas algorithmSub Thomas (a() As Double, b() As Double, c() As Double, d() As Double, x() As

Double)

'Tridiagonal system ofequations

Dim n As Integer, i As Integer

n = UBound(b)

ReDim w(n) As Double, g(n) As Double

w(1) = b(1)

g(1) = d(1) / w(1)

Fori = 2Ton

w(i) = b(i) a(i) * c(i 1) / w(i 1)

g(i) = (d(i) a(i) * g(i 1)) / w(i)

Next ix(n) = g(n)

Fori = n 1To1 Step 1

x(i) = g(i) c(i) * x(i + 1) / w(i)

Next i

nd Sub


48/54

Direct vs Iterative

Iterative methods are generalization ofthe direct iteration we learned for onevariable x=g(x) equations.

Iterative methods are used for verylarge but sparse systems (reservoirsimulation).

You can not predict the number ofoperations for an indirect method andyou need a starting guess and aconvergence criterion.

The method we are going to learn is theGauss-Seidel method.


49/54

Iterative methods

Gauss Seidel


50/54

Gauss-Seidel

Suppose we are trying to solve a

system of three equations:

1x2xx

11x2x4x

8xx2x3

321

321

321

!

!

!


51/54

Gauss-Seidel

These equations can be rewritten as:

2

xx1x

4

x2x11x

3xx28x

213

312

321

!

!

!


52/54

Gauss-Seidel Iteration

Gauss-Seidel starts from an initial

guess of the xs and updates that

guess in an attempt to converge to asolution:

2

xx1x

4x2x11x

3

xx28x

1k

2

1k

11k

3

k31k11k

2

k

3

k

21k

1

!

!

!


53/54

Diagonally Dominant Matrices

Diagonally dominant matrices:

The absolute value of the diagonal element is

greater than the sum of the absolute values

of all the other elements in a given row.

Identity matrix is diagonally dominant

Diagonally Dominant?

YES NO

-

1032

252

214

-

1032

232

214


54/54

Diagonal Dominance

Gauss-Seidel will only work if/only if

the matrix is diagonally dominant.

07. Matrix methods

Documents

Transcript of 07. Matrix methods