Notes Part1.pdf

CH 3073: Numerical Methods

CH 5500: Numerical Methods I

Autumn 2014-15

Instructor: Kishalay Mitra

Department of Chemical Engineering

Indian Institute of Technology Hyderabad

([email protected])

Tue / Thu: 11:30 AM – 1 PM

Room # 132


mailto:[email protected]

Overview

Topics

Linear Algebraic Equations (5 lecs)

Nonlinear Algebraic Equations (3 lecs)

Function Approximation (6 lecs)

Ordinary Differential Equations–Initial Value Problems (ODE-IVPs)

(10 lecs)

Programming in MATLAB (10 classes)

Pre-requisites of the Course

BE / BTech in Chemical Engg / Chem Tech

Programming Skills (MATLAB)

Textbook for the Course

S. K. Gupta, Numerical Methods for Engineers, New Age Intl.

Publishers, New Delhi, 2nd Ed., 2009.

Indian Institute of Technology Hyderabad2

Examples

Topics

Linear Algebraic Equations

Nonlinear Algebraic Equations


Examples (contd.)

Topics

ODE - IVPs

ODE - BVPs


Evaluation


No. Type Final

Weightage

Tentative

Time

Remarks

1. Mid Semester Exam 30 End Sept. Absolute

Grading

2. End Semester Exam 30 End Nov. Absolute

Grading

3. Class Tests, Computer

Programming

15+15 Almost

each

class

Relative

Grading

4. Attendance & Class

Performance

10 85%

attendance

Total = 100

Programming Practices

Problems

Finding sum of a digit and reverse the order

In an array, store all the Fahrenheit values given Celsius values from -100C

to 100C

Sum the following series

Tabulate values of below function for integer values of k = 0 to 15

Use of subroutines (preferably)

Solution of a quadratic equation

Write program to evaluate any nth order polynomial

Fitting a straight line through number of given points

Write program to generate Fibonacci sequence

Given a sequence of numbers, sort them in ascending and descending order

Write programs to multiply two matrices taking their sizes, values of elements

as inputs


!12

1...!7!5!3

12753

n

xxxxxSum

nn

!/ kaeSum ka

Programming

Programming Review

EVALUATE S = 1 + 2 + 3 + ………. + n

and COMPARE WITH analytical results

S = n(n + 1)/2

GET FAMILIAR WITH C, FORTRAN,

MATLAB®

PLOT S vs. n

GET FAMILIAR WITH MATLAB®, MS

Excel


Chapter 1

LINEAR ALGEBRIC EQUATIONS

WHEN DO WE HAVE SOLUTIONS

GAUSS ELIMINATION

THOMAS’ ALGORITHM

LU DECOMP

GAUSS – SEIDEL, SOR, ETC.


Representing MENU

SET OF M EQUATIONS IN N UNKNOWNS

SIMULTANEOUS LINEAR ALGEBRAIC

EQUATIONS

a11x1 + a12 x2 + …………..+ a1N xN = b1

a21x1 + a22 x2 + …………..+ a2N xN = b2

.

.

.

.

aM1x1 + aM2 x2 + …………..+ aMN xN = bM

e.g. x1 + x2 = 4

2x1 + 3x2 = 11

PDE ODE NLE SLE


Cramer’s Rule

Example 1.1

X1 + X2 = 4

2X1 + 3X2 = 11


Unique solution exists in case det(A) is nonsingular

Additionally, if b = 0, |Aj| = 0

all xj = 0 (trivial solution)

In case A is singular

First case: Identical equations

1 equation 2 unknown – Infinite solutions

Second case: Incompatible and no solution exist

Reviewing Rank r

(AUGMENTED A ≡ Aug A)

Aug A ≡ [ A b]

One highest possible size non zero determinant

∴ r = 2

∴ r = 1


1 1 4

2 3 11

1 10

2 3

11

Solution exists

IF and ONLY IF

RANK A = RANK (Aug A) COMPATABLE EQUATIONS

FOR M = N (FOR CHEMICAL ENGG. PROBLEMS)

IF r (A) = N UNIQUE SOLUTION

IF r (A) < N INFINITE SOLNS Assume N-r variables


Example

Example 1.3

X1 + X2 = 4

2 X1 + 3 X2 = 11

2 X1 + 3 X2 = 11

4 X1 + 6 X2 = 22

2 X1 + 3 X2 = 11

4 X1 + 6 X2 = 20


Homogeneous Case

ANxN xNx1 = bNx1

IF

Homogeneous equations

a11x1 + a12x2 = 0

a22x1 + a22x2 = 0; AND r(A) = 2 = N

Unique Solution Trivial solution

NON-TRIVIAL SOLUTIONS IF r(A) < N


4 2

Homogeneous Case (contd.)

NON-TRIVIAL SOLUTIONS IF r(A) < N

r(A) = 1

N – r = 2 – 1 = 1

ASSUME x1 = 5; THEN x2 = -10

x1 = 10; THEN x2 = -20


Review


𝐀 𝐱 = 𝐛 : FOCUS ON 𝐀 𝐍 × 𝐍 ONLY

SOLUTIONS ONLY IF RANK 𝐀 = 𝐑𝐀𝐍𝐊 𝐀 ⋮ 𝐛 ≡ 𝐑𝐀𝐍𝐊 𝐀𝐔𝐆 𝐀

IF RANK 𝐀 = 𝐍, UNIQUE SOLUTION

IF RANK 𝐀 < 𝐍, MANY SOLUTIONS

HOMOGENEOUS EQUATIONS (b = 0)

TRIVIAL SOLUTION x = 0, IF RANK 𝐀 = 𝐍

NONTRIVIAL SOLUTION, IF RANK 𝐀 < 𝐍

Direct Methods


𝐆𝐚𝐮𝐬𝐬 𝐄𝐥𝐢𝐦𝐢𝐧𝐚𝐭𝐢𝐨𝐧 (𝟏𝟖𝟎𝟗)

A x = b

RANK 𝐀 = 𝐍

Cramer’s Rule: Excessive Calculations (N2 × N!)

Another route: A-1b

Equally expensive and suffers from additional problem of sensitivity

(round off errors)

0

G E Example


Example 1.4

0

Direct Methods


PIVOT

19

𝐆𝐚𝐮𝐬𝐬 𝐄𝐥𝐢𝐦𝐢𝐧𝐚𝐭𝐢𝐨𝐧 (𝟏𝟖𝟎𝟗)

A x = b

RANK 𝐀 = 𝐍

Cramer’s Rule: Excessive

Calculaitons

G E (contd.)


PIVOT

20

G E (contd.)


Product of diagonal terms gives you the value of determinant

Computational Complexity


Pivoting


NOTE

23

Which one to prefer – row or column shift? Which row / column to

shift? One that gives you largest absolute value in the pivot position

Process ends when zero cannot be removed from pivot position and

this occurs when set of equations does not have unique solutions

G E Example (Pivoting)


Example 1.5

Pivoting Practice


25

Get correct result with pivoting and wrong result

without pivoting

Thomas’ Algorithm


𝐛𝟏 𝐜𝟏 𝟎 𝟎 𝟎 𝟎 … 𝟎 𝟎𝐚𝟐 𝐛𝟐 𝐜𝟐 𝟎 𝟎 𝟎 … 𝟎 𝟎𝟎 𝐚𝟑 𝐛𝟑 𝐜𝟑 𝟎 𝟎 … 𝟎 𝟎𝟎 𝟎 𝟎 𝟎 𝟎 𝟎 … 𝟎 𝟎⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮𝟎 𝟎 𝟎 𝟎 𝟎 … 𝐚𝐍−𝟏 𝐛𝐍−𝟏 𝐜𝐍−𝟏𝟎 𝟎 𝟎 𝟎 𝟎 … 𝟎 𝐚𝐍 𝐛𝐍

𝐱𝟏𝐱𝟐𝐱𝟑𝐱𝟒⋮𝐱𝐍−𝟏𝐱𝐍

=

𝐝𝟏𝐝𝟐𝐝𝟑𝐝𝟒⋮𝐝𝐍−𝟏𝐝𝐍

26

Finite Difference BVP

Stage operations

Thomas’ Algorithm (contd.)


27

LU Decomposition (Factorization)


( )

( )

( ) ( )

( ) ( )

( )( ),

( ) ( )

,

. .

. .

.

. . . . . . .

. . . . . . .

. . . . . . .

. . . .

1

21

1

11

1 2

31 32

1 2

11 22

N 11N N 1N1

1 N 1

11 N 1 N 1

1 0 0 0 0

a1 0 0 0

a

a a1 0 0 0

a aL

aa1

a a

28

Cases where A does not

change but b changes

Multiplying factors are multiplied with the pivot row and added

with the other rows – we take –ve of multiplying factors

LU Decomp (contd.)


We can show

A = L U; A x = L (U x) = b

L y = b SOLVE BY FORWARD /DOWNWARD SWEEP

• Remember this b is original b, not the modified b as in case of

Gauss Elimination

U x = y THEN SOLVE BY REVERSE SWEEP

29

( )

( )

( ) ( )

( ) ( )

( )( ),

( ) ( )

,

. .

. .

.

. . . . . . .

. . . . . . .

. . . . . . .

. . . .

1

21

1

11

1 2

31 32

1 2

11 22

N 11N N 1N1

1 N 1

11 N 1 N 1

1 0 0 0 0

a1 0 0 0

a

a a1 0 0 0

a aL

aa1

a a

LU Decomp Example


Example 1.8

Solution: [-11 5 4]

Ly=b

Ux=y

LU Decomp Example


By R2 – (1/7) R1 &

R3 – (2/7) R1

35

14

15

x

x

x

932

151

217

3

2

1

15)7

2(35

15)7

1(14

15

x

x

x

2)7

2(91)

7

2(37)

7

2(2

2)7

1(11)

7

1(57)

7

1(1

217

3

2

1

7

2157

8315

x

x

x

7

59

7

190

7

5

7

340

217

3

2

1

M21 = (1/7)

M31 = (2/7)

LU Decomp Example


By R3 – (19/34) R2

7

83

34

19

7

2157

8315

x

x

x

7

5

34

19

7

5900

7

5

7

340

217

3

2

1

7

2157

8315

x

x

x

7

59

7

190

7

5

7

340

217

3

2

1

M21 = (1/7)

M31 = (2/7)

M32 = (19/34)

238

191100

7

5

7

340

217

U

134/197/2

017/1

001

L

LU Decomp Example


238

57337

8315

x

x

x

238

191100

7

5

7

340

217

3

2

1 By GE, x1 = 1, x2 = 2, x3 = 3

35

14

15

y

y

y

134/197/2

017/1

001

Ly

3

2

1

By Forward Sweep, y1 =

15, y2 = 83/7, y3 = 5733/238

238

57337

8315

x

x

x

238

191100

7

5

7

340

217

Ux

3

2

1

By Backward Sweep, x1 =

1, x2 = 2, x3 = 3

Ly=b

Ux=y

LU … Different b


77

35

45

y

y

y

134/197/2

017/1

001

Ly

3

2

1

By Forward Sweep, y1 =

45, y2 = 200/7, y3 =

11466/238

238

114667

20045

x

x

x

238

191100

7

5

7

340

217

Ux

3

2

1

By Backward Sweep, x1 =

4, x2 = 5, x3 = 6

LU Factorization


33

2322

131211

333231

2221

11

U00

UU0

UUU

LLL

0LL

00L

LU

932

151

217

A

7

5U1ULUL

7

34U5ULUL

;7

1L1UL

238

1911U9ULULUL;2U2UL

34

19L3ULUL;1U1UL

7

2L;2UL;7U7UL

1LLL

2323221321

2222221221

211121

33333323321331131311

3222321231121211

311131111111

332211

Dooli

ttle

’s M

eth

od

Row Column

multiplication:

9 equations,

12 unknowns –

3 to be assumed

238/191100

7/57/340

217

134/197/2

017/1

001

A

U – same as GE

LU Factorization (contd.)


33

2322

131211

333231

2221

11

U00

UU0

UUU

LLL

0LL

00L

LU

932

151

217

A

34

5U1ULUL

7

34L5ULUL

;1L1UL

238

1911L9ULULUL;

7

2U2UL

7

19L3ULUL;

7

1U1UL

2L;2UL;7L7UL

1UUU

2323221321

2222221221

211121

33333323321331131311

3222321231121211

311131111111

332211

Cro

ut’

sM

eth

od

Row Column

Multiplication:

9 equations,

12 unknowns –

3 to be assumed

100

34/510

7/27/11

238/19117/192

07/341

007

A

LU Factorization (contd.)


T

33

2322

131211

332313

2212

11

UL

U00

UU0

UUU

UUU

0UU

00U

LU

210

121

012

A

Cholesky’s algorithm for symmetric positive definite A:

Take only the +ve roots

Twice as fast as GE

Row Column multiplication: 9 equations, 12 unknowns – 3 to be assumed

3/200

3/22/30

02/12

3/23/20

02/32/1

002

A

LU Decomp (contd.)


Benefit of LU Decomposition:

Solving Ax = b when only b is changing as in Finite difference

technique of ODE-BVP

FOR ONE A AND M bs:

# OF MULTIPLICATIONS/ DIVISIONS =

Substantially lower than GE being solved repetitively

WORKS IF A IS DIAGONALLY DOMINANT

|𝐚𝐢𝐢| ≥

𝐣=𝟏;𝐣≠𝐢

𝐍

|𝐚𝐢𝐣|

OTHER LU DECOMPS: CHOLESKY PROB. 1.10 &

DOOLITTLE/CROUT PROB. 1.9

38

Gauss Jordon


A, b & I (unit diagonal matrix) are written together

ELIMINATE TERMS IN A, BOTH BELOW AND ABOVE THE

DIAGONAL to convert it into I – in turn, I is converted into A-1 & b

is converted into solution i.e. the values of x

Twice as time consuming as GE

39

GJ Example


Example 1.9

R2-R1, R3-2R1

R2/2

R2+R1

R3/-1

(1/2)R3+R2, (-3/2)R3+R1

Iterative Techniques


GAUSS – JACOBI / GAUSS – SIEDEL (G S) / SOR (SUCCESSIVE OVER

RELAXATION)

11

k

NN1

k

313

k

21211k

1a

xaxaxabx

)()()()( .........

22

)k(

NN124

)k(

323

)1k(

1212)1k(

2a

xa.........axaxabx

NN

k

NNN

k

N

k

NNk

Na

xaxaxabx

)1(

11

)1(

22

)1(

11)1(.........

GUESS x1(1), x2

(1), …, xN(1)

A x = b: a11x1 + a12x2 + a13x3 + --- + a1NxN = b1

41

For G-Jacobi, k

For G-Siedel,

k+1

Iterative Techniques (contd.)


i.e., IF A IS DIAGONALLY DOMINANT (DD), WILL CONVERGE (IF A IS NOT

DD, MAY OR MAY NOT CONVERGE) – sufficient, not necessary condition

# MULTIPLICATIONS + DIVISIONS = MN2 (M = # OF ITERATIONS)

CONVERGENCE CRITERION:

MOST COMMON:

TOL: USER SPECIFIED

42

GS Example


Example 1.10

Iterative Techniques (contd.)


Solved Example: 1.11 – Read

Reference: Press, Teukolsky, Vettering & Flannery - NUMERICAL RECIPES

Assignment: 1, 2, 3, 4, 6, 8, 10

Lab Assignment: Write a general program for G-Jacobi, G-Siedel, SOR to solve 13,

15, 18 (Keep a copy of your programs – we will use them later in the course)

0 xi(k) xi,GS

(k+1) xi,SOR(k+1)

a a/2

SCHEMATICALLY

( ) ( ); , ,...., ( )

,k k 1

X X i 1 2 N ALLi i GS

( ) ( ) ( ) ( )

, , ; , ,....,k 1 k k 1 k

i SOR i i GS ix x w x x i 1 2 N

44

w > 1 over relaxation,

w < 1 under relaxation,

w = 1 same as GS;

w typically in between 1 & 2, but no rule

Purpose is to speed up the process,

hence we generally go for w > 1

SOR Example


Example 1.11

[1 2 1]

Comparison Iterative schemes


k

2

k

3

1k

3

k

1k

2

1k

2k

1

x4x

2

xx2x

2

x1x

1k

2

k

3

1k

3

1k

1k

2

1k

2k

1

x4x

2

xx2x

2

x1x

1k

3

k

GS,3

1k

3

k

3

1k

2

k

GS,2

1k

2

k

2

1k

1

k

GS,1

1k

1

k

1

k

GS,2

k

GS,3

1k

3

k

GS,1k

GS,2

1k

2k

GS,1

xx5.1xx

xx5.1xx

xx5.1xx

x4x

2

xx2x

2

x1x

GJ GS SOR

MATLAB Exercise


Gauss Elimination

function x=gauss(a,b)

% A = [2 1 0; 1 2 1; 0 1 1]; B = [1 2 4]’; p = gauss(A,B);

% a - (n x n) matrix; b - column vector of length n

Steps:

% Step 0: get number of rows in matrix a, get length of b & send

message in case they are not same

% Step 1: form (n,n+1) augmented matrix

% Step 2: Start from row 1 and do for all rows that Pivot row

divided by pivot element

% Step 3: For other rows convert all elements below pivot into 0

% Step 4: begin back substitution

MATLAB Exercise (contd.)


Gauss Elimination

function x=gauss(a,b)

m=size(a,1); % get number of rows in matrix a

n=length(b); % get length of b

if (m ~= n)

error('a and b do not have the same number of rows')

end

% Step 1: form (n,n+1) augmented matrix

a(:,n+1)=b;



for i=1:n

%Step 2: Start from row 1 and do for all rows

% Pivot row divided by pivot element

a(i, i:n+1) = a(i, i:n+1) / a(i, i);

%Step 3: For other rows convert all elements below pivot into 0

for j=i+1:n

a(j, i:n+1) = a(j, i:n+1) - a(j, i)*a(i, i:n+1);

end

end



%Step 4: begin back substitution

for j=n-1:-1:1

a(j,n+1) = a(j,n+1) - a(j,j+1:n)*a(j+1:n,n+1);

end

%return solution

x=a(:,n+1)



for i=1:n

a(i, i:n+1)=a(i, i:n+1)/a(i, i);

for j=i+1:n

a(j, i:n+1) = a(j, i:n+1)

- a(j, i)*a(i, i:n+1);

end

end

for j=n-1:-1:1

a(j,n+1) = a(j,n+1) –

a(j,j+1:n)*a(j+1:n,n+1);

end

a(1,1:4)=a(1, 1:4)/a(1,1);

a(2, 1:4) = a(2, 1:4)- a(2, 1)*a(1,1:4);

a(3, 1:4) = a(3, 1:4)- a(3, 1)*a(1,1:4);

i = 1

j = 2

j = 3

a(2,2:4)=a(2, 2:4)/a(2,2);

a(3, 2:4) = a(3, 2:4)- a(3, 2)*a(2,2:4);

i = 2

j = 3

a(2,4) = a(2,4) – a(2,3:3)*a(3:3,4);j = 2

j = 1 a(1,4) = a(1,4) – a(1,2:3)*a(2:3,4);

a(3,3:4)=a(3, 3:4)/a(3,3);i = 3



a(1,1:4)=a(1, 1:4)/a(1,1);

a(2, 1:4) = a(2, 1:4)- a(2, 1)*a(1,1:4);

a(3, 1:4) = a(3, 1:4)- a(3, 1)*a(1,1:4);

i = 1

j = 2

j = 3

a(2,2:4)=a(2, 2:4)/a(2,2);

a(3, 2:4) = a(3, 2:4)- a(3, 2)*a(2,2:4);

i = 2

j = 3

4110

2121

1012

4110

5.115.10

5.005.01

333.000

167.010

5.005.01

9100

167.010

5.005.01

a(3,3:4)=a(3, 3:4)/a(3,3);i = 3



a(2,4) = a(2,4) – a(2,3:3)*a(3:3,4);j = 2

j = 1 a(1,4) = a(1,4) – a(1,2:3)*a(2:3,4);

9100

167.010

5.005.01

x3 = 9

x2 + 0.67x3 = 1

x1+ 0.5x2 + 0x3 = 0.5

x3 = 9 = a(3,4)

x2 + 0.67 a(3,4) = 1

x2 = 1 - 0.67 a(3,4) = a(2,4)

x1+ 0.5 a(2,4) + 0 a(3,4) = 0.5

x1 = 0.5 - 0.5 a(2,4) - 0 a(3,4) = a(1,4)

9100

567.010

5.005.01

9100

567.010

305.01

G E Example using MATLAB


4110

2121

1012

4110

5.115.10

5.005.01

333.000

167.010

5.005.01

9100

167.010

5.005.01

9100

567.010

5.005.01

9100

567.010

305.01

Notes Part1.pdf

Documents

Transcript of Notes Part1.pdf