Lecture3(Linear Equation)

8/7/2019 Lecture3(Linear Equation)

1/75

1

System of Linear EquationsSystem of Linear Equations

This presentation covers

explanations for these two topics

shown, together with worked

examples.DefinitionDefinition

Solving Linear EquationsSolving Linear Equations


2/75

2

DefinitionDefinition


3/75

3

DefinitionDefinitionLinear equation: an equation of the form ax+by=0 where a and bare not both zero.

Linear system of equations: a system of equations such as

is a linear system of equations.

Both equations must be considered together.

Linear system can either be of two, three or more variables.

!

!

feydx

cbyax


4/75

4

DefinitionDefinition

Think back to linear equations. For instance,

consider the linear equation

y = 3x 5. A "solution" to this equation was

anyx,y-point that "worked" in the equation.So (2, 1) was a solution because, plugging in

2 forx:

3x 5 = 3(2) 5 = 6 5 = 1 =y

On the other hand, (1, 2) was not a solution,

because, plugging in 1 forx:

3x 5 = 3(1) 5 = 3 5 = 2


5/75

5

Now consider the following two-variable

system of linear equations:y= 3x 2y= x 6

Since the two equations

above are in a system, we

deal with them together at

the same time. In particular,

we can graph themtogether on the same axis

system, like this.


6/75

6


system of linear equations:

y= 3x 2y= x 6

Solution for a single equationis any point that lies on the line

for that equation. A solution for

a system of equations is any

point that lies on each line in

the system. For example, thered point at right is not a

solution to the system,

because it is not on either line.


7/75

7



y= 3x 2y= x 6

The blue point at right is not asolution to the system,

because it lies on only one of

the lines, not on both of them.


8/75

8



y= 3x 2y= x 6

The purple point at right is asolution to the system,

because it lies on both of the

lines.


9/75

9

Solving Linear System in TwoSolving Linear System in Two

VariablesVariables

Graphing

Substitution method

Addition method (also known as elimination

method)


10/75


11/75

11

GraphingGraphing

The first graph shows two distinct non-parallel lines

that cross at exactly one point. This is called an

"independent" system of equations, and the solution is

always some x,y-point.


12/75

12

GraphingGraphing

The second graph shows two distinct lines that are parallel.Since parallel lines never cross, then there can be nointersection; that is, for parallel lines, there can be nosolution. This is called an "inconsistent" system of

equations, and it has no solution.


13/75

13

GraphingGraphing

The third graph appears to show only one line. Actually, it'sthe same line drawn twice. These "two" lines, really being thesame line, then "intersect" at every point along their length.This is called a "dependent" system, and the "solution" is thewhole line.


14/75

14

Example 1Example 1

Solve the following system by graphing.2x 3y=24x+ y= 24

Solution


15/75

15

Example 1Example 1

Solution (5,4)


16/75


17/75

17

Example 3Example 3

Solve the following system by graphing.7x+ 2y = 16

21x 6y = 24

Solution


18/75

18

SummarySummary

i. Solve both equations fory

ii. Compare the slopes to decide how many solutionsthe system has

iii. If the system has one solutions graph the two linesin the same plane

iv. Identify the point of intersection

v. Check the point in both equations

To solve a linear system in two variable bygraphing


19/75

19

SummarySummary

A linear system in two variables may have one solution,no solution, or infinitely many solutions.

We use the slope andy-intercepts of the givenequations to determine how many solutions a system

has: Different slopes one solution

Same slopes, differenty-int no solutions

Same slopes, samey-int infinite many solutions


20/75

20

SummarySummary

When graphing to determine the solutions tothe system: First solve fory

Second compare the slopes to determine howmany solutions

Third if one solution graph both lines on thesame plane

Fourth identify point of intersection Lastly check solution in both equations


21/75

21

Solving by SubstitutionSolving by SubstitutionT

his method works by solving one of the equations forone of the variables, and then plugging this into the other

equation, "substituting" for the chosen variable and

solving for the other. Then back-solve for the first

variable.

Example 4Example 4

Solve the following system by substitution.

2x 3y= 2

4x+ y= 24


22/75

22

4x+y= 24y= 4x+24

Substitute for "y" in the first equation, and solve forx:

2x 3(4x+24) = 22x+ 12x 72 = 2

14x= 70x= 5

Solution

y= 4(5) + 24 = 20 + 24 = 4

Then the solution is (x, y) = (5, 4).


23/75

23

The addition method is also called the method of

elimination. If we have the equation "x + 6 = 11", you

would write "6" under either side of the equation, and

add down to get "x=

5" as the solution.

x+ 6 =11

6 6

x = 5

Solving by AdditionSolving by Addition


24/75

24

Solve the following system using addition.

2x+ y= 93x y= 16

Example 5Example 5

Solution

2x+ y= 9

3x y=16

5x = 25

with x = 5, and then back solve, using either of the original

equations, to find the value ofy. Using the first equation:2(5) + y= 9

10 + y= 9

y= 1

Then the solution is (x, y)

=(5,

1).


25/75

25

Solve the following using addition.

12x 3y= 6

4x y= 2

Exercise 1Exercise 1


26/75

26

Solving Linear System in Three orSolving Linear System in Three or

More VariablesMore VariablesMethods for solving linear equations of three variables:

Direct methods: find the exact solution in a finitenumber of steps

Iterative methods: produce a sequence a sequenceof approximate solutions hopefully converging to the

exact solution.

Matrix algebra is used to solve a system of linear

equations.


27/75

27

Linear SystemsLinear Systems

/

.

.

.

3333232131

2323222121

1313212111

bxaxaxa

bxaxaxa

bxaxaxa

!

!!

!

//1///

.

.

.

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa


28/75

28

Matrix AlgebraMatrix Algebra

System ofm linear equations in n unknowns

Matrix-vector notation Extended coefficient matrix


29/75

29

Solving Linear Systems

Solve Ax=b, where Ais an nvn matrix andb is an nv1 column vector

Can also talk about non-square systems

whereA is mvn, b is mv1, andxis nv1

Overdeterminedifm>n:more equations than unknowns

Underdeterminedifn>m:more unknowns than equationsCan look for best solution using least squares


30/75

30


Recap from Lecture 2:

1. Inverting matrix

2. Cramers Rule


31/75

31


Recap from Lecture 2:

Inverting matrix

Usually not a good idea to computex=A-1b

Inefficient

Prone to round off error


32/75

32

Echelon Form of a Matrix

An mxn matrix A is said to be row echelon form, if it satisfiesthe following properties:

1. All zero rows, if there any, appear at the bottom of the

matrix.2. Each leading entry (or the first nonzero entry from the

left) of a row is in a column to the right of the leadingentry of row above it.

3. All entries in a column below a leading entry are zeros.


33/75

33

Echelon Form of a Matrix

Matrices in echelon form:

000000

0000

0

000

0

00

000 *

****

,*

**

,*

**

,

**

x

x

x

x

x

x

x

x

x

(x) may have any nonzero value and the entries (*) may

have any value including zero


34/75

34

Reduced Echelon Form of a Matrix

An mxn matrix A is said to be reduced row echelon form, ifit satisfies the following properties:

1. The first entry from the left of a nonzero row is a 1. Thisentry is called a leading one of its row.

2. All entries above and below a leading 1 are zeros.

000000

010000

010

000

10

01

100

010

001

100

01***

,*

*

,,*


35/75

35

Elementary Row operation Matrix

An elementary row operation on a matrix A is any one ofthe following operations:

1. Type I: Interchange any two rows..2. Type II: Multiply a row by a nonzero number

3. Type III: Add a multiple of one row to another.

An mxn matrix B is said to be row equivalent to an mxn

matrix AifB can be obtained by applying a finite sequenceof elementary row operations to A.


36/75

36

Echelon Matrix

!

*

*

*

*

000000

**0000

*****0

******

A

Free variablesFree variables


37/75

37

Reduced Row Echelon Matrix

Free variables

!

1

00

0

000000

*10000*0**10

*0**01

A

Free variables


38/75

38

Example 6

B

A

!

!

130

580

122

212

580

122

212

226

122

212

113

122

212

122

123

~

~

~

~

R1R2

2R2R2

-3R1+R2R2

R1+R3R3

A and B are row equivalentA and B are row equivalent


39/75

39

Steps in Row Reduction-Pivoting

1. Begin with the leftmost nonzero column. This is called pivotcolumn.

2. Select a nonzero entry (having the smallest absolute value) in

the pivot column as a pivot element. If a pivot element not atpivot position then use interchange row operations to movethis entry into pivot position.

3. Perform row reduction into row echelon form(Obtain 0 below the pivot element using row replacement operations byadding suitable multiple of the top row to the row below that)

and row reduction into reduced row echelon form.(Obtain 0 above and below the pivot element using row replacement by

adding suitable multiple of the top row to the row below it)

4. Repeat (1) to (3) on the matrix consisting of the remainingrows.


40/75

40

Example 7

!

2342

3211

8312

3113

A

Apply elementary row operations to transform thefollowing matrix into echelon form

Solution

1. Compute the vector for checking column.

2. Follow the aforementioned steps.


41/75

41

Example 7

!

0000

4100

8120

3211

A

Row echelon form of matrixA


42/75


43/75

43

The rank is the number of the pivots ofA, which is

also the same as the number of nonzero rows of

an echelon form ofA. To compute it, we reduceA

to echelon form and count the number of nonzero

rows or the number of pivot columns.

Rank of a MatrixRank of a Matrix

Example 8Compute the rank of the following matrix

!

321

010

123

A


44/75

44

Example 8Example 8

Solution

!

800

010

321

840

010

321

123

010

321

321

010

123

~

~

~

A

There are 3nonzero rows,

hence the rank

(A)=3


45/75

45

Exercise 3Exercise 3

!

420

121

210

A

Find the rank of matrixA


46/75

46


Gaussian Elimination

Gauss Jordan Elimination


47/75

47

Gauss Elimination Method

Form the augmented matrix (Ab) corresponding to Ax=b.

Transform augmented matrix (A b) to row echelon matrix(Ud).

Solution (if any)xto the system obtained by solving the

linear system Ux=dcorresponding to (U d) using backwardsubstitution.


48/75

48


Solve the following linear system using Gauss elimination.

a. x1+2x2+3x3=62x1-3x2+2x3=14

3x1+x2-x3=-2

b. x1+2x2+3x3=64x1+5x2+6x3=24

2x1+7x2-12x3=-2

c. 3x1-5x2+2x3=6x1+2x2-x3=1-x1+9x2-4x3=-4


49/75

49


Solution:

a. Unique solution (r=n)

b. No solution

c. Infinite solution r


50/75

50


Gaussian Elimination Method for Solving M x = b

A Direct MethodFinite Termination for exact result (ignoring round off)

Produces accurate results for a broad range of

matrices

Computationally Expensive


51/75


52/75


53/75

53

Gauss-Jordan Elimination Method

Solution:

a. Unique solution (r=n)

(Also unique solution ifA

0)

b. No solution

c. Infinite solution r


54/75

54

Consistency of Solutions

The linear system of equations Ax=b has asolution, or said to be consistent IFF

Rank{A}=Rank{A|b}

A system is inconsistent whenRank{A}


55/75

55

LU Factorization Method

Variant of Gaussian elimination that decomposes a matrix asa product of a lower triangular and an upper triangular matrix.

Widely used method on computer for solving a linear system.

When Uis an upper triangular matrix all of whose diagonal

entries are different from zero, then the linear system UX=B

can be solved without transforming the augmented matrix

[UB] to reduced row echelon form.

This is the preferred general method for solving linearThis is the preferred general method for solving linear

equations.equations.


56/75

56


nb

MMMM

bMMM

buu

buuu

...

...

...

...

...

000

0

3

22322

1131211

The solution is obtained by the following algorithm

.,,...,,, 121

1

11

11

1

!

!

!

!

!

nnju

xub

x

u

xubx

u

bx

jj

j

nk

kjkj

j

nn

nnnnn

nn

nn

This is merely back substitution


57/75

57


nnnnnnb

M

b

b

b

llll

MMMM

lll

ll

l

3

2

1

321

333231

2221

11

0

00

000

...

...

...

...

...

In similar manner, ifL is a lower triangular matrix of whose all diagonalentries are different from zero, then the linear system LX=B can be solved

by forward substitution.

.,...,, njl

xlb

x

lxlbx

l

bx

jj

j

j

j

j 2

1

1

22

1212

2

11

1

1

!

!

!

!

!

The solution is given by


58/75

58

Example 9

Solve the linear system

134

3226

102

428

25

10

21

1

2

1

!

!

!

!

!!

xxx

x

x

x

j

5x1 =10

4x1-2x2 =28

2x1+3x2+4x3 =26

Solution


59/75


60/75

60


!

)(

)(

1

)(

11

)3(

3

)3(

33

)2(

2

)2(

23

)2(

22

)1(

1

)1(

13

)1(

12

)1(

11

4,3,2,1,

3,12,11,1

2,31,3

1,2

0000

000

00

0

1

1

0

001

0001

00001

n

nn

n

nn

n

nn

n

n

n

nnnn

nnn

a

aa

aa

aaa

aaaa

mmmm

mmm

mm

m

A////

.

.

.

.

/.

///

.

.

.

Compact storage: The diagonal entries of L matrix are all 1s,they dont need to be stored. LU is stored in a single matrix.

There are infinitely many different ways to decompose A.Most popular one: U=Gaussian eliminated matrix

L=Multipliers used for elimination


61/75

61

1 0 2 3 1* 2 0 * 0 1* 3 0 * 0.5 2 3

0.5 1 0 0.5 0.5* 2 1* 0 0.5* 3 1* 0.5 1 2

! ! !

LU


Suppose we are given:

Then we can writeA= LU where:

Lets check that:

2 3

1 2

!

A

1 0

0.5 1

!

L

2 3

0 0.5

!

U


62/75

62

Example 10


6x1-2x2-4x3+4x4 =2

3x1-3x2 -6x3+x4 =-4

-12x1+8x2+21x3-8x4 =8

-6x1-10x3+7x4 =-43

!

!

1211

0122

0012

10001

71006

821812

1633

4426

L

A

!

8000

2500

1420

4426

U

!

43

8

4

2

B

Solution:


63/75

63

Example 10

Then solve AX=B by writing LUX=B. Let UX=Zand solve LZ=B

!

43

8

4

2

1211

0122

0012

1

0001

4

3

2

1

z

z

z

z

By forward substitution,

32243

2228

52

14

2

3214

213

12

1

!!

!!

!!

!

zzzz

zzz

zz

z


64/75

64

Example 10

Solve UX=Z,

!

32

2

5

2

8000

2500

1420

4426

4

3

2

1

x

x

x

x

Hence,

546

4422

962

45

215

22

48

32

432

1

43

2

1

4

3

4

.

.

.

!

!

!

!

!

!

!

!

xxxx

xxx

xx

x


65/75

65

Decomposition Methods

Solve UX=Z,

Hence,

Doolittle decomposition

Crout decomposition

Cholesky decomposition (for symmetric

matrices)


66/75

66

Crout Decomposition

Solve UX=Z,

Hence,

!

!!

!

!

!

3323321331223212311131

2313212212211121

131211

333231

232221

131211

33

2322

131211

3231

21

3

2

1

3

2

1

333231

232221

131211

00

0

1

01

001

uululululul

uuluululuuu

aaa

aaaaaa

u

uu

uuu

ll

lLUA

x

x

x

X

b

b

b

B

aaa

aaa

aaa

A ,,


67/75

67

Crout Decomposition

Solve UX=Z,

Hence,

233213

11

31

3333

12

11

21

2212

11

31

3232

11

31

31

13

11

21

232312

11

21

2222

11

21

21

1313

1212

1111

ulaa

aau

aa

aaa

a

aau

a

al

aaaaua

aaau

aal

au

au

au

!

!!

!!!

!

!

!

,/,

,,


68/75

68

Crout Decomposition

Solve UX=Z,

Hence,

Thus the matrices L and Ubecome known. NowAX=B

becomes

LUX=B LY=B, where Y= UX


69/75

69

Example 11


3x1+2x2+7x3 =4

2x1+3x2 +x3 =5

3x1+4x2+x3 =7

!

!!

3323321331223212311131

2313212212211121

131211

33

2322

131211

3231

21

143

132

723

00

0

1

01

001

uululululul

uuluulul

uuu

u

uu

uuu

ll

lLA

Solution:


70/75

70

Example 11

713

11

5

61

521433

3

77

212

3

23

3

2

7

2

3

33

3231

232221

13

12

11

!

!!

!!!

!

!

!

u

ul

uul

u

u

u

,/,

,,


71/75

71

Example 11

!

5

8

00

3

11

3

50

723

15

6

1

013

2

001

A

Write UX=Ywhich gives LY=B

!

!

5

1

3

74

7

5

4

15

61

013

2021

3

2

1

3

2

1

y

yy

y

yy


72/75

72

Example 11

!

!

8

7

8

9

8

7

5

1

3

7

4

5

800

3

11

3

50

723

3

2

1

3

2

1

x

x

x

x

x

x

Hence the original system reduces to

!

!

5

1

3

74

7

5

4

15

61

013

2021

3

2

1

3

2

1

x

x

x

x

x

x


73/75

73

Steps in LU Decomposition


74/75

74

Iterative Methods

If systems of linear equations are very large,the computational effort of direct methods is

prohibitively expensive

Three common classical iterative techniques forlinear systems

The Jacobi method

Gauss-Seidel method


75/75

Lecture3(Linear Equation)

Documents

Transcript of Lecture3(Linear Equation)