Eigen Values, Eigen Vectors_afzaal_1

8/13/2019 Eigen Values, Eigen Vectors_afzaal_1

1/81

Linear Algebra

Eigenvalues, Eigenvectors


2/81

2

Eigenvalues and Eigenvectors

If A is an n x n matrix and x is a vector in , then

there is usually no general geometric relationship

between the vector x and the vector Ax. However,

there are often certain nonzero vectors x such that x

and Ax are scalar multiples of one another. Suchvectors arise naturally in the study of vibrations,

electrical systems, chemical reactions, quantum

mechanics, mechanical stress, economics and

geometry.

nR


3/81

3

Eigenvalues, Eigenvectors

If A is an n n, then a non zero vector xin Rnis called an eigenvectorof Aif Axis a scalarmultiple of x; that is

for some scalar . The scalar is called aneigenvalue of A, and x is called theeigenvectorof A corresponding to .

Eigen values are also called the propervalues or characteristic values they arealso called the Latent roots.

Ax x


4/81


5/81

5

Graphical Interpretation

Eigen values and Eigen vectors have also a useful

graphical interpretation in R2 and R3. If is the

eigenvalue of A corresponding to x then Ax=x so

that the multiplication by A dilates x, contracts x, or

reverses the direction of x, depending upon the value.

Dilation(>1) Contraction(0<


6/81

6

Finding Eigen values

To find the eigenvalues of nn matrix A we

write Ax=x as

For to be the eigenvalue ,there must be a

nonzero solution of the above equation.

However above equation will have a nonzerosolution if and only if

( ) 0

Ax x

I A x

det( ) 0I A


7/81

7

Characteristic Equation

det(I-A)=0 is called the characteristic

equation of A; the scalars satisfying this

equation are called the eigenvalues of A.

When expanded , the determinant det(I-A) isa polynomial in called the characteristic

polynomialin A.

Example: Find the eigenvalues and

eigenvectors of the matrix

3 2

1 0A


8/81

8

Example 1(Eigenvalues)

Solution:

The characteristic polynomial in A.

Characteristic Equation:

1 0 3 2

0 1 1 0

3 2

1

I A

I A

23 2det( ) 3 21

I A

2

3 2 0


9/81

9

Example 1(Eigenvectors)

The solution of this equation are =1 and =2;

these are the eigenvalues of A.

Since Ax=x

Eigenvector corresponding to =1

Putting =1 gives

1

2

3 20

1

x

x

1

2

2 20

1 1

x

x


10/81

10


The above matrix gives two equations

which gives the general solution in the form

1 2x x

1 2

1 2

2 2 0

0

x x

x x

2

1x 1 1 xLet

1

Then

The eigenvector corresponding to is

1

1


11/81

11


Similarly 12

1

2

1 2

1 2

1 2

2 1

3 20

1

Putting 2

1 2

0givesequations1 2

2 0

2 0

Thesolution of aboveequations is

2

1 2

Eigenvector correspponding to 2is

2

1

x

x

x

x

x x

x x

x x

let x then x


12/81

12

Multiple Eigenvalues

The order Mof an eigenvalue as a root of

characteristic polynomial is called the

algebraic multiplicityof .

The number m of linearly independenteigenvectors corresponding to is called the

geometric multiplicityof .

Thus m is the dimension of the eigenspace

corresponding to this .


13/81

13

Multiple Eigenvalues

The characteristic polynomial has a degree

n, the sum of all algebraic multiplicity must

equal n.

In general m M The defectof is defined as

Example :Find the eigenvalues and

eigenvectors of the matrix

M m

2 2 3

2 1 6

1 2 0

A


14/81

14

Example 2(1/7)

Eigenvalues

The characteristic polynomial

have three latent root (eigenvalues) as

2 2 3

2 1 6

1 2

det( ) 0

A I

A I

3 2 21 45 0

1 2 35, 3


15/81

15

Example 2(2/7)

The algebraic multiplicity of =-3 is 2

M-3=2

and that of =5 is M5=1

Eigenvectors: To find the eigenvectors apply

the gauss elimination to the system (A-I)x=0

first =5 and =-3.

Putting =5 gives1

2

3

7 2 3

( ) 2 4 6 0

1 2 5

x

A I x x

x


16/81

16

Example 2(3/7)

Performing Gauss elimination on the matrix

above gives1 2 5

0 1 2

0 0 0

A I

1 2 3

2 3

2 5 0

2 0

x x x

x x


17/81

17

Example 2(4/7)

The above matrix can be written in the

equation form which gives

1 2 3

2 3

3 2 1

1

(2 5 )

2

Let 1 then 2 and 1

The eigen vector corresponding to 5

1

2

1

x x x

x x

x x x


18/81

18

Example 2(5/7)

Putting =-3 and performing Gauss

Elimination gives

1 2 3

3 0 0 0

0 0 0

A I

1 2 3

1 2 3

2 3 0

2 3

x x x

or

x x x


19/81

19

Example 2(6/7)

Or

So there are two linearly independent vectors

corresponding to

1

0

3

0

1

2

32

3

2

1

xx

x

x

x

3


20/81

20

Example 2(7/7)

Since there are two Linear Independent

vectors corresponding to =-3 the geometric

multiplicity of

m-3=2m5=1

The defect of =-3 is

-3 =M-3m-3-3=0

5=0


21/81

21

Example:

Find the Eigenvalues and Eigenvectors of the matrix

Solution:

The algebraic multiplicity is

00

10A

0

1

10

01

00

10IA

001

det 2

IA

021

20 MM


22/81

22

Example (Contd)

To find Eigenvector

The solution is

The Eigenvector is

The geometric multiplicity is

Thus

0

0

00

10

2

1

x

x

12 .0xx

0

1

12

1

xx

x

10 mm

00 Mm


23/81

23

Theorems

If A is an nn triangular matrix (upper

triangular, lower triangular or diagonal) then

the eigenvalues of A are entries on main

diagonal of A. Example :

1 2 3

1 0 0

2

21 0

3

1 5 8

4

TheEigenvaluesare

1 2 1

, ,2 3 4

A


24/81

24

Theorems


25/81

25

Theorems


26/81

26

Example


27/81

27

Theorems

If k is a positive integer, is an eigenvalue of

a matrix A, and x is a corresponding

eigenvector, then k is an eigenvalue of Ak

and x is a corresponding eigenvector.


28/81

28

Example:

Eigenvalues of are:

1282,11 7327

1

1 2 3

0 0 2

If 1 2 1

1 0 3then find Eigenvalues for if eigenvalues

for are 1, 2

A

7A

A

7A


29/81

29

Theorems


30/81

30

Theorems

If x is an eigenvector of a matrix A to an

eigenvalue ,so is kxwith any k 0.


31/81

31

Example

Let

The eigenvalues of A are 3, 0, and 2. Theeigenvalues of B are 4 and 1.

What does it mean for a matrix to have an eigenvalueof 0 ?

This happens if and only if the equation has

a nontrivial solution. Equivalently, Ax=0 has anontrivial soltion if and only if A is not invertible. Thus

0 is an eigenvalue of A if and only if A is notinvertible.

435

012

004

and

200

600

863

BA

0xxA


32/81

32

Diagonalization

The Eigenvector Problem:

Given an matrix A, does there exist abasis for consisting of eigenvectors of A?

The Diagonalization Problem:Given an matrix A, does there exist aninvertible matrix P such that isa diagonal matrix?

Apparently above two problems are different butthey are equivalent thats why diagonalizationproblem is considered in the discussion ofeigenvectors.

n nnR

n n1D P AP


33/81

33

Diagonalizable Matrix

Definition:

A square matrix A is said to bediagonalizable if there exist an invertiblematrix P such that is a diagonalmatrix.

Theorem:

If A is an matrix then the following are

equivalent: A is diagonalizable

A has n linearly independent eigenvectors.

1D P AP

n n


34/81

34

Procedure for Diagonalization

1. Find eigenvalues of A.

2. Find n linearly independent eigenvectors

of A, say,

3. Form matrix P having asits columns.

4. Construct . The matrix D will be

diagonal with as itssuccessive diagonal entries, where is the

eigenvalue corresponding to .

1 2 3 4, , , ,..., .np p p p p

1 2 3 4, , , ,..., np p p p p

1D P AP

1 2 3 4, , , ,...,

n

i

, 1, 2,...,ip i n


35/81

35

Example 1 (1/10)

Find a matrix P that diagonalizes

0 0 2

1 2 1

1 0 3

A

Step 1

Find the eigenvalues of A

Solution:

1 0 0 0 0 2

0 1 0 1 2 1

0 0 1 1 0 3

I A


36/81

36

0 0 0 0 2

0 0 1 2 1

0 0 1 0 3

0 2 1 2 1

1 0 3

0 2

det( ) 1 2 1

1 0 3

I A

Example 1 (2/10)


37/81

37

The characteristic equation of A is3 25 8 4 0

In factored form

2

1 2 0 Verify!

Thus, the eigenvalues of A are

1 2 31, 2

Example 1 (3/10)


38/81

38

Step 2

Find the corresponding eigenvectors

By definition1

2

3

x

x

x

x

is an eigenvector of A corresponding to if and

only if x is a non-trivial solution of

that is

0I A x

1

2

3

0 2 0

1 2 1 0

1 0 3 0

x

x

x

Example 1 (4/10)


39/81

39

If , then21

2

3

2 0 2 0

1 0 1 0

1 0 1 0

x

x

x

Solving this system yields

1 3 x x x2,,x3 (free variables)

The eigenvectors of A corresponding to

are nonzero vectors of the form 2

0 1 0

0 0 1

0 1 0

s s

t t s t

s s

x

Example 1 (5/10)


40/81

40

As

are linearly independent, so they form the basisfor eigenspace corresponding to 2

1 0

0 , 1

1 0

If , then1

1

2

3

1 0 2 0

1 1 1 0

1 0 2 0

x

x

x

Example 1 (6/10)


41/81

41

Solving this system yields

1 32 x x x3 (free variable)

The eigenvectors corresponding to are

nonzero vectors of the form

1

2 2

1

1

s

s s

s

x So2

1

1

2 3x x

=>

1 2 3

1 0 2

0 , = 1 , 1

1 0 1

p p p

Example 1 (7/10)


42/81

42

Step 3 It is easy to verify that are

linearly independent so that

diagonalizes A.

Step 4 The resulting diagonal matrix is

1 2 3{ , , }p p p

1 0 20 1 1

1 0 1

P

1

2 0 0

0 2 0

0 0 1

D P AP

Example 1 (8/10)


43/81

43

To verify that

1

2 0 0

0 2 0

0 0 1

A P P

We can compute

PA DP Verify it for the given case

Example 1 (9/10)


44/81

44

Remark:

There is no preferred order for the columns of

P. Since the ithdiagonal entry of is an

eigenvalue corresponding to ith

column vectorof P, changing the order of columns of P will

change the order of eigenvalues on the

diagonal of .

1P AP

1P AP

1 2 0

0 1 1

1 1 0

P

For example

1

2 0 0

0 1 0

0 0 2

D P AP=>

Example 1 (10/10)


45/81

45

Example 2 (1/3)

Diagonalize

The characteristic equation of A is

Thus, is the only eigenvalue of A, and

corresponding eigenvectors are the solutions of

3 2

2 1

A

1

( ) 0 I A x

2( 1) 0.


46/81


47/81

47

Example (3/3)

Since A does not have two linearly

independent eigenvectors i.e. the eigenspace

is 1-dimensional, therefore, A is not

diagoalizable.


48/81

48

Conditions for Diagonalizability

Theorem 1:

If are eigenvectors of A

corresponding to distinct eigenvalues

,thenare linearly independent.

Theorem 2:

If an matrix A has ndistinct eigenvalues,

then A is diagonalizable.

1 2 3 4, , , ,..., nv v v v v

n n

1 2 3 4, , , ,..., n 1 2 3, , ,..., nv v v v


49/81

49

Example

Whether A is

diagonalizable or not?0 1 0

0 0 1

4 17 8

A

The eigenvalues of A are

1 2 34, 2 3, 2 3

Since A has three distinct eigenvalues, so A can

be diagonalized. We can verify that

1

4 0 0

0 2 3 0

0 0 2 3

D P AP


50/81

50

LU-Factorization

With Gaussian elimination and Gauss-Jordan

elimination, a linear system is solved by

operating systematically on the augmented

matrix.Another approach is based on factoring the

coefficient matrix into a product of lower (L)

and upper (U) triangular matrix (LU-

Decomposition). LU-decomposition speeds up the solution of

Ax b


51/81

S l i Li S t b


52/81

52

Step 3

Use (2) to rewrite (1) as and solve for y

Step 4

Substitute yin (2) and solve for x

L y = b

x b

y

Multiplication by A

Solving Linear Systems by

LU-Factorization


53/81

53

LU Factorization

The LU factorization is motivated by the fairlycommon industrial and business problem of solving asequence of equations, all with the same coefficientmatrix A:

(1)

When A is invertible, one could compute and thencompute and so on. However, it is moreefficient to solve the first equation in (1) by row

reduction and obtain an LU factorization of A at thesame time. Thereafter, the remaining equations in (1)are solved with LU factorization.

n21 bx,,bx,bx AAA 1A

,, 21

1

1 bAbA


54/81

E l S l i Li


55/81

55

1

2

3

2 0 0 1 3 1 2

3 1 0 0 1 3 24 3 7 0 0 1 3

x

x

x

Step 1

Rewriting the system as

Step 2

Defining vector y as y=[y1, y2, y3]T

1 1

2 2

3 3

1 3 1

0 1 3

0 0 1

x y

x y

x y

ExampleSolving Linear

Systems by LU-Factorization

E ample Sol ing Linear


56/81

56

Equivalently1

1 2

1 2 3

2 2

3 2

4 3 7 3

y

y y

y y y

Solving using forward substitution, we get

1 2 31, 5, 2y y y

Step 3

1

2

3

1 3 1 1

0 1 3 5

0 0 1 2

x

x

x

ExampleSolving Linear

Systems by LU-Factorization


57/81


58/81

58

LU decomposition

Theorem:

If A is a square matrix that can be reduced to

a row-echelon form U without using row

interchanges, then A can be factored asA = LU, where L is a lower triangular matrix.


59/81

59

Procedure for LU decomposition

Step 1 : Reduce A to a row-echelon form U without

row interchanges, keeping track of the multipliers

used to introduce the leading 1sand the multipliers

used to introduce the zeros below the leading 1s.

Step 2: In each position along the main diagonal ofL, place the reciprocal of the multiplier that introduced

the leading 1 in that position in U.

Step 3: In each position below the main diagonal of

L, place the negative of the multiplier used tointroduce the zero in that position in U.

Step 4: Form the decomposition A=L U

LU Decomposition


60/81

60

LU Decomposition

Example

Find an LU-decomposition of

Solution: We begin by reducing A to row-echelon form,keeping track of all multipliers.

573

119

026

A

LU Decomposition


61/81

61

LU Decomposition

Example

573

119

026

573

119

0311 61Multiplier

580

120

0311

3Multiplier

9Multiplier

5802

110

03

11

2

1Multiplier

1002

110

0311

8Multiplier

1002

110

0311

1Multiplier

LU Decomposition


62/81

62

LU Decomposition

Example

Constructing L from the multipliers yields the LU-

decomposition

1002

1100311

183

029006

LUA

Iterative Solution of Linear


63/81

63

Iterative Solution of Linear

Systems

Although Gaussian elimination or Gauss-

Jordan elimination is generally the method of

choice for solving a linear system of n

equations in n unknowns, there are otherapproaches to solving linear systems, called

iterativeor indirect methods, that are better in

certain situations.

The Gauss-Seidel method is the mostcommonly used iterative method.


64/81

64

Gauss-Seidel Method

Basic Procedure:

Algebraically solve each linear equation for xi

Assume an initial guess

Solve for each xiand repeat

Use absolute relative approximate error after each

iteration to check if error is within a prespecified

tolerance.


65/81

65

A set of nequations in nunknowns:

11313212111 ... bxaxaxaxa nn

22323222121 bxa...xaxaxa nn

nnnnnnn

bxaxaxaxa ...332211

. .

. .

. .

Gauss-Seidel Method (Algorithm)

The system Ax=b is reshaped by solving the firstequation for x1, the second equation for x2, and thethird forx3, and n

thequation forxn.


66/81

66

Rewriting each equation

11

131321211

a

xaxaxabx nn

nn

nn,nnnn

n

,nn

n,nnn,nn,n,nn

n

nn

a

xaxaxabx

a

xaxaxaxab

x

axaxaxabx

112211

11

12212211111

1

22

232312122



67/81

67

Solve for the unknowns

Assume an initial guess forx

n

-n

2

x

x

x

x

1

1

Use rewritten equations to

solve for each value of xi.

Important:Remember to

use the most recent value

of xi. Which means to

apply values calculated to

the calculations remainingin the current iteration.



68/81

68

Calculate the Absolute Relative Approximate Error

100new old

i i

x newi

i

x x

x

So when has the answer been found?

The iterations are stopped when the absolute relative

approximate error is less than a pre-specified tolerance for all

unknowns.



69/81


70/81

70

Gauss-Seidel Method: Example 1

Given the system of equations

15312 321 x-xx

2835 321 xxx

761373 321 xxx

1

0

1

3

2

1

x

x

x

With an initial guess of

The coefficient matrix is:

12 3 5

1 5 33 7 13

A

Will the solution convergeusing the Gauss-Seidel

method?


71/81

71


12 3 5

1 5 3

3 7 13

A

Checking if the coefficient matrix is diagonally dominant

43155232122

aaa

10731313 323133 aaa

8531212 131211 aaa

The inequalities are all true and hence the matrix A is

strictlydiagonally dominant.

Therefore: The solution should converge using the Gauss-

Seidel Method


72/81

72


1

2

3

12 3 5 1

1 5 3 28

3 7 13 76

x

x

x

Rewriting each equation

12

531 321

xxx

5328 312 xxx

13

7376 213

xxx

With an initial guess of

1

0

1

3

2

1

x

x

x

50000.0

12

150311

x

9000.4

5

135.0282

x

0923.3

13

9000.4750000.03763

x


73/81


74/81

74


8118.3

7153.314679.0

3

2

1

x

xx

After Iteration #1

14679.0

12

0923.359000.4311

x

7153.35

0923.3314679.0282

x

8118.3

13

900.4714679.03763

x

Substituting the x values into the equations

After Iteration #2

0923.3

9000.4

5000.0

3

2

1

x

x

x


75/81

75


Iteration #2 absolute relative approximate error

x 1

0.14679 0.50000100 240.62%

0.14679

x 2 3.7153 4.9000 100 31.887%3.7153

x 3

3.8118 3.0923100 18.876%

3.8118

The maximum absolute relative error after the first iterationis 240.62%

This is much larger than the maximum absolute relative

error obtained in iteration #1. Is this a problem?


76/81

76


Repeating more iterations, the following values are obtained

1x

2x

3x

Iteration x1 x2 x3

1

23

4

5

6

0.50000

0.146790.74275

0.94675

0.99177

0.99919

67.662

240.6280.23

21.547

4.5394

0.74260

4.900

3.71533.1644

3.0281

3.0034

3.0001

100.00

31.88717.409

4.5012

0.82240

0.11000

3.0923

3.81183.9708

3.9971

4.0001

4.0001

67.662

18.8764.0042

0.65798

0.07499

0.00000

4

3

1

3

2

1

x

x

x

0001.4

0001.3

99919.0

3

2

1

x

x

xThe solution obtained

is close to the exact solution of


77/81

77


Solve the following linear system by Gauss-Seidel method.

1 2 3

1 2 3

1 2 3

3 2

6 4 11 1 5 2 2 9

x x x

x x xx x x

Solution

We can verify that the system is not strictlydiagonally dominant, so the Gauss-Seidel

method does not guaranteed to converge.

1 3 1

6 4 11

5 2 2

A


78/81

78

Definiteness of Matrices

A matrix is Positive Definite iff xTAx>0 for allx.

A matrix is Negative Definiteiff xTAx0 for some x

and xTAx


79/81

79

Checking for Definiteness

There are different methods to check thedefiniteness of a matrix. One method is based

on Eigenvalues of the given matrix. If all

eigenvalues of the matrix are positive, then

the matrix would be positive definite. If all the

eigenvalues are negative, then the matrix

would be negative definite.


80/81

Checking for Definiteness -


81/81

Checking for Definiteness

Example

Check the definiteness of the matrix.

12 2

2 10A

Solution

12 2

2 10

12 2

2903

1 2

1

6E E

Since the two diagonal elements are +ve,

therefore matrix A is +ve definite.

Eigen Values, Eigen Vectors_afzaal_1

Documents

Transcript of Eigen Values, Eigen Vectors_afzaal_1