Lecture 17Introduction to Eigenvalue Problems
Shang-Hua Teng
Eigenvalue Problems• Eigenvalue problems occur in many areas of
science and engineering– E.g., Structure analysis
• It is important for analyzing numerical and linear algebra algorithms– Impact of roundoff errors and precision requirement
• It is widely used in information management and web-search
• It is the key ingredient for the analysis of Markov process, sampling algorithms, and various approximation algorithms in computer science
Eigenvalues and Eigenvectors
• Standard Eigenvalue Problem: Given an n by n matrix A, find a scalar and nonzero vector x such that
A x = x is eigenvalue, and x is corresponding
eigenvector
Spectrum of Matrices
• Spectrum(A) = (A) = set of all eigenvalues of A
• Spectral radius (A) = (A) = max {||: in (A)}
• Spectral analysis
• Spectral methods
Geometric Interpretation
• Matrix expands or shrinks any vector lying in direction of eigenvector by scalar factor
• Expansion of contraction factor given by corresponding eigenvalue
• Eigenvalues and eigenvectors decompose complicated behavior of general linear transformation into simpler actions
Examples: Eigenvalues and Eigenvectors
1
0,2 ,
0
1,1
20
01
2211 xx
A
Note: x1 and x2 are perpendicular to each other
Examples: Eigenvalues and Eigenvectors
1
1,2 ,
0
1,1
20
11
2211 xx
A
Note: x1 and x2 are not perpendicular to each other
Examples: Eigenvalues and Eigenvectors
1
1,4 ,
1
1,2
31
13
2211 xx
A
Note: x1 and x2 are perpendicular to each other
Examples: Eigenvalues and Eigenvectors
1
1,1 ,
1
1,2
5.15.0
5.05.1
2211 xx
A
Note: x1 and x2 are perpendicular to each other
Examples: Eigenvalues and Eigenvectors
1 :where
1, ,
1,
01
10
2211
- i
ixi
ixi
A
Note: x1 and x2 are not perpendicular to each other : eigenvalues or eigenvectors may not be real!!!
Simple Facts of Eigenvalue Problem
• If (,x) is a eigenvalue-eigenvector pair of A, then for any k, (k, x) is a eigenvalue-eigenvector pair of Ak.
• If (,x) is a eigenvalue-eigenvector pair of A, then for any c, (c, x) is a eigenvalue-eigenvector pair of cA.
Algebraic Interpretation:Equation for the Eigenvalues
A x = x
(A - I ) x = 0• The eigenvectors make up the
nullspace of (A – I ) if we know .
Eigenvalue First
• If (A - I ) x = 0 has a nonzero solution, then– A - I is not invertible
– The determinant of A - I must be zero.
Characteristic Equation for Eigenvalues
• The number l is an eigenvalue of A if and only if (A - I ) is singular:
det( A - I ) = 0
Characteristic Polynomial for Eigenvalues
• det (A - I ) = 0 is a polynomial in of degree at most n.
• The spectrum of A is the set of roots of this characteristic polynomial:
• Fundamental Theorem of Algebra implies that n by n matrix A always has n eigenvalues, but they need be neither distinct nor real
Examples: Characteristic Polynomial
So Spectrum(A) = {1,2}
2120
01det
10
01
20
01det)det(
20
01
IA
A
Examples: Characteristic Polynomial
So Spectrum(A) = {2, 4}
4286
13331
13det
10
01
31
13det)det(
31
13
2
IA
A
Examples: Characteristic Polynomial
So Spectrum(A) = {i, -i}
ii
IA
A
11
1det
10
01
01
10det)det(
01
10
2
A Possible Methods for Solving the Eigenvalue Problems
• Compute the characteristic polynomial of A in by expanding det(A – I) = 0
• Find the roots of the characteristic polynomial
• For each eigenvalue , solve (A – I) x=0 to find an eigenvector x.
Practical Difficulties
• Computing eigenvalues using characteristic polynimial is not recommended or used because– Roots of polynomial of degree > 4 cannot
always be computed in finite number of steps– A lot of work is needed in computing
coefficients of the characteristic polynomial– Computer has round-off errors
Examples: Characteristic Polynomial
So Spectrum(A) = {1+, 1-}
11
111
1det )det(
an smaller thslighly number a is where 1
1
2
machine
IA
A
But in machine, 2 < machine is equal to 0So, the algorithm will returnSpectrum(A) = {1,1}
Theory and Practice
• Characteristic polynomial is a powerful theoretical tool but usually is not useful computationally.
Special Matrix
• What is Spectrum( I )?– Multiplicity is the number of times root appears
when polynomial written as product of linear factors
– det(I – I ) = (1-)n
• What is Spectrum( upper or lower triangular matrix )?
Bad News
• Elimination does not preserve the ’s.
2 and 0 has 11
11
1 and 0 has 00
11
21
21
Diagonalizing A Matrix• Suppose the n by n matrix A has n linearly
independent eigenvectors x1, x2,…, xn.
• Eigenvector matrix S: x1, x2,…, xn are columns of S.
• Then
n
ASS
11
is the eigenvalue matrix
Matrix Power Ak
• S-1AS = implies A = S S-1
• implies A2 = S S-1 S S-1 = S S-1
• implies Ak = S kS-1
Random walks
How long does it take to get completely lost?
0
0
0
0
0
1
Random walks Transition Matrix
1
2
345
6
0
0
0
0
0
1
02
1
4
100
2
13
10
4
1000
3
1
2
10
2
1
3
10
004
10
3
10
004
1
2
10
2
13
1000
3
10
100
P
Matrix Powers
• If A is diagonalizable as A = S S-1 then for any vector u, we can compute Aku efficiently
– Solve S c = u– Aku = S kS-1 S c = S k c
• As if A is a diagonal matrix!!!!
Independent Eigenvectors from Different Eigenvalues
• Eigenvectors x1, x2,…, xk that correspond to distinct (all different) eigenvalues are linear independent.
• An n by n matrix that has n different eigenvalues (no repeated ’s) must be diagonalizable
Proof: Show that
implies all ci = 0
011 kk xcxc
Top Related