Eigenvalues and eigenvectors
description
Transcript of Eigenvalues and eigenvectors
Eigenvalues and eigenvectors
BirthsDeaths
Population increase
Population increase = Births – deaths
t
Equilibrium
tttttttttt NdbNdNbNN )1(1
N: population sizeb: birthrated: deathrate
ttt
t
t
t
t
tt
t
tt
t
tt
t
tt
dbN
deathsN
birthsNdeathsbirths
NNr
Ndeathsd
Nbirthsb
tt RNN 1
The net reproduction rate R = (1+bt-dt)
If the population is age structured and contains k age classes we get
k
ikkkk NbNbNbNbN
122110 ...
The numbers of surviving individuals from class i to class j are given by
211
122
011
)1(...
)1()1(
kkk NdN
NdNNdN
Leslie matrix
Assume you have a population of organisms that is age structured.Let fX denote the fecundity (rate of reproduction) at age class x.
Let sx denote the fraction of individuals that survives to the next age class x+1 (survival rates).Let nx denote the number of individuals at age class x
We can denote this assumptions in a matrix model called the Leslie model. We have w-1 age classes, w is the maximum age of an individual.
L is a square matrix.
1
2
1
0
...
wn
nnn
tN
000000...............0...0000...0000...000
...
2
2
1
0
13210
w
w
s
ss
sfffff
L
tt LNN 1
Numbers per age class at time t=1 are the dot product of the Leslie matrix with the abundance vector N at time t
01 NLN tt
k
ikkkk NbNbNbNbN
12211 ...
211
122
011
)1(...
)1()1(
kkk NdN
NdNNdN
ttn
nnn
s
ss
sfffff
n
nnn
1
2
1
0
2
2
1
0
13210
11
2
1
0
...
...
000000...............0...0000...0000...000
...
...
...
ww
w
w
vThe sum of all fecundities gives
the number of newborns
vn0s0 gives the number of
individuals in the first age class
Nw-1sw-2 gives the number of individuals in the last classv
The Leslie model is a linear approach.It assumes stable fecundity and mortality rates
The effect pof the initial age composition disappears over timeAge composition approaches an equilibrium although the whole
population might go extinct.Population growth or decline is often exponential
An example
Age class N0 L1 1000 0 0.5 1.2 1.5 1.1 0.2 0.0052 2000 0.4 0 0 0 0 0 03 2500 0 0.8 0 0 0 0 04 1000 0 0 0.5 0 0 0 05 500 0 0 0 0.3 0 0 06 100 0 0 0 0 0.1 0 07 10 0 0 0 0 0 0.004 0
Generation0 1 2 3 4 5 6 7 8 9 10 11 12
1000 6070.05 4335.002 3216.511 3709.4 3822.356 3338.88 3195.559 3199.811 3037.552 2873.77 2783.134 2681.0592000 400 2428.02 1734.001 1286.604 1483.76 1528.942 1335.552 1278.224 1279.924 1215.021 1149.508 1113.2542500 1600 320 1942.416 1387.201 1029.284 1187.008 1223.154 1068.442 1022.579 1023.939 972.0165 919.60631000 1250 800 160 971.208 693.6003 514.6418 593.504 611.5769 534.2208 511.2894 511.9697 486.0083
500 300 375 240 48 291.3624 208.0801 154.3925 178.0512 183.4731 160.2662 153.3868 153.5909100 50 30 37.5 24 4.8 29.13624 20.80801 15.43925 17.80512 18.34731 16.02662 15.33868
10 0.4 0.2 0.12 0.15 0.096 0.0192 0.116545 0.083232 0.061757 0.07122 0.073389 0.064106
At the long run the population dies out.Reproduction rates
are too low to counterbalance the high mortality rates
0.01
0.1
1
10
100
1000
10000
0 5 10 15 20 25
Abun
danc
e
Time
12345
6
7
Important properties:1. Eventually all age classes
grow or shrink at the same rate
2. Initial growth depends on the age structure
3. Early reproduction contributes more to population growth than late reproduction
Leslie matrix
tt LNN 1
01 NLN tt
0.01
0.1
1
10
100
1000
10000
0 5 10 15 20 25
Abun
danc
e
Time
12345
6
7
Does the Leslie approach predict a stationary point where population abundances doesn’t change any more?
ttt NLNN 1
We’re looking for a vector that doesn’t change direction when multiplied with the Leslie matrix.
This vector is called the eigenvector U of the matrix.Eigenvectors are only defined for square matrices.
ULU
0dtdN
0][0UILULU
I: identity matrix
000000...............0...0000...0000...000
...
1
2
2
1
4321
k
k
d
dd
dfbbbb
L
k
t
n
nnn
...3
2
1
N
ftCefgN
gfNdtdN
The insulin – glycogen system
At high blood glucose levels insulin stimulates glycogen synthesis and inhibits glycogen breakdown.
The change in glycogen concentration can be modelled by the sum of constant production
and concentration dependent breakdown
01
0
gf
N
gfNAt equilibrium we have
01001
10
0
01
1
00111
0
2
2
2
2
gf
NN
gf
NN
Ngf
NN
gfN
TT
The vector {-f,g} is the eigenvector of the dispersion matrix and gives the stationary point.
The value -1 is called the eigenvalue of this system.
X
YHow to transform vector A
into vector B?
A
B
BXA
97
31
5.25.121
Multiplication of a vector with a square matrix defines a new
vector that points to a different direction.
The matrix defines a transformation in space
X
Y
A
B
BXA
Image transformationX contains all the information necesssary
to transform the image
The vectors that don’t change during transformation are the eigenvectors.
AXA
In general we defineUXU
U is the eigenvector and the eigenvalue of the square matrix X
0][0][0
0
UΛXUIXIUXU
UXUUXU
UXU
The basic equation
nnmnmm
n
n
nnmnmm
n
n
u
uu
u
uu
aaa
aaaaaa
u
uu
u
uu
aaa
aaaaaa
......00
.........0...00...0
......
..................
.........
..................
1
1
1
1
21
12221
11211
1
1
1
1
21
12221
11211
The matrices A and L have the same properties. We have diagonalized the matrix A.
We have reduced the information contained in A into a characteristic value , the eigenvalue.
2
1
2
1
2221
1211
uu
uu
aaaa
L
22
11
22
11
2
1
22
11
2221
1211
00
vuvu
vuvu
vuvu
aaaa
A nxn matrix has n eigenvalues and n eigenvectorsSymmetric matrices and their transposes have identical eigenvectors and eigenvalues
vu
00)(
)()(
;
2
1
2
12211
221122112
1
2
1
2
1
2
1
2
1
2221
2111
2
1
2
1
2221
2111
2
1
2
1
2
1
2
1
2221
2111
2
1
2
1
2
1
2
1
2
1
2
1
2221
2111
2
1
2
1
2
1
2221
2111
2
1
2
1
2221
2111
vv
uu
uvuv
uvuvuvuvvv
uu
uu
vv
vv
aaaa
uu
uu
aaaa
vv
vv
uu
yy
aaaa
uu
uu
vv
uu
vv
uu
aaaa
vv
vv
vv
aaaa
uu
uu
aaaa
vu
T
v
T
u
TT
T
v
T
T
uu
TT
vu
Eigenvectors of
symmetric matrices are orthogonal.
1 1
2 1 1 0 2 10 0
3 4 0 1 3 4
(2 )(4 ) 3 1; 5
i i[A I] u 0
1 21 1
2 2 1 2
(2 )u u 0u u2 1 1 0 2 1 1 10 0 u
u u3 4 0 1 3 4 3u (4 )u 0 1 3
2 1 1 1 11
3 4 1 1 1
2 1 1 5 15
3 4 3 15 3
i i iA u u
How to calculate eigenvectors and eigenvalues?
The equation is either zero for the trivial case u=0 or if [A-I] =0
0
000
0][
2221
1211
2221
1211
2221
1211
aaaa
aaaa
aaaa
u
uIAuAu
A
42
4)
2(
)(
0)(
0
22211
221112212211
2,1
22211
2211122122211
2211122122112
12212
22112211
2221
1211
aaaaaaaa
aaaaaaaa
aaaaaa
aaaaaa
aaaa
1121
2111
aaaa
A
221112,1 aa
2221
2111
aaaa
A
2)(4
422
22112
2122112,1
22211
22112
212211
2,1
aaaaa
aaaaaaa
The general solutions of 2x2 matrices
Distance matrix Dispersion matrix
1
31
21
13
2
2
122
1122
1212222
1111212
2211
2222121
1212111
2
1
1
2221
11211
............
............
......
...
...
0......
0...0...
0...
........................
...
mmmmm
m
mmmmm
nn
nn
mmmmm
mm
mm
mmmm
m
a
aa
u
u
uu
aa
aa
uauaua
uauauauauaua
uauaua
uauauauauaua
u
uu
aa
aaaaa
A
0
0][
2
1
2221
1211
2221
1211
i
i
i
i
uu
aaaa
aaaa
uIAuAu
A
0)(0)(
0
222121
212111
2
1
2221
1211
uauauaua
uu
aaaa
This system is indeterminate
Matrix reduction
111
1212
12121111212221
12111
11
001
1
aau
uaauaa
aau
0
0][
2
1
2221
1211
2221
1211
i
i
i
i
uu
aaaa
aaaa
uIAuAu
A
0......
............
0][
............
...
1
1
111
1
111
mmmm
m
mmm
m
u
u
aa
aa
aa
aa
uIAuAu
A
0...
0...
............
det
22
11
1
111
m
mmm
mmm
m
bbb
aa
aa
Characteristic polynomial
Eigenvalues and eigenvectors can only be computed analytically to the fourth power of m.
Higher order matrices need numerical solutions
Higher order matrices
222
1222
2222221
21
0)(1
aau
uaau
The power method to find the largest eigenvalue.
The power method is an interative process that starts from an initial guess of the eigenvector to approximate the eigenvalue
uAu
00
0 uAu Let the first component u11 of u1 being 1.
Rescale u1 to become 1 for the first component. This gives a second guess for .
11
1 uAu
Repeat this procedure until the difference n+1 – n is less than a predefined number e.
22
2 uAu
A4 0 1
-2 0 12 0 1
X0 X1 X2 X3 X4 X5 X6 X7 X81 5 4.6 4.565217 4.561905 4.561587 4.561556 4.561553 4.5615531 -1 -1.4 -1.43478 -1.4381 -1.43841 -1.43844 -1.43845 -1.438451 3 2.6 2.565217 2.561905 2.561587 2.561556 2.561553 2.561553
u0 u1 u2 u3 u4 u5 u6 u7 u81 1 1 1 1 1 1 1 11 -0.2 -0.30435 -0.31429 -0.31524 -0.31533 -0.31534 -0.31534 -0.315341 0.6 0.565217 0.561905 0.561587 0.561556 0.561553 0.561553 0.561553
0 1 2 3 4 5 6 7 81 5 4.6 4.565217 4.561905 4.561587 4.561556 4.561553 4.561553
Having the eigenvalues thew eigenvectors come immediately from solving
the linear system
0...
........................
...
2
1
1
22221
112111
mmmmm
m
u
uu
aa
aaaaa
using matrix reduction
Some properties of eigenvectors
11 L
UUAAUUUΛAUUΛΛU
If L is the diagonal matrix of eigenvalues:
The product of all eigenvalues equals the
determinant of a matrix.
n
i i1det A
The determinant is zero if at least one of the eigenvalues is
zero.In this case the matrix is
singular.
The eigenvectors of symmetric matrices are orthogonal
0':)(
UUA symmetric
Eigenvectors do not change after a matrix is multiplied by a scalar k.
Eigenvalues are also multiplied by k.
0][][ uIkkAuIA
If A is trianagular or diagonal the eigenvalues of A are the diagonal
entries of A.
A Eigenvalues 2 3 -1 3 2
3 2 -6 34 -5 4
5 5
Page Rank
In large webs (1-d)/N is very small
DC
CDC
B
BDB
A
ADAD
D
DCDC
B
BCB
A
ACAC
D
DBD
C
CBCB
A
ABAB
D
DAD
C
CAC
B
BABAA
pckdp
ckdp
ckdpp
ckdpp
ckdp
ckdpp
ckdp
ckdpp
ckdpp
ckdp
ckdp
ckdppp
0
0
0
0
D
C
B
A
D
C
B
A
C
CD
B
BD
A
AD
D
DC
B
BC
A
AC
D
DB
C
CB
A
AB
D
DA
C
CA
B
BA
pppp
pppp
ckd
ckd
ckd
ckd
ckd
ckd
ckd
ckd
ckd
ckd
ckd
ckd
0
0
0
0
A standard eigenvector problem
uPu
The requested raking is simply contained in the largest eigenvector of P.
Nd
ckdp
ckdp
ckdpp
Nd
ckdp
ckdp
ckdpp
Nd
ckdp
ckdp
ckdpp
Nd
ckdp
ckdp
ckdpp
C
CDC
B
BDB
A
ADAD
D
DCD
B
BCB
A
ACAC
D
DBD
C
CBC
A
ABAB
D
DAD
C
CAC
B
BABA
1
1
1
1
85.0185.0185.0185.01
41
10002/15.0115.002/15.015.0115.0
0001
D
C
B
A
pppp
A B
C D
D
C
B
A
D
C
B
A
C
C
B
B
A
A
D
D
B
B
A
A
D
D
C
C
A
A
D
D
C
C
B
B
pppp
pppp
ckd
ckd
ckd
ckd
ckd
ckd
ckd
ckd
ckd
ckd
ckd
ckd
0
0
0
0
-15
-10
-5
0
5
10
15
-10 -8 -6 -4 -2 0 2 4 6 8 10
Y
X-10
-5
0
5
10
-10 -8 -6 -4 -2 0 2 4 6 8 10
Y
X
Principal axes
u1 u1
u2
u2
The principal axes span a new Cartesian system . Principal axes are orthogonal.
-15
-10
-5
0
5
10
15
-10-8
-6-4
-20
24
68
10
Y
X
u1
u2
The data points of the new system are close to the new x-axis. The variance
within the new system is much
smaller.
UMXV )( );();()2;1( YXYXuu
Principal axes define the longer and shorter radius of an oval around the scatter of data points.
The quotient of longer to short principal axes measure how close the data points are associated
(similar to the coefficient of correlation).
Major axis regression
-15
-10
-5
0
5
10
15
-10 -8 -6 -4 -2 0 2 4 6 8 10
Y
X
u1
u2
The largest major axis defines a regression line through the data points {xi,yi}.
The major axis is identical with the largest eigenvector of the associated
covariance matrix.The length of the axes are given by the
eigenvalues. The eigenvalues measure therefore the
association between a and y.
0)( UID Major axis regression minimizes the Euclidean
distances of the data points to the regression line.
2)(4 222222
2,1YXXYYX sssss
222
21 1
y
xy
ss
u
u
222
1222
222
12
22
212
1
aau
aa
uu
u
XmYb
ss
uum
MAR
y
xyMAR
2
21
22
2
2
YXY
XYX
ssss
ΣD
The first principal axis is given by the largest
eigenvalue
The relationship between ordinary least square (OLS) and major axis (MAR) regression
YX
XY
sssr 2
X
XYOLS s
sm 2Y
XYMAR s
sm
X
YOLS s
srm 2Y
YXMAR s
ssrm
2
2
Y
XOLSMAR s
smm
Going Excel
X Y
0.7198581 2.486935 0.084878 0.23295 0.2683467 0.351561 0.23295 0.791836 0.9514901 2.705939 0.8280125 2.097096 0.4483117 0.87799 Eigenvalues 0.9386594 2.705279 0.015021 0 0.301201 0.960431 0.861692 0
0.4835976 1.8985610.7188817 1.9567160.7992589 2.734179 Eigenvectors0.3955747 0.958425 0.957858 0.2872410.802986 2.721408 -0.28724 0.957858
0.5685275 1.256930.0697029 0.2961540.3099076 0.6880480.9044138 2.466684 Eigenvectors0.3853803 0.981417 0.979137 0.20320.2352667 0.945944 -0.2032 0.9791370.8249179 2.3787330.8661037 2.569350.066128 0.603999 Parameters
a 3.334687b -0.23791
Values to draw the MAR regression0.1 0.0955610.9 2.76331
Covariance matrix
y = 2.8818x + 0.0185
00.5
11.5
22.5
3
0 0.2 0.4 0.6 0.8 1
YX
y=3.3347-0.2379
X Y
1 =3*A4+2=A4+A4*(LOS
()-0.5)=B4+B4*(LOS
()-0.5)
1 5 0.620370962 4.053401137 71.08896 93.07421 2 8 1.405829923 5.955994942 93.07421 264.3759 3 11 3.932365359 14.33527103 4 14 3.369965031 10.78551619 Eigenvalues 5 17 6.754318881 16.24109695 33.55803 6 20 3.140987171 27.95607099 301.9068 7 23 4.848193641 25.997851798 26 11.07692858 36.238371939 29 10.48809173 30.37969645 Eigenvectors10 32 5.174301246 46.52139803 0.927438 0.37397711 35 5.632905042 34.70506056 -0.37398 0.92743812 38 13.32938498 44.3523179413 41 16.94257054 25.7327815114 44 15.78715334 25.52100704 Parameters15 47 21.75787273 52.94515263 a 2.47993316 50 12.77137686 48.07135318 b 8.84797417 53 13.25068353 38.6710865218 56 22.70011271 28.7831935 Values to draw the MAR regression19 59 25.34824478 40.06533423 0.5 10.0879420 62 26.61513272 60.76764637 27 75.8061821 65 24.21743343 57.47387651
Covariance matrix
y = 1.37x + 15.86
01020304050607080
0 10 20 30
Y
X
y=2.48+8.85
y=3+2
Errors in the x and y variables cause OLS regression to predict lower slopes. Major axis regression is closer to the correxct slope.
Ordinary least squares
regression (OLS)
Major axis regression
(MAR)
If both variables have error terms MAR should be preferred.
The MAR slope is always steeper than the OLS slope.
Latitude of capitals (decimal degrees)
Days below zero
Latitude of capitals (decimal degrees)
Years below zero
41.33 34 41.33 0.09444442.5 60 42.5 0.166667 80.38689 420.4235 80.38689 1.167843
48.12 92 48.12 0.255556 420.4235 3803.503 1.167843 0.02934837.73 1 37.73 0.00277839.55 18 39.55 0.05 Eigenvalues53.87 144 53.87 0.4 33.50203 0.01237950.9 50 50.9 0.138889 3850.388 80.40386
43.82 114 43.82 0.31666751.15 64 51.15 0.17777842.65 102 42.65 0.283333 Eigenvectors27.93 1 27.93 0.002778 0.993839 0.110831 0.014528 0.99989449.22 12 49.22 0.033333 -0.11083 0.993839 -0.99989 0.01452841.92 11 41.92 0.03055635.33 1 35.33 0.00277845.82 114 45.82 0.316667 Parameters Parameters37.1 1 37.1 0.002778 a1 8.96715 a2 0.01453
35.15 2 35.15 0.005556 b1 -350.55 b2 -0.4872250.1 119 50.1 0.330556
55.63 85 55.63 0.236111 Values to draw the MAR regression36.4 2 36.4 0.005556 1 -341.583 -0.47269
59.35 143 59.35 0.397222 80 366.8216 0.67517962 35 62 0.097222
60.32 169 60.32 0.469444 MAR48.73 50 48.73 0.138889 a1/a2 617.146552.38 97 52.38 0.269444 OLS36.1 0 36.1 0 a1/a2 359.459537.9 2 37.9 0.005556
Covariance matrix Covariance matrix
y = 5.32x - 179.40
100
200
300
400
0 20 40 60 80
D
L
y=8.97-350.6
y = 0.0148x - 0.498
0
0.2
0.4
0.6
0.8
1
0 20 40 60 80
D
L
y=0.0145-0.487
MAR is not stable after rescaling of only one of the variables
OLS regression retains the correct scaling factor, MAR
does not.
MAR should not be used for comparing slopes if variables have different dimensions and were measured in different units, because
the slope depends on the way of measurement.If both variables are rescaled in the same manner (by the same
factor) this problem doesn’t appear.
Days/360
1
n 1 1 1
n 1 1 n 1 n n 1 n 1i i
A U U A U U
A (U U )(U U )(U U )...
A U ( U U...) U U U U I U U U
L L
L L L
L L
n n n 1
i
n n 1i
A U I U
A U U
Scaling a matrix
A U U L
A simple way to take the power of a square matrix
The variance and covariance of data according to the principal axis
)()(11
............
...
1
111
);( MXMXΣ
T
mmm
m
yx n
UMXV )( );();()2;1( yxyxuu
UΣUUMXMXUVVΣ );()2;1()2;1()2;1( 11)()(
11
11
yxTTT
uuuuT
uu nnn
kkTkkkk
Tkkyx
Tkk uuuuuu );(
2
0);( kTjkkk
Tjkyx
Tjjk uuuuuu
The vector of data points in the new system comes from the
transformation according to the principal axes
(x;y)y
x
u2
u1
The variance of variable k in the new system is equal to the eigenvalue of the kth
principal axis. The covariance of variables j and k in the new system is zero. The new variables are
independent.
Eigenvectors are normalized to the length of one
The eigenvectors are orthogonal