Properties of the Horst Algorithm for the Multivariable Eigenvalue Problem
-
Upload
hakeem-campbell -
Category
Documents
-
view
32 -
download
2
description
Transcript of Properties of the Horst Algorithm for the Multivariable Eigenvalue Problem
Properties of the Horst Properties of the Horst Algorithm for the Algorithm for the
Multivariable Eigenvalue Multivariable Eigenvalue ProblemProblem
Michael SkalakNorthwestern University
Outline of ProblemOutline of Problem
nnm
i i 1
mmmm
m
m
AAA
AAA
AAA
A
21
22221
11211
with ii nnii RA
Given such that and a symmetric, positive definite matrix
mnn ,...,1
Multivariable Eigenvalue Problem• Find real scalars and a real column vector
such that
where is the identity matrix of size
and is partitioned into blocks
with
m ,...,1nRx
mix
xAx
i ,...,1,1
},...,{ ][][1
1 mnm
n IIdiag ][ inI in
ini Rx
TTm
T xxx ],...,[ 1
nRx
ExampleExampleGiven the symmetric and positive definite matrix , ,
the vector
is a solution, as
442.12638.4581.0
638.4367.10297.0
581.0297.0990.8
473.1645.3
162.2349.0
052.295.1
473.1162.2052.2
645.3349.0950.1
919.9821.3
821.3740.6
A
21 n 32 n2m
717.0684.0131.0994.0177.0 x
21 841.17405.12 xxAx
Statistical ApplicationStatistical ApplicationFind the maximum correlation coefficient of random variables, each of
size
Maximize
subject to
Hence the solution is the global maximum of for vectors in
, where is a ball of radius 1 centered at the origin in dimensions.
AxxT
mmini ,...,1
mnn BB ...1
AxxT
nBn
mixi ,...,11
Power MethodPower Method
The power method finds the eigenvector with the largest eigenvalue for the usual single-variate eigenvalue problem.
end
yx
y
Axy
kfor
k
kk
kk
kk
)(
)()1(
)()(
)()(
,...2,1
Horst AlgorithmHorst Algorithm
end
end
yx
y
xAy
mifor
kfor
ki
kik
i
ki
ki
m
j
kjij
ki
)(
)()1(
)()(
1
)()(
:
:
:
,...,1
,...2,1
Proven to converge monotonically by Chu and Watterson [SIAM J. Sci. Comput. (14), No. 5, pp. 1089-1106]
Finds the which maximizes x AxxT
ExampleExample
For that same matrix, consider the Horst algorithm with the
starting point
First iteration:
577.0577.0577.0707.0707.0)0( x
968.0252.0
511.3
398.3884.
)1(1
)1(1)2(
1
)1(1
)1(1
2
1
)0(11
)1(1
yx
y
xAyj
j
511.0548.0662.0
807.12
578.6059.7527.8
)1(2
)1(2)2(
2
)1(2
)1(2
2
1
)0(22
)1(2
yx
y
xAyj
j
Dependence on Initial ConditionsDependence on Initial Conditions
Convergence point can depend on initial conditions:
Like many other maximization algorithms, the Horst algorithm can converge to a local instead of global max.
885.0464.0023.0436.0900.0577.0577.0577.0707.0707.0)0( x
717.0684.0131.0994.0108.0267.0535.0802.0894.0447.0)0( x
414.31AxxT
284.30AxxT
ResultsResults
For any , can have at least as many convergent points
For any m, there can be at least convergence points, and as few as one.
In at least a nontrivial special case (two convergence points, ) the portion of the region which converges to the global max can not be arbitrarily small
in 1in
3
1
3m
3,2 nm
Number of Convergence PointsNumber of Convergence PointsThere exist 3 matrices, for all such that there exist
convergence points.
The block matrix, with meaning a matrix of size with convergence points
is symmetric, positive definite, and has convergence points
With a little manipulation, this proves that for any size there exist
matrices with at least convergence points.
m
13,2,1 inm i m
2
1
0
0
m
m
A
A
mA m
21mm
3
1
3m
m
Convergence to Global MaxConvergence to Global MaxSuppose there is some transformation on the matrix that can arbitrarily move eigenvectors arbitrarily. After the transformation, the matrix is rescaled so that the largest element remains constant.
fe
ed
c
bcba
321 dxfecdxedbdxcbadx
Axxd T
Case 1Case 1
The difference of the values between the local mins and the global max is bounded . Then the derivative must increase without bound. However, since all elements of the matrix are less than a constant, this cannot happen.
0
Case 2Case 2
The values of the local mins approach the global max as the vectors approach. Since one of the local mins is the global min, the function become closer and closer to constant, which cannot happen since the derivative is bounded below in at least one direction.