G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed...
-
Upload
beatrix-west -
Category
Documents
-
view
215 -
download
1
Transcript of G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed...
![Page 1: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/1.jpg)
Global Optimality of the Successive MaxBet Algorithm
USC ENITIAA de NANTES
FranceMohamed HANAFI
and
Jos M.F. TEN BERGE
Department of psychology University of Groningen
The Netherlands
![Page 2: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/2.jpg)
Global Optimality of the Successive MaxBet Algorithm
Summary. 1. The Successive MaxBet Problem (SMP). 2. The MaxBet Algorithm. 3. Global Optimality : Motivation/Problems. 4. Conclusions and Open questions.
![Page 3: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/3.jpg)
1. The Successive MaxBet Problem (S.M.P)
jk pp ,kjA
kjAA Kjk ,,2,1,
KK , Blocks Matrix
s.p.s.d
KKKK
K
K
AAA
AAA
AAA
A
21
22221
11211
![Page 4: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/4.jpg)
Auuu '
order 1
K
jkjkjk
1,
' uAu
Maximize
Subject to
11
''
K
kkkuuuu
1. The Successive MaxBet Problem (S.M.P)
Kk ,,2,1 1' kkuu
![Page 5: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/5.jpg)
order s
Kk ,,2,1
Auuuuu '21 ,...,, K
Maximize
Subject to1' kkuu
0uU kk'{
121 skkkk uuuU Kk ,,2,1
1. The Successive MaxBet Problem (S.M.P)
![Page 6: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/6.jpg)
2. The Successive MaxBet AlgorithmTen Berge (1986,1988)
Order 1
K
jjkjk
1
uAv
1. Take arbitrary initial unit length vectors ku
2. Compute :
3. rescale vk to unit length, and set uk= vk
4. Repeat steps 2 and 3 till convergence
Kk ,,2,1
Kk ,,2,1
Kk ,,2,1
![Page 7: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/7.jpg)
Order s
2. The Successive MaxBet AlgorithmTen Berge (1986,1988)
''jjpkjkkpkj jk
UUIAUUIA
K
jjkjk
1
uAv
1. Take arbitrary initial unit length vectors ku
2. Compute :
3. rescale vk to unit length, and set uk= vk
4. Repeat steps 2 and 3 till convergence
Kk ,,2,1
Kk ,,2,1
Kk ,,2,1
![Page 8: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/8.jpg)
Property 1 : Convergence of the MaxBet Algorithm
u
u
![Page 9: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/9.jpg)
Property 2 : Necessary Condition of Convergence
1' kkuu
Kk ,,2,1
KKKmmmm
m
m
u
u
u
u
u
u
AAA
AAA
AAA
22
11
2
1
21
22221
11211
K ....21u
![Page 10: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/10.jpg)
3. Motivation and results
1. MaxBet Algorithm depends on the starting vector
2. MaxBet algorithm does not guarantee the computation of the global solution of SMP
![Page 11: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/11.jpg)
43 23 -13 0 -7
23 31 10 1 0
-13 10 64 -19 -2
0 1 -19 24 18
-7 0 -2 18 58
A
11A 12A
22A21A
3. Motivation and results : an example
2K 21 p 32 p
![Page 12: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/12.jpg)
42185 64023
(u)= 10621Function value{ 3846.7 5978.4
(v)= 9825.1
0.67 0.36 0.20 0.53 0.30
{Starting Vector *u
0.64 0.31 0.64 0.24 0.10
*v
0.69 0.72 0.58 -0.43 -0.68
v{Solution Vector
0.94 0.31 -0.92 0.35 0.11
u
![Page 13: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/13.jpg)
3. Motivation and results: Two Questions
Q1. How can we know that the solution computed by the Maxbet algorithm is global or not ?
Q2. When the solution is not global, how can we reach using this solution the global solution ?
![Page 14: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/14.jpg)
K
K
pKKKKK
Kp
Kp
IAAA
AIAA
AAIA
A
21
222221
112111
,...,,2
1
21
3. Motivation and Results : Proceeding
Global solution of SMP
Spectral properties (eigenvalues and eigenvectors) of K ,...,, 21
A
![Page 15: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/15.jpg)
When, for a solution, {u, 1,
2, …,
K} satisfies
KKKKKKK
K
K
u
u
u
u
u
u
AAA
AAA
AAA
22
11
2
1
21
22221
11211
RESULT 1
then u is the global solution of SMP.
is negative semidefinite, K ,...,, 21A
Result 1
![Page 16: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/16.jpg)
we have :vAv= vAv 1v1
v1
2v2v2
… KvK
vK
= (v) 1
2 …
K,
hence(v)1+2+…+K= (u)
the matrix A is negative semidefinite,
ELEMENTS OF PROOF (Result 1)
![Page 17: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/17.jpg)
To what extent the previous sufficient condition (Result 1):
is necessary ?
K ,...,, 21A(matrix is negative semi definite)
3. Motivation and Results
![Page 18: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/18.jpg)
matrix blocks 2,2 is A
RESULT 2
22
11
2
1
2221
1211
u
u
u
u
AA
AA
When u is the global maximum of S.M.P it verifies :
2
1
2122221
12111
,p
p
IAA
AIAA
21 ,Athen matrix is negative semi definite
Result 2
![Page 19: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/19.jpg)
Suppose has a positive eigenvalue 21 ,A
021 , wwA
1. w is block-normed vector
2. w is not block-normed vector
2.1. w is not block orthogonal to u
2.2. w is block orthogonal to u
ELEMENTS OF PROOF (Result 2)
![Page 20: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/20.jpg)
1. w is block-normed vector
021 ,
' wAw
uAww 21'
w is better solution than u
![Page 21: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/21.jpg)
2. w is not block-normed vector2.1. w is not block orthogonal to u
222
22211
222112
)'(8)''(
)''(
wuwwww
wwww
222
22211
2222
)'(8)''(
)'(16
wuwwww
wu
wuv
v is better solution than u
![Page 22: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/22.jpg)
*wuv
v is better solution than u
211
122
)'(
)'(
dud
dudq
0'
0'
22
11
ud
ud
qww t*
w is not block-normed vector2.2. w is block orthogonal to u
![Page 23: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/23.jpg)
RESULT 2
Result 3
positive are elements allwith
matrix blocks , is KKA
K
kkpp
1
ghaA phg ....,2,1,
then matrix is negative semi definite K ,...,, 21A
When u is the global maximum of S.M.P
![Page 24: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/24.jpg)
positive are elements allwith
matrix blocks , is KKA
Suppose has a positive eigenvalue
0
K ,...,, 21A
wwA K,...,, 21
ELEMENTS OF PROOF (Result 3)
![Page 25: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/25.jpg)
p
hgghhg
p
hgghhg auuauuu
,,
||
u has all elements of the same sign
ELEMENTS OF PROOF (Result 3)
k
K
kkkK
1
'',...,,
'
21wwAwwwAw
w has all elements of the same sign
![Page 26: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/26.jpg)
2K
sign same theof elements allnot with
matrix blocks , is KKA
The sufficient condition (Result 1) :
is not necessary
K ,...,, 21A(matrix is negative semi definite)
Result 4
![Page 27: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/27.jpg)
45 -20 5 6 16 3
-20 77 -20 -25 -8 -21
5 -20 74 47 18 -32
6 -25 47 54 7 -11
16 -8 18 7 21 -7
3 -21 -32 -11 -7 70
ELEMENTS OF PROOF (Result 4)
A
3K
21 p
22 p
23 p
![Page 28: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/28.jpg)
0.49-0.87 0.80 0.59 0.56-0.82
(u) =378.96
Random research with 10.000.000 starting vectors
=0.48 u =
ELEMENTS OF PROOF (Result 4)
![Page 29: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/29.jpg)
- Possible Application in statistics :
Multivariate Methods (Analysis of K sets of data )
4. General Conclusions
1. Generalized canonical correlationAnalysis: Horst (1961)
3. Soft Modeling Approach :Estimation of latent variables under mode B
Wold (1984); Hanafi (2001)
2. Rotation methods : MaxDiff, MaxBet, generalized Procrustes Analysis
Gower(1975); Van de Geer(1984);Ten Berge (1986,1988)
![Page 30: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/30.jpg)
-
- Necessary condition for the case K=3 when matrix A has not all
elements of the same sign?
4. Perspective and Little Open Question
2X
0
0
2'
1'
aXa
aXa
nn,
??0a
nn,1X
![Page 31: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/31.jpg)
![Page 32: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/32.jpg)
![Page 33: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/33.jpg)
uf
Ku
u
u
u2
11k
'kuu
K,...,,k 21
Motivation: Illustration 1 MaxBet Algorithm depends on the starting vector
![Page 34: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/34.jpg)
The Successive MaxBet Problem (S.M.P)and
Multivariate Methods
kpn, K,,,k 21kX
K
kkpn
1
, KXXXX 21
![Page 35: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/35.jpg)
Some multivarite methods
Generalized canonical correlation methods
Rotation methods(Agreement methods)
SOFT MODELING APPRAOCH(Approch)
![Page 36: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/36.jpg)
n
XXA
'
kjAA
Rotation methods
njk
kj
XXA
'
S M P = MaxBet method Van de Geer (1984) Ten Berge (1986,1988)
Kjk ,,2,1,
S M P = MaxBet method Van de Geer (1984) Ten Berge (1986,1988)
Kjk ,,2,1,
S M P = MaxDiff method Van de Geer (1984) Ten Berge (1986,1988)
Kjk ,,2,1,
jk
jkn
jk
kj
0
' XXA
S M P = MaxBet method Van de Geer (1984) Ten Berge (1986,1988)
Kjk ,,2,1, Kjk ,,2,1,
jkn
K
jkn
jk
jk
kj XX
XX
A '
'
2
S M P = Generalized Procrustes Analysis Gower(1975), Ten Berge (1986,1988)
![Page 37: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/37.jpg)
kX 'kkkk WPX
Generalized canonical correlation methods
SVD
kjAA n
jkkj
PPA
'
SMP = Horst method(1961)
kjAA njk
kjkj
PPA
'
S M P = Soft Modeling Appraoch (Hanafi 2001)
1,1,0 jkkj
Mode B soft modeling approach
![Page 38: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/38.jpg)
uuAuuuIAu ''' mm mK Auu '
uIAu m' MaximizeK,...,,k 211 Subject to k
'kuu
1k'kuu K,...,,k 21
Auu ' MaximizeK,...,,k 211 Subject to k
'kuu
![Page 39: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/39.jpg)
Auuuuu '21 ,...,, K
Kk ,,2,1 1' kkuu
Maximize
Subject to
KKKmmmm
m
m
u
u
u
u
u
u
AAA
AAA
AAA
22
11
2
1
21
22221
11211
0u
![Page 40: G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed HANAFI and Jos M.F. TEN BERGE Department of psychology.](https://reader036.fdocuments.net/reader036/viewer/2022070402/56649f225503460f94c3b3d6/html5/thumbnails/40.jpg)
Multivariate Eigenvalue Problem Watterson and Chu(1993)
kppp ....2 solutions ofnumber 21
1' kkuuKk ,,2,1
KKKmmmm
m
m
u
u
u
u
u
u
AAA
AAA
AAA
22
11
2
1
21
22221
11211