Statistical Consistency of Kernel Canonical Correlation Analysis
Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.
-
Upload
jaden-dwyer -
Category
Documents
-
view
221 -
download
2
Transcript of Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.
![Page 1: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/1.jpg)
Agenda of Week VII
Review of Week VI
Multiple regression
Canonical correlation
![Page 2: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/2.jpg)
Review of Week VI
Inference on MV Mean Vector Two populations
Inference on MV Mean Vector Multi-populations: MANOVA
![Page 3: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/3.jpg)
Multiple Regression
Basic assumptions Linearity
X is an n by K matrix with rank K
Regression
Homoscedasticity
y X
[ | ] 0E X
2[ ] [ ' | ]Var E X I
![Page 4: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/4.jpg)
Multiple Regression
Hypotheses
0 : 0 . : 0aH vs H
0 : 0 . : 0i a iH vs H
By ANOVA
When the null hypothesis is rejected,
For all i.
![Page 5: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/5.jpg)
Multiple Regression
' ( ) '( )
' ' ' ' ' '
' 2 ' ' '
Min y X y X
y y X y y X X X
y y X y X X
1
'2 ' 2 ' 0
( ' ) '
X y X X
X X X y
OLS model
OLS solution
![Page 6: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/6.jpg)
Multiple RegressionANOVA table
Source Sum of squares d.f. Mean square F
Regression 2' 'b X y ny k- 1 2' '
1
b X y ny
k
Residual 'e e n- k 'e e
n k
Total 2'y y ny n- 1 2'
1
y y ny
n
2' '1
'
b X y nyk
e en k
Coefficient of
determination
22
'1
'
e eR
y y ny
![Page 7: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/7.jpg)
Multiple Regression
2 2 1'. [ ] ( ' )
e es Est V s X X
n k
2
kk kkt
s S
t-statistics (d.f. = n-k)
Where Skk is the kth diagonal element of (X’X)-1
Standard error of the regression
![Page 8: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/8.jpg)
Multiple Regression
2
2
kkt s S
Standard error of coefficient
![Page 9: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/9.jpg)
Canonical Correlation
Correlation structure between a group of independent variables and a group of dependent variables A kind of multiple regression with more than two
dependent variables Example: Physical size group vs. Exercise group How much explained the variation of a group of
variables by other group of variables?
![Page 10: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/10.jpg)
Canonical Correlation
Basic model
1
.
.i
p
X
X X
X
Group of independent variables
1
.
.i
q
Y
Y Y
Y
Group of dependent variables
( ) ( ), ( ) ( )
( , ) ( )XX XX YY YY
XY XY
Cov X S Cov Y S
Cov X Y S
Covariance matrixes
![Page 11: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/11.jpg)
Canonical Correlation
Basic model
1 1
1 1
' ......
' ......
i i ip p
i i iq q
V X X X
W Y Y Y
LC of two groups of variables
' '( , ) ( ( , ) )
' ' ' 'XY XY
XX YY XX YY
SV W r V W
S S
Correlation between V and W
![Page 12: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/12.jpg)
Canonical Correlation
Correlation testing
0 : 0 : 0xy a xyH vs H Hypotheses
22
1
3
1, 12 3
2 2
2 2
1
1
( 1)2
4( )1.5 ,2 5
p qp q ms
Wilks r
pqF
pqms
p qp qm n sp q
Correlation
between V and W
![Page 13: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/13.jpg)
Canonical Correlation
,( , ) ( , )
. .
( ) ( ) 0,
( ) ( ) 1
Max V W or r V W
s t
E V E W
V V V W
Objective: To find α and β maximizing ρ or r.
![Page 14: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/14.jpg)
Canonical Correlation
1 1' '2 2
1 1 1 1,XX YYV e S X W f S Y
1 1' '2 21 1' , 'XX YYe S f S
1st canonical variable
kth canonical variable
1 1' '2 2,k k XX k k YYV e S X W f S Y
![Page 15: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/15.jpg)
Canonical Correlation
1 1 1( , )r r V W
1st canonical correlation coefficient
kth canonical coefficient coefficient
( , )k k kr r V W
![Page 16: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/16.jpg)
Canonical Correlation
1 112 2
1 112 2
,
k
XX XY YY YX XX
k
YY YX XX XY YY
e eigen vector of kth eigen value of
S S S S S
f eigen vector of kth eigen value of
S S S S S
![Page 17: Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.](https://reader036.fdocuments.net/reader036/viewer/2022070305/5514296c550346ec488b5bfd/html5/thumbnails/17.jpg)
Canonical Correlation
( , ) , ( , )XX X YY Yr V X AS D r W Y BS D
Canonical loading matrix
Canonical cross loading matrix
( , ) , ( , )XY Y YX Xr V Y AS D r W X BS D