Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances...
Transcript of Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances...
![Page 1: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/1.jpg)
Recent advances on Regularized Generalized Canonical
Correlation Analysis
Arthur Tenenhaus 2013/05/16
![Page 2: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/2.jpg)
2
Glioma Cancer Data(Department of Pediatric Oncology of the Gustave Roussy Institute)
Transcriptomic data (X1)
CGH data (X2)
outcome (X3)
![Page 3: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/3.jpg)
3
Glioma Cancer Data: from a multi-block viewpoint(Department of Pediatric Oncology of the Gustave Roussy Institute)
ξ2
ξ1
ξ3
C13 = 1
C23 = 1
C12 = 1
![Page 4: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/4.jpg)
3
Glioma Cancer Data: from a multi-block viewpoint(Department of Pediatric Oncology of the Gustave Roussy Institute)
ξ2
ξ1
ξ3
C13 = 1
C23 = 1
![Page 5: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/5.jpg)
3
Glioma Cancer Data: from a multi-block viewpoint(Department of Pediatric Oncology of the Gustave Roussy Institute)
ξ2
ξ1
ξ3
C13 = 1
C23 = 1
C12 = 0
![Page 6: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/6.jpg)
4
Block components
![Page 7: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/7.jpg)
4
Block components
Block components should verified two properties at the same time:
(i) Block components well explain their own block.
(ii) Block components are as correlated as possible for connected blocks.
![Page 8: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/8.jpg)
Covariance-based criteriacjk = 1 if blocks are linked, 0 otherwise and cjj = 0
SUMCOR (Horst, 1961)
SSQCOR (Mathes, 1993 ; Hanafi, 2004)
SABSCOR (Mathes, 1993 ; Hanafi, 2004)
SUMCOV (Van de Geer, 1984)
SSQCOV (Hanafi & Kiers, 2006)
SABSCOV (Krämer, 2006)
Some modified multi-block methods
Some multi-block methods
![Page 9: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/9.jpg)
Covariance-based criteriacjk = 1 if blocks are linked, 0 otherwise and cjj = 0
SUMCOR (Horst, 1961)
SSQCOR (Mathes, 1993 ; Hanafi, 2004)
SABSCOR (Mathes, 1993 ; Hanafi, 2004)
SUMCOV (Van de Geer, 1984)
SSQCOV (Hanafi & Kiers, 2006)
SABSCOV (Krämer, 2006)
Some modified multi-block methods
Some multi-block methods
![Page 10: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/10.jpg)
Covariance-based criteriacjk = 1 if blocks are linked, 0 otherwise and cjj = 0
SUMCOR (Horst, 1961)
SSQCOR (Mathes, 1993 ; Hanafi, 2004)
SABSCOR (Mathes, 1993 ; Hanafi, 2004)
SUMCOV (Van de Geer, 1984)
SSQCOV (Hanafi & Kiers, 2006)
SABSCOV (Krämer, 2006)
Some modified multi-block methods
![Page 11: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/11.jpg)
Covariance-based criteriacjk = 1 if blocks are linked, 0 otherwise and cjj = 0
SUMCOR (Horst, 1961)
SSQCOR (Mathes, 1993 ; Hanafi, 2004)
SABSCOR (Mathes, 1993 ; Hanafi, 2004)
SUMCOV (Van de Geer, 1984)
SSQCOV (Hanafi & Kiers, 2006)
SABSCOV (Krämer, 2006)
Some modified multi-block methods
![Page 12: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/12.jpg)
Covariance-based criteriacjk = 1 if blocks are linked, 0 otherwise and cjj = 0
SUMCOR (Horst, 1961)
SSQCOR (Mathes, 1993 ; Hanafi, 2004)
SABSCOR (Mathes, 1993 ; Hanafi, 2004)
SUMCOV (Van de Geer, 1984)
SSQCOV (Hanafi & Kiers, 2006)
SABSCOV (Krämer, 2006)
Some modified multi-block methods
GENERALIZED CANONICAL CORRELATION ANALYSIS
![Page 13: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/13.jpg)
Covariance-based criteriacjk = 1 if blocks are linked, 0 otherwise and cjj = 0
SUMCOR (Horst, 1961)
SSQCOR (Mathes, 1993 ; Hanafi, 2004)
SABSCOR (Mathes, 1993 ; Hanafi, 2004)
SUMCOV (Van de Geer, 1984)
SSQCOV (Hanafi & Kiers, 2006)
SABSCOV (Krämer, 2006)
Some modified multi-block methods
GENERALIZED CANONICAL CORRELATION ANALYSIS
![Page 14: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/14.jpg)
Covariance-based criteriacjk = 1 if blocks are linked, 0 otherwise and cjj = 0
SUMCOR (Horst, 1961)
SSQCOR (Mathes, 1993 ; Hanafi, 2004)
SABSCOR (Mathes, 1993 ; Hanafi, 2004)
SUMCOV (Van de Geer, 1984)
SSQCOV (Hanafi & Kiers, 2006)
SABSCOV (Krämer, 2006)
Some modified multi-block methods
GENERALIZED CANONICAL CORRELATION ANALYSIS
GENERALIZED CANONICAL COVARIANCE ANALYSIS
![Page 15: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/15.jpg)
Covariance-based criteriacjk = 1 if blocks are linked, 0 otherwise and cjj = 0
SUMCOR:
SSQCOR:
SABSCOR:
SUMCOV:
SSQCOV:
SABSCOV:
![Page 16: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/16.jpg)
RGCCA optimization problem
Subject to the constraints
and:
where:
argmax&1,&2,…,&+ ,
,-./0 g 1cov56/&/ , 60&078+
/≠0
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,… , 2
![Page 17: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/17.jpg)
RGCCA optimization problem
Subject to the constraints
and:
where:
A monotone convergent algorithmrelated to this optimization problem
will be described.
argmax&1,&2,…,&+ ,
,-./0 g 1cov56/&/ , 60&078+
/≠0
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,… , 2
![Page 18: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/18.jpg)
RGCCA optimization problem
Subject to the constraints
and:
where:
A monotone convergent algorithmrelated to this optimization problem
will be described.
Schäfer and Strimmer formula can be used for an optimal determination of the shrinkage constants
argmax&1,&2,…,&+ ,
,-./0 g 1cov56/&/ , 60&078+
/≠0
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,… , 2
![Page 19: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/19.jpg)
Special cases
Choice of the shrinkage constant τj (part 1)
argmax&1,&2*
*cov(/1&1, /2&2)
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,2 Subject to the constraints
![Page 20: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/20.jpg)
Special cases
Components X1a1 and X2a2 are well correlated.
Choice of the shrinkage constant τj (part 1)
argmax&1,&2*
*cov(/1&1, /2&2)
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,2 Subject to the constraints
![Page 21: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/21.jpg)
Special cases
Components X1a1 and X2a2 are well correlated. 1st component is stable
Choice of the shrinkage constant τj (part 1)
argmax&1,&2*
*cov(/1&1, /2&2)
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,2 Subject to the constraints
![Page 22: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/22.jpg)
Special cases
Components X1a1 and X2a2 are well correlated.
No stability condition for 2nd component1st component is stable
Choice of the shrinkage constant τj (part 1)
argmax&1,&2*
*cov(/1&1, /2&2)
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,2 Subject to the constraints
![Page 23: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/23.jpg)
9
0 1
Favoringcorrelation
Favoringstability
τj
Schäfer and Strimmer formula can be used for an optimal determination of the shrinkage constants
Subject to the constraints
argmax&1,&2,…,&+ ,
,-./0 g 1cov56/&/ , 60&078+
/≠0
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,… , 2
Choice of the shrinkage constant τj (part 2)
![Page 24: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/24.jpg)
Choice of the design matrix C
Hierarchical models
![Page 25: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/25.jpg)
Choice of the design matrix C
Hierarchical models
![Page 26: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/26.jpg)
Choice of the design for NeuroImaging-Genetic datasets
X1 DNA arrays (SNP)
p1 ~106
X2 Functional MRI
p2 ~104
X3Developmental disorders - Reading difficulties - Basic numerical knowledge -…- Visuo-spatial abilities - Visuo-motor abilities
p3 ~10
n ~100c12=1 c23=1
c13=0
![Page 27: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/27.jpg)
12
PLS Regression Wold S., Martens & Wold H. (1983): The multivariate calibration problem in chemistry solved by the PLS method. In Proc. Conf. Matrix Pencils, Ruhe A. & Kåstrøm B. (Eds), March 1982, Lecture Notes in Mathematics, Springer Verlag, Heidelberg, p. 286-293.
Redundancy analysis Barker M. & Rayens W. (2003): Partial least squares for discrimination, Journal of Chemometrics, 17, 166-173.
Regularized CCA Vinod H. D. (1976): Canonical ridge and econometrics of joint production. Journal of Econometrics, 4, 147–166.
Inter-battery factor analysis Tucker L.R. (1958): An inter-battery method of factor analysis, Psychometrika, vol. 23, n°2, pp. 111-136.
MCOA Chessel D. and Hanafi M. (1996): Analyse de la co-inertie de K nuages de points. Revue de Statistique Appliquée, 44, 35-60
SSQCOV Hanafi M. & Kiers H.A.L. (2006): Analysis of K sets of data, with differential emphasis on agreement between and within sets, Computational Statistics & Data Analysis, 51, 1491-1508.
SUMCOR Horst P. (1961): Relations among m sets of variables, Psychometrika, vol. 26, pp. 126-149.
SSQCOR Kettenring J.R. (1971): Canonical analysis of several sets of variables, Biometrika, 58, 433-451
MAXDIFF Van de Geer J. P. (1984): Linear relations among k sets of variables. Psychometrika, 49, 70-94.
PLS path modeling Tenenhaus M., Esposito Vinzi V., Chatelin Y.-M., Lauro C. (2005): PLS path modeling. Computational Statistics and Data (mode B) Analysis, 48, 159-205.
Generalized Orthogonal Vivien M. & Sabatier R. (2003): Generalized orthogonal multiple co-inertia analysis (-PLS): new multiblock component MCOA and regression methods, Journal of Chemometrics, 17, 287-301.
Caroll’s GCCA Carroll, J.D. (1968): A generalization of canonical correlation analysis to three or more sets of variables, Proc. 76th Conv. Am. Psych. Assoc., pp. 227-228.
special cases of RGCCA (among others)two-block case
multi-block case
![Page 28: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/28.jpg)
13
Monotone convergent algorithm for the RGCCA criteria
argmax&1,&2,…,&+ ,
,-./0 g 1cov56/&/ , 60&078+
/≠0
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,… , 2 Subject to the constraints
![Page 29: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/29.jpg)
13
Monotone convergent algorithm for the RGCCA criteria
• Construct the Lagrangian function related to the optimization problem.
argmax&1,&2,…,&+ ,
,-./0 g 1cov56/&/ , 60&078+
/≠0
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,… , 2 Subject to the constraints
![Page 30: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/30.jpg)
13
Monotone convergent algorithm for the RGCCA criteria
• Construct the Lagrangian function related to the optimization problem.
• Cancel the derivative of the Lagrangian function with respect to each aj.
argmax&1,&2,…,&+ ,
,-./0 g 1cov56/&/ , 60&078+
/≠0
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,… , 2 Subject to the constraints
![Page 31: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/31.jpg)
13
Monotone convergent algorithm for the RGCCA criteria
• Construct the Lagrangian function related to the optimization problem.
• Cancel the derivative of the Lagrangian function with respect to each aj.
• Use the Wold’s procedure to solve the stationary equations (≈ Gauss-Seidel algorithm).
argmax&1,&2,…,&+ ,
,-./0 g 1cov56/&/ , 60&078+
/≠0
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,… , 2 Subject to the constraints
![Page 32: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/32.jpg)
13
Monotone convergent algorithm for the RGCCA criteria
• Construct the Lagrangian function related to the optimization problem.
• Cancel the derivative of the Lagrangian function with respect to each aj.
• Use the Wold’s procedure to solve the stationary equations (≈ Gauss-Seidel algorithm).
• This procedure is monotonically convergent: the criterion increases at each step of the algorithm.
argmax&1,&2,…,&+ ,
,-./0 g 1cov56/&/ , 60&078+
/≠0
!1 − $% &var!*%+% & + $% -+% -2 = 1, % = 1,… , 2 Subject to the constraints
![Page 33: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/33.jpg)
The RGCCA algorithm (primal version)
![Page 34: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/34.jpg)
The RGCCA algorithm (primal version)
Initialstep
![Page 35: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/35.jpg)
The RGCCA algorithm (primal version)
Outer Estimation(explains the block)
Initialstep
![Page 36: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/36.jpg)
The RGCCA algorithm (primal version)
Outer Estimation(explains the block)
Initialstep
cjk = 1 if blocks are linked, 0 otherwise and cjj = 0
InnerEstimation(explains relation between block)
Choice of weights ejh:- Horst :
- Centroid :
- Factorial :
![Page 37: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/37.jpg)
The RGCCA algorithm (primal version)
Outer Estimation(explains the block)
Initialstep
cjk = 1 if blocks are linked, 0 otherwise and cjj = 0
InnerEstimation(explains relation between block)
Choice of weights ejh:- Horst :
- Centroid :
- Factorial :
pj × pjDimension =
![Page 38: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/38.jpg)
The RGCCA algorithm (primal version)
Outer Estimation(explains the block)
Initialstep
Iterate until convergence
of the criterion
cjk = 1 if blocks are linked, 0 otherwise and cjj = 0
InnerEstimation(explains relation between block)
Choice of weights ejh:- Horst :
- Centroid :
- Factorial :
pj × pjDimension =
![Page 39: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/39.jpg)
The RGCCA algorithm (dual version)
Initial step jα
( )
( ) jjtjjj
tjj
tj
jjtjjj
j
zIXXn
XXz
zIXXnα
1
1
11
11
−
−
"#$
%&' +−
"#$
%&' +−
=
ττ
ττ
Iterate until convergence
of the criterion
cjk = 1 if blocks are linked, 0 otherwise and cjj = 0
Inner Estimation (explains relation between
block)
kjkjkj e yz ∑
≠
=
Choice of weights ejh: - Horst :
- Centroid :
- Factorial :
jkjk ce =
( )( )kjjkjk ce yy ,corsign=
( )kjjkjk ce yy ,cov=
jtjjj αXXy =
Outer Estimation (explains the block)
( )[ ] 1)1( 1 =−+ j
tjjnjj
tjj
tj αXXIXXα ττ
Dimension = !×!"
jtjj αXa =
![Page 40: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/40.jpg)
Glioma Cancer Data: from an RGCCA viewpoint(Department of Pediatric Oncology of the Gustave Roussy Institute)
ξ2
ξ1
ξ3
RGCCA with factorial scheme - τ1 = 1, τ2 = 1 and τ3 = 0
C13 = 1
C23 = 1
![Page 41: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/41.jpg)
Glioma Cancer Data: from an RGCCA viewpoint(Department of Pediatric Oncology of the Gustave Roussy Institute)
ξ2
ξ1
ξ3
RGCCA with factorial scheme - τ1 = 1, τ2 = 1 and τ3 = 0
C13 = 1
C23 = 1
C12 = 0
![Page 42: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/42.jpg)
17
![Page 43: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/43.jpg)
18
Bayesian Discriminant Analysis of localization on y1 and y2
y1
y2
![Page 44: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/44.jpg)
19
ObservedPredicted DIPG Hemispheres Midline
DIPG 18 1 1
Hemispheres 0 17 4
Midline 2 6 7
ObservedPredicted DIPG Hemispheres Midline
DIPG 20 0 1
Hemispheres 0 19 4
Midline 0 5 7
Accuracy = 82%
Accuracy = 75%
Table 1. Learning phase
Table 2. Testing phase (leave-one-out)
y1
y2
Predictive performance
![Page 45: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/45.jpg)
20
Block components
![Page 46: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/46.jpg)
20
Block components
![Page 47: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/47.jpg)
Variable selection for RGCCA
and:
where:
argmax&1,&2,…,&+ ,
,-./0 g 1cov56/&/ , 60&078+
/≠0
!"#$ "22 = 1, $ = 1,… , *
"#$ "1 ≤ ,$ , $ = 1,… , * Subject to the constraints
![Page 48: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/48.jpg)
Sparse GCCA
![Page 49: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/49.jpg)
Sparse GCCA
Initialstep
![Page 50: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/50.jpg)
Sparse GCCA
Outer Estimation(explains the block)
Initialstep
![Page 51: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/51.jpg)
Sparse GCCA
Outer Estimation(explains the block)
Initialstep
cjk = 1 if blocks are linked, 0 otherwise and cjj = 0
InnerEstimation(explains relation between block)
Choice of weights ejh:- Horst :
- Centroid :
- Factorial :
![Page 52: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/52.jpg)
Sparse GCCA
Outer Estimation(explains the block)
Initialstep
cjk = 1 if blocks are linked, 0 otherwise and cjj = 0
InnerEstimation(explains relation between block)
Choice of weights ejh:- Horst :
- Centroid :
- Factorial :2),1(
),1(
jjtj
jjtj
jzX
nS
zXn
Sa
λ
λ=
!(#,"$)=sign(#)max(0,"|#|−$)
jκ≤1ja
is chosen such λ jthat
![Page 53: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/53.jpg)
Sparse GCCA
Outer Estimation(explains the block)
Initialstep
Iterate until convergence
of the criterion
cjk = 1 if blocks are linked, 0 otherwise and cjj = 0
InnerEstimation(explains relation between block)
Choice of weights ejh:- Horst :
- Centroid :
- Factorial :2),1(
),1(
jjtj
jjtj
jzX
nS
zXn
Sa
λ
λ=
!(#,"$)=sign(#)max(0,"|#|−$)
jκ≤1ja
is chosen such λ jthat
![Page 54: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/54.jpg)
23
![Page 55: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/55.jpg)
24
List of selected variables from GE data
FOXG1 PTPN9 CYP4Z1 ARFGAP3ZFHX4 WNT5A PI16 PDLIM4EEPD1 COL10A1 TRIM43 VIPR2GRID2 PBX3 BTC ACADLEMX1 TKTL1 PKNOX2 LAMB3DLX2 LY6D SERPINB10 DCAF6ITM2C CRYGD TAAR2 NET1SEMA3D HOXA3 ZNF469 ELOVL2PTHLH KRTAP9-9 FAM196B DAAM2RASL12 LHX1 SLC22A3 CHCHD7PPAPDC1A ZNF483 HOXB2 FAIMHCG4 NLRP7 SLC25A2 HOXA2TRIM16L ABI3BP HES4 SPEF2NR0B1 MCF2 SYT9 C8orf47LHX2 SATB2 C2orf88 DLEC1RNF182 HTR1D CLDN3 FZD7KIAA0556 LOXHD1 GLUD2 PLIN4VAX2 IRX1 OMP KAL1ABP1 NRN1 KCND2 LRRC55SFRP2 C14orf23 C17orf71 FAM89AHERC3 IRX2 ADAMTS20 RSPH1SPDEF C1orf53 SLC1A6 AKR1C3ONECUT2 GLIS1 SORD C11orf86OTX1 HELB VPS37B TBX15OSR1 DLX1 NR2E1 SEMG2
KRASAPOLD1CDKN2BCDKN2ACNOT2ABCC9CAPS2IAPPPPFIBP1NAV3SLCO1A2PTHLHELK3KIAA1467ETNK1RAB3IPTMTC1DDX11GLIPR1L2ITPR2
TMEM19HEBP1BHLHE41C12orf36RAB21C12orf72GSG1C9orf53GLIPR1PTPRBE2F7KIAA0528LGR5ZDHHC17MRPS35C12orf70TBC1D15SSPN
List of selected variables from CGH data
STK38LCAPRIN2SOX5AMN1THAP2PYROXD1PHLDA1CSRP2KRR1PTPRRTM7SF3ZFC3H1CCDC91KCNC2SLCO1B1BCAT1LYRM5RASSF8MED21
FGFR1OP2
BBS10TSPAN11GPRC5DGPRC5ADENND5BNAP1L1KLHDC5DDX47C12orf28LDHBFAR2ST8SIA1LRMPEMP1C12orf11OSBPL8KCNJ8TSPAN8CASC1KCNMB4
![Page 56: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/56.jpg)
25y1
y2
Bayesian Discriminant Analysis of localization on y1 and y2
![Page 57: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/57.jpg)
26
ObservedPredicted DIPG Hemispheres Midline
DIPG 20 0 1
Hemispheres 0 20 3
Midline 0 4 8
ObservedPredicted DIPG Hemispheres Midline
DIPG 20 0 1
Hemispheres 0 22 3
Midline 0 2 8
Accuracy = 89.2% (82% non sparse)
Accuracy = 85.7%(75% non sparse)
Table 1. Learning phase
Table 2. Testing phase (leave-one-out)
y1
y2
Predictive performance
![Page 58: Recent advances on Regularized Generalized …najim/gdr-estimation/tenenhaus.pdf · Recent advances on Regularized Generalized Canonical Correlation Analysis Arthur Tenenhaus 2013/05/16](https://reader031.fdocuments.net/reader031/viewer/2022021902/5b98537a09d3f2fd558bd82e/html5/thumbnails/58.jpg)
Conclusions
• Depending on the dimension of the blocks, you can use either the primal or the dual algorithm.
• The dual representation of the RGCCA algorithm allows:• Analysing high dimensional blocks. • recovering nonlinear relationship between blocks (choice
of the kernel function).
• Sparse constraints are useful when the relevant variables are masked by (too many) noisy variables.
• Sparse constraints are useful when we want to identify a small number of significant variables which are active in the relationships between blocks.