PhD Hearing (Oct 15, 2003) Predictive Computer Models for Medical Classification Problems Predictive...

PhD Hearing (Oct 15, 2003)

Predictive Computer Models Predictive Computer Models for Medical Classification Problemsfor Medical Classification Problems

PhD progress report (2000.8 ~ 2003.10)

Student : Chuan LUPromoters: Prof. Dr. Ir. Sabine Van Huffel Prof. Dr. Ir. Johan Suykens Advisers : Prof. Dr. Dirk Timmerman Prof. Dr. Ir. Joos Vandewalle Prof. Dr. Jan Beirlant


OverviewOverview PhD topic Doctoral Programme:

courses, publication, meetings... Research Work Work plan and timing


PhD TopicPhD Topic The development, statistical analysis and

clinical evaluation of a new class of predictive models which optimally extract information from patient data.

The attention is focused on intelligence machine learning methods such as neural networks, kernel based algorithms, and their integration with Bayesian framework.


Joint Research ActivitiesJoint Research Activities Classification of ovarian tumors

logistic regression (LR) artificial neural networks (ANNs) Bayesian least squares support vector machines (LS-SVMs)

Prediction of pregnancy of unknown location (PUL) LR, LS-SVMs, relevance vector machines (RVMs)

Variable selection for medical classification problems: (Bayesian framework)


The Doctoral ProgrammeThe Doctoral Programme ‘Direct Tuition’ – (520 h)

Doctoral Courses Case Studies in Biomedical Data Processing (25x6=150 h) Phd training course: Longitudinal Data,Incomplete Data,

and Causal Inference (18x1=18 h) Courses in master of statistics

Basic Concepts of Statistical Modeling (60x4=240 h) Applied Statistical Models (60x4=240 h)

Seminars Presentation at BioMed Seminar, SISTA (50 x 1 h) Presentation at SISTA seminar on Feb 28, 2002 (50x0.5 h)


The Doctoral ProgrammeThe Doctoral Programme Other Study Activities and Achievements

Publications (1) – with first authorship[1]‘Preoperative prediction of malignancy of ovarian tumors using least

squares support vector machines’. Artificial Intelligence in Medicine, vol. 28, no. 3, Jul. 2003, pp. 281-306. (200 h)

[2] ‘Using Artificial Neural Networks to Predict Malignancy of Ovarian Cancers’, in Proc. Of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society - EMBC2001, Istanbul, Turkey, Oct. 2001, CD-ROM. (100 h)

[3] ‘Classification of ovarian tumor using Bayesian least squares support vector machines’, accepted for publication in the 9th Conference on Artificial Intelligence in Medicine Europe (AIME 03), Oct 18-22, Cyprus. (100+60 h)

[4] ‘Bayesian Least Squares Support Vector Machines for Classification of Ovarian Tumors’, Internal Report 02-105, ESAT-SISTA, K.U.Leuven (Leuven, Belgium), 2002.


The Doctoral ProgrammeThe Doctoral Programme Publications (2) – with coauthorship[1] ‘Prediction of mental development of preterm newborns at birth time using LS-

SVM’, in Proc. of ESANN'02, Bruges, Belgium, Apr. 2002, pp. 167-172.[2] ‘Color Doppler and Gray-Scale Ultrasound Evaluation of the Postpartum uterus’,

Ultrasound Obstet. Gynecol., vol. 20, 2002, pp. 586-591[3] ‘Prospective evaluation of blood flow in the myometrium and in the uterine arteries

in the puerperium’, Jan 2003, accepted for publication in Ultrasound in Obstetrics and Gynecology.

[4] ‘Subjective use of serum human chorionic gonadotrophin and progesterone levels for the investigation of pregnancies of unknown location : analysis of interuser variability and experience’, 2003.

[5] ‘Can ectopic pregnancies be predicted using serum hormone levels ?’, submitted, 2003.

[6] ‘A novel neural approach to inverse problems with discontinuities (the GMR neural network)’, in IJCNN'03, Portland, Oregon, Jul. 2003, pp. 3106-3111.

[7] ‘Direct Torque control of induction motors by use of the GMR Neural network’, in IJCNN'03, Portland, Oregon, July 20-24, 2003, pp. 2106-2111.



Participation in Scientific Meetings 9th Annual Meeting of the Belgian Statistical Society-BSS2001, Oostende,

Oct 2001, Poster Presentation: ‘Prediction of Malignancy of Ovarian Tumors Using Logistic Regression and Artificial Neural Network Models’

the Advanced NATO Study Institute on Learning Theory and Practice (NATO-ASI LTP 2002), Leuven, Belgium, July 8-19, 2002, Poster Presentation: ‘Blackbox classifiers for preoperative discrimination between malignant and benign ovarian tumors’.

10th Annual Meeting of the Belgian Statistical Society-BSS2002, Kerkrade, The Netherlands, October 18-19, 2002 Poster Presentation: ‘Comparative study on variable selection for nonlinear classifiers ’

Poster presentation at study day of IAP network 2001, 2002 and 2003. Belgian Day of Biomedical Engineering in Brussels, , October 17th 2003,

Poster presentation: ‘Variable selection using linear sparse Bayesian models for medical classification problems’.



Supervision of licentiate thesis (2x150+100=400 h) Thesis supervision for Master of applied statistics: ‘Mathematical models

for predicting the evolution of a pregnancy of unknown location’, 2003 Thesis supervision for Master of applied statistics: ‘Prediction of

pregnancy evolution and ectopic pregnancy’ , 2002. Thesis supervision for ERASMUS student from university de Picardie

Jules Verne, France, 2003, Approches neuronales pour la résolution de problèmes inverses avec discontinuités


Research Research - Building blocks- Building blocks Explorative data analysis (EDA) Probabilistic modeling techniques Variable selection Applications


Explorative Data AnalysisExplorative Data Analysis

Gain insights into a data set: structure, imprtant variables, outiers, and the model suggested by the data.

Techniques: scatterplots, boxplots, histograms, PCA, FA, CCA, biplots, etc.

New nonlinear techniques in EDA: kernel PCA, kernel CCA, and nonlinear biplots. Uncovering the nonlinear structure of the data, aid in nonlinear modeling such as LS-SVM.


Explorative Data AnalysisExplorative Data Analysis

Fig. Biplot of Ovarian Tumor data.

The observations are plotted as points (o - benign,

x - malignant), the variables are plotted as vectors from the origin.

- visualization of the correlation between the variables - visualization of the relations between the variables and clusters.


Explorative Data AnalysisExplorative Data Analysis Fig. Nonlinear Biplot for kernel

PCA with RBF kernels Data projected onto pairs of PCs (PCs

with the largest correlation with y were selected for visualization) , computed by kernel trick.

Approximate decision boundary: ridge regression (y=1) using pairs of PCs

For kth variable, pseudosamples generated by: setting data mean as starting point, varying the value of variable k while fixing the others. Variable trajectory: tracing the projection of the pseudosample onto the pairs of PCs.


Probabilistic ModelingProbabilistic Modeling Probabilistic modeling needed in medical Decis. Supp.

the uncertainty and different mis. class. cost. Traditional statistical linear probabilistic classifiers:

Linear discriminant analysis (LDA) Logistic regression (LR)

Bayesian MLPs Bayesian + Kernel based modeling:

Bayesian LS-SVM classifiers (Suykens 1999, 2001, 2002)

Sparse Bayesian modeling and relevance vector machines (RVMs) (Tipping 2001,2003)


RecipeRecipe

Goal: Linear model y=wTx Nonlinear model Dealing with uncertainty Model selection Sparseness

Ingredients: Kernel trick: x(x) higher dim. feature space Bayesian framework


Bayesian InferenceBayesian Inference Find the maximum a posterior (MAP) estimates of model parameters wMP

and bMP, using conventional LS-SVM training. The posterior probability of the parameters can be estimated via

marginalization using Gaussian probability at wMP, bMP

Assuming a uniform prior p(Hj) over all model, rank the model by the evidence p(D|Hj) evaluated using Gaussian approximation.

( , ) ((

)( ,

(,) )

)=

p D H p Hpp D H

D HH

pD

: Infer hyperparameter Level 2

: Compare modelsLevel 3 ( ) (

( )) ()

( )j j

j j

p D H p Hp D

p DpH D H

, e( , )

( ,xp( ( ,

, , ) ( , , ), , ))

p D w b H p w bw b J

Hp w

D Hb

PD H

: for rbf kernels, (model kernel parameter, e.g.

hyperp: regularizaramete ation prs, e arame rg. s. )te

H

: infer , for given , w b H Level 1


Variable selectionVariable selection

Importance in medical classification problems economics of data acquisition accuracy and complexity of the classifiers gain insights into the underlying medical problem.

Filter approaches: filter out irrelevant attributes before induction occurs

Wrapper approaches: focus on finding attributes that are useful for performance for a specific type of model, rather than necessarily finding the relevant ones.


Variable selectionVariable selection Heuristic search:

forward, backward, stepwise hill-climbing, branch and bound…

Variable selection criteria: Correlation, fisher score, mutual information Evidence in Bayesian framework Classification performance, e.g. AUC Sensitivity analysis: change in the objective function J by

removing variable i: DJ(i) Statistical chi-square test


Variable selectionVariable selection We focus on evidence (marginal likelihood) based

method within the Bayesian framework Forward / stepwise selection Bayesian LS-SVM Sparse Bayesian models Accounting for uncertainty in variable selection


ApplicationApplication - Ovarian tumor classification- Ovarian tumor classification

Problem develop a reliable diagnostic tool to discriminate

preoperatively between benign and malignant tumors. assist clinicians in choosing the appropriate treatment.

Data (from IOTA project) Patient data collected at Univ. Hospitals Leuven, Belgium, 1994~1999 425 records, 25 features. 291 benign tumors, 134 (32%) malignant tumors.



Forward variable selection based on Bayesian LS-SVMEvolution of the model evidence

10 variables were selected based on the training set (first treated 265 patient data) using RBF kernels.



Predictive power of the models given the selected variablesROC curves on test Set (data from 160 newest treated patients)



Performance on test set with rejection based on, e.g.,

The rejected patients need further examination by human experts

Posterior probability essential for medical decision making

Reject AUC Accuracy Sensitivity Specificity10% (16) 0.9420 88.97 83.72 91.4

5% (8) 0.9343 87.50 82.61 89.8

0% (0) 0.9184 84.38 77.78 87.74

Reject AUC Accuracy Sensitivity Specificity10% (16) 0.9420 88.97 83.72 91.4

5% (8) 0.9343 87.50 82.61 89.8

0% (0) 0.9184 84.38 77.78 87.74

| ( 1| , , ) - 0.5 | uncertaintyP y x D H | ( 1| , , ) - 0.5 | uncertaintyP y x D H


Application-Application- binary cancer classification binary cancer classification based on microarray databased on microarray data

Variable selection using linear sparse Bayesian logit model, LOO CV accuracy

#Var RVM LS-SVM LR LDALeukemia all: 7129 0.9310 0.958 N/A N/A

sel: 4 1 1 1 0.986

Colon all: 2000 0.823 0.871 N/A N/Asel: 5 0.984 1 1 1

#Var RVM LS-SVM LR LDALeukemia all: 7129 0.9310 0.958 N/A N/A

sel: 4 1 1 1 0.986

Colon all: 2000 0.823 0.871 N/A N/Asel: 5 0.984 1 1 1

cancer no. samples no. genes task

leukemia 72 7192 2 subtypes

colon 62 2000 disease/normal


Application-Application- brain tumor multiclass brain tumor multiclass classification based on MRS spectra dataclassification based on MRS spectra data

4 types of brain tumors, 205x138 magnitude value

Variable selection using linear sparse Bayesian logit model, 30 runs of random CV accuracy#Var RVM LS-SVM LR LDA

all: 138 69.95% 68.48% N/A N/A±2.88% ±3.03% N/A N/A

sel:27 74.07% 75.34% 74.61% 75.05%±2.82% ±3.55% ±3.64% ±3.47%

#Var RVM LS-SVM LR LDAall: 138 69.95% 68.48% N/A N/A

±2.88% ±3.03% N/A N/Asel:27 74.07% 75.34% 74.61% 75.05%

±2.82% ±3.55% ±3.64% ±3.47%


Work Plan and TimingWork Plan and Timing Nov - Overview paper on Linear and nonlinear preoperative

classification of ovarian tumors (chapter proposal accepted for edited book "Knowledge Based Intelligent System for Health Care.")

Dec – Jan 2003, paper on variable selection Feb 2004 – model averaging Model evaluation using IOTA data. April 2004 – writing draft of thesis Sept 2004 - Defense

PhD Hearing (Oct 15, 2003) Predictive Computer Models for Medical Classification Problems Predictive...

Documents

Transcript of PhD Hearing (Oct 15, 2003) Predictive Computer Models for Medical Classification Problems Predictive...