Using Bayesian Networks to Analyze Expression Data
By Friedman Nir, Linial Michal, Nachman Iftach, Pe'er Dana (2000)
Presented byNikolaos Aravanis
Lysimachos ZografosAlexis Ioannou
Outline
• Introduction• Bayesian Networks• Application to expression data• Application to cell cycle expression patterns• Discussion and future work
The Road to Microarray Data Analysis
• Development of microarrays– Measure all the genes of an organism
• Enormous amount of data
• Challenge: Analyze datasets and infer biological interactions
Most Common Analysis Tool
• Clustering Algorithms
• Allocate groups of genes with similar expression patterns over a set of experiments
• Discover genes that are co-regulated
Problems
• Data give only a partial picture– Key events are not reflected (translation and
protein (in) activation)
• Amount of samples give few information for constructing full detailed models
• Using current technologies even few samples have high noise to signal ratio
Possible Solution
• Analyze gene expression patterns that uncover properties of the transcriptional program
• Examine dependence and conditional independence of the data
• Bayesian Networks
Bayesian Networks
• Represent dependence structure between multiple interacting quantities
• Capable of handling noise and estimating the confidence in the different features of the network
• Focus on interactions whose signal is strong• Useful for describing processes composed of locally
interacting components• Statistical foundations for learning Bayesian networks
and the statistics to do so are well understood and have been successfully applied
• Provide models of causal influence
Informal Introduction to Bayesian Networks
•Let P(X,Y) be a joint distribution over variables X and Y
•X and Y independent if P(X,Y) = P(X)P(Y) for all values X and Y
•Gene A is transcriptor factor of gene B•We expect their expression level to be dependent•A parent of B
•B trascription factor of C•Expression levels of each pair are dependent
•If A does not directly affect C, if we fix the expression level of B, we will observe A and C are independent
•P(A|B,C) = P(A|B) (A and C conditionally independent of B) I(A;C|B)
Informal Introduction to Bayesian Networks (contd’)
•Component of Bayesian Networks is that each variable is a stochastic function of its parents
•Stochastic models are natural in gene expression domain•The biological models we want to process are stochastic•Measurements are noisy
Representing Distributions with
Bayesian Networks• Representation of joint probability distribution
consisting of 2 components• Directed acyclic graph (G)• Conditional distribution for each variable given its
parents in G• G encodes Markov Assumption
• By applying chain rule this decomposes in product
form
iiii PaentsNonDescend |;,
Equivalence Classes of BNs
A BN implies further independence assumptions
=> Ind(G)
>1 graphs can imply the same assumptions
=> Equivalent networks if Ind(G)=Ind(G')
Equivalence Classes of BNs
A BN implies further independence statements
=> Ind(G)
>1 graphs can imply the same statements
=> Equivalent networks if Ind(G)=Ind(G')
Ind(G)=Ind(G')=Ø
Equivalence Classes of BNs
For equivalent networks: DAGs have the same underlying undirected
graph. PDAGs are used to represent them.
Equivalence Classes of BNs
For equivalent networks: DAGs have the same underlying undirected
graph. PDAGs are used to represent them.
Disagreeingedge
Question:
Given dataset D, what BN, B=<G,Θ> best matches D?
Answer:
Statistically motivated scoring function to evaluate each BN: e.g. Bayesian Score
S(G:D)=logP(G|D)=logP(D|G)+logP(G)+C,
where C is a constant independent of G
and P(D|G)=∫P(D|G,Θ)P(Θ|G)dΘis the marginal likelihood over all parameters for
G.
Learning BNs
Question:
Given dataset D, what BN, B=<G,Θ> best matches D?
Answer:
Statistically motivated scoring function to evaluate each BN: e.g. Bayesian Score
S(G:D)=logP(G|D)=logP(D|G)+logP(G)+C,
where C is a constant independent of G
and P(D|G)=∫P(D|G,Θ)P(Θ|G)dΘis the marginal likelihood over all parameters for
G.
Learning BNs
Learning BNs (contd)Steps:
– Decide priors (P(Θ|D), P(G))
=> Use of BDe priors
(structure equivalent, decomposable)
– Find G to maximize S(G:D)
NP hard problem
=>local search using local permutations of candidate G
(Heckerman et al. 1995)
Learning Causal Patterns– Bayesian Network is model of dependencies
– Interest in modelling the process that generated them.
=> model the flow of causality in the system of interest and create a Causal Network (CN).
A Causal Network models the probability distribution
as well as the effect of causality.
CNs VS BNs:
- CNs interpret parents as immediate causes
(c.f. BNs)
- CNs and BNs relate when using the
Causal Markov Assumption :
“given the values of a variable's immediate causes, it is independent of its earlier causes”, if
this holds, then BN==CN
Learning Causal Patterns
CNs VS BNs:
- CNs interpret parents as immediate causes
(c.f. BNs)
- CNs and BNs relate when using the
Causal Markov Assumption :
“given the values of a variable's immediate causes, it is independent of its earlier causes”, if
this holds, then BN==CN
Learning Causal Patterns
X
YX
Yequivalent BNs
but not CNs
Applying BNs to Expression Data
Expression level of each gene as a random variable
Other attributes (e.g temperature, exp. conditions) that affect the system can be
modelled as random variables Bayesian Net/ Dependency structure can
answer queries CON: problems in computational complexity and the statistical significance of the resulting
networks. PRO: genetic regulation networks are sparse
Representing Partial Models
– Gene networks: many variables
=> >1 plausible models (not enough data) – we can learn up to equivalence class.
Focus on feature learning in order to have a causal network:
Representing Partial ModelsFeatures:
- Markov relations (e.g. Markov Blanket)
- Order relations (e.g. X is an ancestor of Y in all networks)
Representing Partial ModelsFeatures:
- Markov relations (e.g. Markov Blanket)
- Order relations (e.g. X is an ancestor of Y in all networks)
Feature learning leads to a Causal Network
Statistical Confidence of Features
– Likelihood that a given feature is actually true.– Can't calculate posterior (P(G|D))
=> Bootstrap method
for i=1...n resample D with replacement -> D';
learn G' from D'; end
Statistical Confidence of Features
Individual feature confidence (IFC)
IFC = (1/n)∑{f(G')}
where f(G') = 1 if the feature exists in G'
Efficient Learning Algorithms
– Vast search space
=> need efficient algorithms– Attention on relevant regions of the search
space
=> Sparse Candidate Algorithm
Efficient Learning Algorithms
Sparse Candidate Algorithm
Identify a small number of candidate parents for each gene based on simple local statistics
(e.g. correlation).
– Restrict our search to networks with the candidate parents
– Potential pitfall: early choice
=> Solution: adaptive algorithm
DiscretizationThe practical side:
Need to define the local probability model for each variable.
=> discretize experimental data into -1,0,1
(expression level lower, similar, higher than control)
Set control by averaging. Set a threshold ratio for significantly
higher/lower.
Application to Cell Cycle Expression Patterns
• 76 gene expression measurements of the mRNA levels of 6177 Saccharomyces cerevisiae ORFs. Six time series under different cell cycle synchronization methods(Spellman 1998).
• 800 differentially expressed, 250 clustered in 8 distinct clusters. Variables for the networks represent the expression level of the 800 genes.
• Introduced an additional variable that denoted the cell cycle phase to deal with the temporal nature of the cell cycle process and forced it as a root in the network
• Applied Sparse Candidate Algorithm to 200- fold bootstrap of the original data.
• Used no prior biological knowledge in the learning algorithm
Network with all edges
Network with edges that represent relations with confidence level above 0.3
YNL058C Local Map
• Edges • Markov• Ancestors• Descendants• SGD entry• YPD entry
Robustness analysis
• Use 250 gene data for robustness analysis
• Create random data set by permuting the order of experiments independently for each gene
• No “real” features are expected to be found
Robustness analysis (contd’)
• Lower confidence for order and Markov relations in the random data set• Longer and heavier tail in the high confidence region in the original data set
• Sparser networks learned from real data• Features learned in original data with high confidence level are not an artifact of the bootstrap estimation
Robustness analysis (contd’)
• Compared confidence level of learned features between 250 and 800 gene data set
• Strong linear correlation• Compared confidence level of learned features between different
discretization thresholds• Definite linear tendency
Biological Analysis
Order relations• Dominant genes indicate potential causal
sources of the cell cycle process
• Dominance score of X
where is the confidence in X being ancestor of Y , k is used to reward high confidence features and t is a threshold to discard low confidence ones
tYXCY
ko YXC
),(, 0),(
),( YXCo
Biological Analysis (contd’)
• Dominant genes are key genes in basic cell functions
Biological Analysis (contd’)
• Top Markov relations reveal functional relations between genes1. Both genes known: The relations make sense biologically 2. One unknown gene: Firm homologies to proteins functionally
related to the other gene3. Two unknown genes: Physically adjacent to the chromosome,
presumably regulated by the same mechanism• FAR1- ASH1, low correlation, different clusters, known though to participate in a mating type switch• CLN2 is likely to be a parent to RNR3, SVS1, SRO4 and RAD41. Appeared in same cluster. No links
between the 4 genes. CLN2 is known to be a central cycle control and there is no clear biological relationship between the others
Markov relations
Discussion and Future Work
• Applied Sparse Candidate Algorithm and Bootstrap resampling to extract a Bayesian Network for the 800 genes data set of Spellman
• Used no prior biological knowledge• Derived biologically plausible conclusions• Capability of discovering causal relationships, interactions between genes
and rich structure between clusters.
• Developing hybrid algorithms with clustering algorithms to learn models over clustered genes
• Extensions:– Learn local probability models dealing with continuous data– Improve theory and algorithms– Include biological knowledge as prior knowledge– Improve search heuristics– Apply Dynamic Bayesian Networks to temporal data– Discover causal patterns (using interventional data)
Top Related