1
Markov Random Fields (MRF)Presenter Kuang-Jui HsuDate 2011/5/23(Tues.)Outline Introduction
Conditional Independence Properties
Factorization Properties
Illustration: Image De-noising
Relation to Directed Graphs
IntroductionBased on a undirected graph
The MRF model has a simple form and is easy to use
Based on conditional independence properties
Conditional Independence Properties In an undirected graph, there are three sets of nodes, denoted A, B, C, and A is conditionally independent of B given C
Shorthand notation: p(A|B, C) = p(A|C)Conditional independence propertyTesting Methods in a Graph
Testing Methods in a Graph
Testing Methods in a Graph
Testing Methods in a Graph
Testing Methods in a Graph
Testing Methods in a Graph
Testing Methods in a Graph
Simple Form
A node will be conditionally independent of all other nodes conditioned only on neighbouring nodesFactorization Properties In a directed graph
Generalized form:In an Undirected Graph
Consider two nodes and that are not connected.
Must be conditionally independent
So, the conditional independence property can be expressed as The set x of all variables with and removed
Factorization PropertyCliqueThis leads us to consider a graphical concept: Clique
Clique:
Maximal Clique:
Define the factors in the potential function by using the clique
Generally, consider the maximal cliques, because other cliques must be the subsets of maximal cliquesPotential FunctionPotential FunctionPotential function over the maximal cliques of the graph
CliqueThe set of variables in that clique
The joint distribution: Partition function: a normalization constant
Equal to zero or positivePartition FunctionThe normalization constant is the major limitationsA model with M discrete nodes each having K states, then the evaluation involves summing over states
Exponential growth Needed for parameter learningBecause it will be a function of any parameters that govern the potential functions
Connection between Conditional Independence And FactorizationDefine :
For any node , the following conditional property holds
All nodes expect
The neighborhood of
Define :
A distribution can be expressed as
The Hammerley-Clifford theorem states that the sets and identical.
Potential Function ExpressionRestrict the potential function to be positive It is convenient to express them as exponentials
Energy functionBoltzmann distribution
The total energy is obtained by adding the energies of each of the maximal energy
Illustration: Image De-noising
Noisy imageDescribed by an array of binary pixel values , where the index i = 1, . . ., D runs over all pixels.
Illustration: Image De-noisingNoise-free imageDescribed by an array of binary pixel values , and randomly flipping the sign of pixels with some small probability
Create the MRF ModelA strong correlation between and
A strong correlation between the neighbouring pixles
MRF model:
The graph has two types of cliques, each of which contain two variables.
The clique form , uses the form of the energy function
The clique form , uses the form of the energy function
The parameters and are positive, and are neighbour
The Energy Function The complete energy function:
The joint distribution
1. postitve2. negativeSolve by ICMFor the purpose of image restoration, find an image x having a high probabilityUse a simple iterative technique called iterated condition mode ( ICM)Simply an application of coordinate-wise gradient ascent The steps of ICM
Evaluate the total energy for -1 and 1choose the lower energy, and update Stop until convergence
Result
Use ICMUse graph-cutRelation to Directed Graphs
Solve the problem of taking a model that is specified using a directed graph and trying to convert it to undirected graph
Directed graph
Undirected graph
Relation to Directed Gaphs
This is easily done by identifying
Relation to Directed Graphs
Consider how to generalize this constructionThis can be achieved if the clique potentials of the undirected graph are given by the conditional distributions of the directed graph.
Ensure that the set of variables that appears in each of conditional distributions is a member of at least one clique of the undirected graph Generalize This Construction
For nodes having one parent
For nodes having more than one parent
Involving the four variables, so they must belong to a single clique if this conditional distribution is to be absorbed in a clique potential
The process has become known as moralizationMoral graphConvert the Directed Graph to the Undirected Graph Discard some conditional independence propertiesConvert the Directed Graph to the Undirected Graph In fact,we can simply using a fully connected undirected graph However, this would discard all conditional propertiesThe moralization adds the fewest extra links and so retain the maximum number of independence properties Special Graph
There are two type of graph that can express different conditional independence properties Type 1: dependence map(D-map)Type 2: Independence map(I-map)Dependence Map(D-Map)
Every conditional independence statement satisfied by the distribution is reflected in the graphA completely disconnected graphIndependence Map(I-Map)
Every conditional independence statement implied by a graph is satisfied by a specific distributionA full connected graphA perfect map: both I-map and D-mapPerfect Map
The set of all distributions P over a given set of variablesDirectedgraphUndirectedgraph
Top Related