Algorithms, Graph Theory, and the Soluon of …mcgrew/media/Events/distinguished/...the Soluon of...
Transcript of Algorithms, Graph Theory, and the Soluon of …mcgrew/media/Events/distinguished/...the Soluon of...
Algorithms,GraphTheory,andtheSolu7onofLaplacianLinear
Equa7ons
DanielA.SpielmanYaleUniversity
Rutgers, Dec 6, 2011
Outline
LinearSystemsinLaplacianMatricesWhat?Why?Classicwaystosolvethesesystems.
Approxima7ngGraphsbyTrees
SparseApproxima7onsofGraphs
LocalGraphClustering
LaplacianLinearSystems
Solvein7mewhere=numberofnon‐zerosentriesofA
7mesfor‐approximatesolu7on.
Enablessolu7onofallsymmetric,diagonally‐dominantsystems,includingsub‐matricesofLaplacians.
O(m logc m)m
log(1/!) !!!x!A!1b
!!A" !
!!A!1b!!A
Ax = b
LaplacianQuadra7cFormof
For x : V ! IR
!1!3 0x :
xTLGx = 15
22 1212
32
xTLGx =!
(u,v)!E
(x (u)! x (v))2
1
3
G = (V,E)
LaplacianQuadra7cFormof
For x : V ! IR
0x :
12
xTLGx =!
(u,v)!E
(x (u)! x (v))2
1
G = (V,E)
0 1
10
0
0
xTLGx = 1
Laplacian Quadratic Form, examples
When x is the characteristic vector of a set S, countstheedgesontheboundaryofS
00
0
1
1
1
S 0xTLGx = |bdry(S)|
Laplacian Quadratic Form, examples
When x is the characteristic vector of a set S, countstheedgesontheboundaryofS
00
0
1
1
1
S 0xTLGx = |bdry(S)|
xTLGx
xTx=
|bdry(S)||S|
=edge‐expansionofS
LearningonGraphs[Zhu‐Ghahramani‐Lafferty’03]
Infervaluesofafunc7onatallver7cesfromknownvaluesatafewver7ces.
Minimize xTLGx =!
(u,v)!E
w(u,v) (x (u)! x (v))2
Subjecttoknownvalues
0
1
0
10.5
0.5
0.6250.375
Taking deriva,ves, minimize by solving Laplacian
Infervaluesofafunc7onatallver7cesfromknownvaluesatafewver7ces.
Minimize xTLGx =!
(u,v)!E
w(u,v) (x (u)! x (v))2
Subjecttoknownvalues
LearningonGraphs[Zhu‐Ghahramani‐Lafferty’03]
OtherApplica7ons
Solveforcurrentwhenfixvoltages
1V
0V
Compu7ngeffec7veresistancesinresistornetworks:
OtherApplica7ons
Solveforcurrentwhenfixvoltages
1V
0V
Compu7ngeffec7veresistancesinresistornetworks:
0.5V
0.5V
0.625V0.375V
LaplacianQuadra7cFormforWeightedGraphs
xTLGx =!
(u,v)!E
w(u,v) (x (u)! x (v))2
G = (V,E,w)
w : E ! IR+ assignsaposi7veweighttoeveryedge
MatrixLGisposi7vesemi‐definitenullspacespannedbyconstvector,ifconnected
LaplacianMatrixofaWeightedGraph
LG(u, v) =
!"#
"$
!w(u, v) if (u, v) " E
d(u) if u = v
0 otherwise
4 -1 0 -1 -2 -1 4 -3 0 0 0 -3 4 -1 0 -1 0 -1 2 0 -2 0 0 0 2
1 2
34
51
1
2
1
3
d(u) =!
(v,u)!E w(u, v)
the weighted degree of u
isadiagonallydominantmatrix
ClassicApplica7ons
Compu7ngeffec7veresistances.
SolvingEllip7cPDEs.
Compu7ngEigenvectorsandEigenvaluesofLaplaciansofgraphs.
SolvingMaximumFlowbyInteriorPointMethods
SolvingLaplacianLinearEqua7onsQuickly
Fastwhengraphissimple,byelimina7on.
Fastapproxima7onwhengraphiscomplicated*,byConjugateGradient
*=randomgraphorhighexpansion
CholeskyFactoriza7onofLaplacians
AlsoknownasY‐Δ
Wheneliminateavertex,connectitsneighbors.
3 -1 0 -1 -1 -1 2 -1 0 0 0 -1 2 -1 0 -1 0 -1 2 0 -1 0 0 0 1
1 2
34
51
1
1
1
1
CholeskyFactoriza7onofLaplacians
AlsoknownasY‐Δ
Wheneliminateavertex,connectitsneighbors.
3 -1 0 -1 -1 -1 2 -1 0 0 0 -1 2 -1 0 -1 0 -1 2 0 -1 0 0 0 1
1 2
34
51
1
1
1
1 .33
.33
.33
3 0 0 0 0 0 1.67 -1.00 -0.33 -0.33 0 -1.00 2.00 -1.00 0 0 -0.33 -1.00 1.67 -0.33 0 -0.33 0 -0.33 0.67
CholeskyFactoriza7onofLaplacians
AlsoknownasY‐Δ
Wheneliminateavertex,connectitsneighbors.
3 -1 0 -1 -1 -1 2 -1 0 0 0 -1 2 -1 0 -1 0 -1 2 0 -1 0 0 0 1
1 2
34
5
1
1 .33
.33
.33
3 0 0 0 0 0 1.67 -1.00 -0.33 -0.33 0 -1.00 2.00 -1.00 0 0 -0.33 -1.00 1.67 -0.33 0 -0.33 0 -0.33 0.67
3 0 0 0 0 0 1.67 -1.00 -0.33 -0.33 0 -1.00 2.00 -1.00 0 0 -0.33 -1.00 1.67 -0.33 0 -0.33 0 -0.33 0.67
3 -1 0 -1 -1 -1 2 -1 0 0 0 -1 2 -1 0 -1 0 -1 2 0 -1 0 0 0 1
3 0 0 0 0 0 1.67 0 0 0 0 0 1.4 -1.2 -0.2 0 0 -1.2 1.6 -0.4 0 0 -0.2 -0.4 0.6
1 2
34
51
1
1
1
1
1 2
34
5.33
.33
1
1 .33
1 2
34
5 .2
1.2
.4
1 0 0 0 0 0 2 -1 0 -1 0 -1 2 -1 0 0 0 -1 2 -1 0 -1 0 -1 2
1 -1 0 0 0 -1 3 -1 0 -1 0 -1 2 -1 0 0 0 -1 2 -1 0 -1 0 -1 2
1 0 0 0 0 0 2 0 0 0 0 0 1.5 -1 -0.5 0 0 -1.0 2 -1.0 0 0 -0.5 -1 1.5
2 3
45
11
1
1
1
1
2 3
45
11
1
1
1
2 3
45
1
1
1
0.5
Theordermaeers
ComplexityofCholeskyFactoriza7on
#ops~Σv(degreeofvwheneliminate)2
Tree #ops~O(|V|)
Planar #ops~O(|V|3/2)Lipton‐Rose‐Tarjan‘79
ComplexityofCholeskyFactoriza7on
#ops~Σv(degreeofvwheneliminate)2
Tree #ops~O(|V|)
Planar #ops~O(|V|3/2)Lipton‐Rose‐Tarjan‘79
Expander likerandom,butO(|V|)edges
#ops≳Ω(|V|3)Lipton‐Rose‐Tarjan‘79
For S ! V
!G = minS!V !(S)
S
ExpansionandCholeskyFactoriza7on
!(S) =|bdry(S)|
min (|S| , |V ! S|)
CholeskyslowwhenexpansionhighCholeskyfastwhenlowforGandallsubgraphs
Cheeger’sInequalityandtheConjugateGradient
Cheeger’sinequality(degree‐dunwtedcase)
=second‐smallesteigenvalueofLG ~d/mixing7meofrandomwalk
!2
neardforexpandersandrandomgraphs
1
2
!2
d! !G
d!
!2!2
d
Cheeger’sInequalityandtheConjugateGradient
Cheeger’sinequality(degree‐dunwtedcase)
=second‐smallesteigenvalueofLG ~d/mixing7meofrandomwalk
!2
ConjugateGradientfinds∊ ‐approxsolu7ontoLG x = b
inmultsbyLGO(!d/!2 log "!1)
isops
1
2
!2
d! !G
d!
!2!2
d
O(dm!!1G log !!1)
Fastsolu7onoflinearequa7ons
ConjugateGradientfastwhenexpansionhigh.
Elimina7onfastwhenlowforGandallsubgraphs.
Fastsolu7onoflinearequa7ons
Elimina7onfastwhenlowforGandallsubgraphs.
Planargraphs
Wantspeedofextremesinthemiddle
ConjugateGradientfastwhenexpansionhigh.
Fastsolu7onoflinearequa7ons
Elimina7onfastwhenlowforGandallsubgraphs.
Planargraphs
Wantspeedofextremesinthemiddle
Notallgraphsfitintothesecategories!
ConjugateGradientfastwhenexpansionhigh.
Precondi7onedConjugateGradient
SolveLG x = bby
Approxima7ngLGbyLH (theprecondi7oner)
Ineachitera7onsolveasysteminLHmul7plyavectorbyLG
∊ ‐approxsolu7onaserO(
!!(LG, LH) log "!1) itera7ons
condi,on number/approx quality
Inequali7esandApproxima7on
if for all x, xTLHx ! xTLGxLH ! LG
Example:ifHisasubgraphofG
xTLGx =!
(u,v)!E
w(u,v) (x (u)! x (v))2
Inequali7esandApproxima7on
!(LG, LH) ! t LH ! LG ! tLHif
if for all x, xTLHx ! xTLGxLH ! LG
CallsuchanHat‐approxofG
Inequali7esandApproxima7on
!(LG, LH) ! t iff
if for all x, xTLHx ! xTLGxLH ! LG
CallsuchanHat‐approxofG
!c : cLH ! LG ! ctLH
Vaidya’sSubgraphPrecondi7oners
Precondi7onGbyasubgraphH
LH ! LG Justneedtoknowts.t. LG ! tLH
EasytoboundtifHisaspanningtree
And,easytosolveequa7onsinLH byelimina7on
H
ApproximateLaplacianSolvers
ConjugateGradient[Hestenes‘51,S7efel’52]
Vaidya‘90:AugmentedMST
Boman‐Hendrickson’01:UsingLow‐StretchSpanningTrees
S‐Teng’04:Spectralsparsifica7on
Kou7s‐Miller‐Peng‘11:Elegance
O(m logc n)
O(m log n)
TheStretchofSpanningTrees
Where
Boman‐Hendrickson‘01:
stT (G) =!
(u,v)!E
path-lengthT (u, v)
LG ! stT (G)LT
TheStretchofSpanningTrees
path‐len3
Where
Boman‐Hendrickson‘01:
stT (G) =!
(u,v)!E
path-lengthT (u, v)
LG ! stT (G)LT
TheStretchofSpanningTrees
path‐len5
Where
Boman‐Hendrickson‘01:
stT (G) =!
(u,v)!E
path-lengthT (u, v)
LG ! stT (G)LT
TheStretchofSpanningTrees
path‐len1
Where
Boman‐Hendrickson‘01:
stT (G) =!
(u,v)!E
path-lengthT (u, v)
LG ! stT (G)LT
TheStretchofSpanningTrees
Inweightedcase,measureresistancesofpaths
Where
Boman‐Hendrickson‘01:
stT (G) =!
(u,v)!E
path-lengthT (u, v)
LG ! stT (G)LT
49
FundamentalGraphicInequality
1 8
1 2 3
8 7
4 5
6
edge k times path of length k
With weights, corresponds to resistors in serial (Poincaré inequality)
1 2 3
8 7
4 5
6
2 3
7
4 5
6
TheStretchofSpanningTrees
Where
Boman‐Hendrickson‘01:
stT (G) =!
(u,v)!E
path-lengthT (u, v)
LG ! stT (G)LT
Low‐StretchSpanningTrees
(Alon‐Karp‐Peleg‐West’91)
(Elkin‐Emek‐S‐Teng’04,Abraham‐Bartal‐Neiman’08)
ForeveryGthereisaTwith
where m = |E|
Solvelinearsystemsin7me O(m3/2 logm)
stT (G) ! m1+o(1)
stT (G) ! O(m logm log2 logm)
Sparsifica7on
Goal:findsparseapproxima7onforeveryG
S‐Teng‘04:ForeveryGisanHwithO(n log7 n/!2) edgesand!(LG, LH) ! 1 + "
Sparsifica7on
Goal:findsparseapproxima7onforeveryG
S‐Teng‘04:ForeveryGisanHwithO(n log7 n/!2) edgesand!(LG, LH) ! 1 + "
S‐Srivastava‘08:withedgesbyrandomsamplingbyeffec7veresistances
O(n log n/!2)
0V
0.53V
0.27V
0.33V0.2V
1V
0V
u
v1/(currentflowatonevolt)
Sparsifica7on
Goal:findsparseapproxima7onforeveryG
S‐Teng‘04:ForeveryGisanHwithO(n log7 n/!2) edgesand!(LG, LH) ! 1 + "
S‐Srivastava‘08:withedges
Batson‐S‐Srivastava‘09
determinis7c,poly7me,andedges
O(n log n/!2)
O(n/!2)
Ultra‐Sparsifiers[S‐Teng]
ApproximateG byatreeplusedges
Sparsifiers Low‐StretchTrees
n/ log2 n
LH ! LG ! c log2 n LH
CholeskyfactortosmallersystemEliminatedegree1and2nodes
Getsystemofsize,solverecursively[Joshi‘97,Reif‘98,S‐Teng’04‘09]
O(n/ log2 n)
Ultra‐Sparsifiers
SolvesystemsinH by:1.Choleskyelimina7ngdegree1and2nodes
2.recursivelysolvingreducedsystem
Time
O(m logc m)
Kou7s‐Miller‐Peng‘11
Solvein7me O(m log n log2 log n log(1/!))
BuildUltra‐Sparsifierby:1.Construc7nglow‐stretchspanningtree2.Addingotheredgeswithprobability
pu,v ! path-lengthT (u, v)
CodebyYiannisKou7s
GivenvertexofinterestfindnearbyclusterSwithsmallexpansion*in7meO(|S|)
LocalGraphClustering[S‐Teng‘04]
*Actually,useconductance.Countver7cesbydegree.
Prove:GivenasetSofsmallexpansionandarandomvertexvofSprobablyfindasetTofsmallexpansion mostofTinsideS
in7meO(|T|)
LocalGraphClustering[S‐Teng‘04]
S
Prove:GivenasetSofsmallexpansionandarandomvertexvofSprobablyfindasetTofsmallexpansion mostofTinsideS
in7meO(|T|)
LocalGraphClustering[S‐Teng‘04]
S
Prove:GivenasetSofsmallexpansionandarandomvertexvofSprobablyfindasetTofsmallexpansion mostofTinsideS
in7meO(|T|)
LocalGraphClustering[S‐Teng‘04]
S
Prove:GivenasetSofsmallexpansionandarandomvertexvofSprobablyfindasetTofsmallexpansion mostofTinsideS
in7meO(|T|)
LocalGraphClustering[S‐Teng‘04]
S
Prove:GivenasetSofsmallexpansionandarandomvertexvofSprobablyfindasetTofsmallexpansion mostofTinsideS
in7meO(|T|)
LocalGraphClustering[S‐Teng‘04]
Sv
Prove:GivenasetSofsmallexpansionandarandomvertexvofSprobablyfindasetTofsmallexpansion mostofTinsideS
in7meO(|T|)
LocalGraphClustering[S‐Teng‘04]
Sv
T
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:startatonenodeateachstep,αfrac7ondriesofwetpaint,halfstaysput,halftoneighbors
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:startatonenodeateachstep,αfrac7ondriesofwetpaint,halfstaysput,halftoneighbors
1 0 0
dry
wet
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:startatonenodeateachstep,αfrac7ondriesofwetpaint,halfstaysput,halftoneighbors
.66 0 0
dry
wet
(α=1/3)
.33
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:startatonenodeateachstep,αfrac7ondriesofwetpaint,halfstaysput,halftoneighbors
.66 0 0
dry
wet
(α=1/3)
.33
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:startatonenodeateachstep,αfrac7ondriesofwetpaint,halfstaysput,halftoneighbors
.33 .33 0
dry
wet
(α=1/3)
.33
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:startatonenodeateachstep,αfrac7ondriesofwetpaint,halfstaysput,halftoneighbors
.22 .22 0
dry
wet
(α=1/3)
.44 .11
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:startatonenodeateachstep,αfrac7ondriesofwetpaint,halfstaysput,halftoneighbors
dry
wet
(α=1/3)
.44 .11
.22 .22 0
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:startatonenodeateachstep,αfrac7ondriesofwetpaint,halfstaysput,halftoneighbors
.17 .22 .06
dry
wet
(α=1/3)
.44 .11
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:
1 0 0
dry
wet
Timedoesn’tmaeer,canpushasynchronously
Approximate:onlypushwhenalotofpaint
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:
1 0 0
dry
wet
Timedoesn’tmaeer,canpushasynchronously
Approximate:onlypushwhenalotofpaint
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:
Timedoesn’tmaeer,canpushasynchronously
Approximate:onlypushwhenalotofpaint
.33 .33 0
dry
wet
.33
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:
.33 .33 0
dry
wet
.33
Timedoesn’tmaeer,canpushasynchronously
Approximate:onlypushwhenalotofpaint
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:
.11 .06
dry
wet
.33 .11
Timedoesn’tmaeer,canpushasynchronously
Approximate:onlypushwhenalotofpaint
.39
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:
.11 .06
dry
wet
.33 .11
Timedoesn’tmaeer,canpushasynchronously
Approximate:onlypushwhenalotofpaint
.39
UsingApproximatePersonalPageRankVectors
Jeh‐Widom‘03,Berkhin‘06,Andersen‐Chung‐Lang’06
Spillingpaintinagraph:
.24 .06
dry
wet
.46 .11
Timedoesn’tmaeer,canpushasynchronously
Approximate:onlypushwhenalotofpaint
.13
Volume‐BiasedEvolvingSetMarkovChain
[Andersen‐Peres‘09]
Walkonsetsofver7cesstartsatonevertex,endsatV
Dualtorandomwalkongraph
Whenstartinsidesetofconductancefindsetofconductance!1/2 log1/2 n
withwork |S| logc n/!1/2
Volume‐BiasedEvolvingSetMarkovChain
[Andersen‐Peres‘09]
Walkonsetsofver7cesstartsatonevertex,endsatV
Dualtorandomwalkongraph
Whenstartinsidesetofconductancefindsetofconductance!1/2 log1/2 n
withwork |S| logc n/!1/2
can we eliminate this?
OpenProblems
FasterandbeeerLow‐StretchSpanningTrees.
Fasterhigh‐qualitysparsifica7on.
Fasterlocalclusteringandgraphdecomposi7on.
Otherfamiliesoflinearsystems.