Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns:...
Transcript of Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns:...
![Page 1: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/1.jpg)
CSE473:Ar+ficialIntelligence
Bayes’Nets
LukeZe?lemoyer---UniversityofWashington[TheseslideswerecreatedbyDanKleinandPieterAbbeelforCS188IntrotoAIatUCBerkeley.AllCS188materialsareavailableath?p://ai.berkeley.edu.]
![Page 2: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/2.jpg)
Probabilis+cModels
§ Modelsdescribehow(apor+onof)theworldworks§ Modelsarealwayssimplifica+ons
§ Maynotaccountforeveryvariable§ Maynotaccountforallinterac+onsbetweenvariables§ “Allmodelsarewrong;butsomeareuseful.”
–GeorgeE.P.Box
§ Whatdowedowithprobabilis+cmodels?§ We(orouragents)needtoreasonaboutunknown
variables,givenevidence§ Example:explana+on(diagnos+creasoning)§ Example:predic+on(causalreasoning)§ Example:valueofinforma+on
![Page 3: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/3.jpg)
Independence
![Page 4: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/4.jpg)
§ Twovariablesareindependentif:
§ Thissaysthattheirjointdistribu+onfactorsintoaproducttwosimplerdistribu+ons
§ Anotherform:
§ Wewrite:
§ Independenceisasimplifyingmodelingassump2on
§ Empiricaljointdistribu+ons:atbest“close”toindependent
§ Whatcouldweassumefor{Weather,Traffic,Cavity,Toothache}?
Independence
![Page 5: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/5.jpg)
Example:Independence?
T W P
hot sun 0.4
hot rain 0.1
cold sun 0.2
cold rain 0.3
T W P
hot sun 0.3
hot rain 0.2
cold sun 0.3
cold rain 0.2
T P
hot 0.5
cold 0.5
W P
sun 0.6
rain 0.4
![Page 6: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/6.jpg)
Example:Independence
§ Nfair,independentcoinflips:
H 0.5
T 0.5
H 0.5
T 0.5
H 0.5
T 0.5
![Page 7: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/7.jpg)
Condi+onalIndependence
![Page 8: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/8.jpg)
Condi+onalIndependence§ P(Toothache,Cavity,Catch)
§ IfIhaveacavity,theprobabilitythattheprobecatchesinitdoesn'tdependonwhetherIhaveatoothache:§ P(+catch|+toothache,+cavity)=P(+catch|+cavity)
§ ThesameindependenceholdsifIdon’thaveacavity:§ P(+catch|+toothache,-cavity)=P(+catch|-cavity)
§ Catchiscondi2onallyindependentofToothachegivenCavity:§ P(Catch|Toothache,Cavity)=P(Catch|Cavity)
§ Equivalentstatements:§ P(Toothache|Catch,Cavity)=P(Toothache|Cavity)§ P(Toothache,Catch|Cavity)=P(Toothache|Cavity)P(Catch|Cavity)§ Onecanbederivedfromtheothereasily
![Page 9: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/9.jpg)
Condi+onalIndependence
§ Uncondi+onal(absolute)independenceveryrare(why?)
§ Condi2onalindependenceisourmostbasicandrobustformofknowledgeaboutuncertainenvironments.
§ Xiscondi+onallyindependentofYgivenZ
ifandonlyif:or,equivalently,ifandonlyif
![Page 10: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/10.jpg)
Condi+onalIndependence
§ Whataboutthisdomain:
§ Traffic§ Umbrella§ Raining
![Page 11: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/11.jpg)
Condi+onalIndependence
§ Whataboutthisdomain:
§ Fire§ Smoke§ Alarm
![Page 12: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/12.jpg)
Condi+onalIndependenceandtheChainRule
§ Chainrule:
§ Trivialdecomposi+on:
§ Withassump+onofcondi+onalindependence:
§ Bayes’nets/graphicalmodelshelpusexpresscondi+onalindependenceassump+ons
![Page 13: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/13.jpg)
GhostbustersChainRule
§ Eachsensordependsonlyonwheretheghostis
§ Thatmeans,thetwosensorsarecondi+onallyindependent,giventheghostposi+on
§ T:TopsquareisredB:Bo?omsquareisredG:Ghostisinthetop
§ Givens:
P(+g)=0.5P(-g)=0.5P(+t|+g)=0.8P(+t|-g)=0.4P(+b|+g)=0.4P(+b|-g)=0.8
P(T,B,G)=P(G)P(T|G)P(B|G)
T B G P(T,B,G)
+t +b +g 0.16
+t +b -g 0.16
+t -b +g 0.24
+t -b -g 0.04
-t +b +g 0.04
-t +b -g 0.24
-t -b +g 0.06
-t -b -g 0.06
![Page 14: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/14.jpg)
Bayes’Nets:BigPicture
![Page 15: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/15.jpg)
Bayes’Nets:BigPicture
§ Twoproblemswithusingfulljointdistribu+ontablesasourprobabilis+cmodels:§ Unlessthereareonlyafewvariables,thejointisWAYtoo
bigtorepresentexplicitly§ Hardtolearn(es+mate)anythingempiricallyaboutmore
thanafewvariablesata+me
§ Bayes’nets:atechniquefordescribingcomplexjointdistribu+ons(models)usingsimple,localdistribu+ons(condi+onalprobabili+es)§ Moreproperlycalledgraphicalmodels§ Wedescribehowvariableslocallyinteract§ Localinterac+onschaintogethertogiveglobal,indirect
interac+ons§ Forabout10min,we’llbevagueabouthowthese
interac+onsarespecified
![Page 16: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/16.jpg)
ExampleBayes’Net:Insurance
![Page 17: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/17.jpg)
ExampleBayes’Net:Car
![Page 18: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/18.jpg)
GraphicalModelNota+on
§ Nodes:variables(withdomains)§ Canbeassigned(observed)orunassigned
(unobserved)
§ Arcs:interac+ons§ Indicate“directinfluence”betweenvariables§ Formally:encodecondi+onalindependence
(morelater)
§ Fornow:imaginethatarrowsmeandirectcausa+on(ingeneral,theydon’t!)
![Page 19: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/19.jpg)
Example:CoinFlips
§ Nindependentcoinflips
§ Nointerac+onsbetweenvariables:absoluteindependence
X1 X2 Xn
![Page 20: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/20.jpg)
Example:Traffic
§ Variables:§ R:Itrains§ T:Thereistraffic
§ Model1:independence
§ Whyisanagentusingmodel2be?er?
R
T
R
T
§ Model2:raincausestraffic
![Page 21: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/21.jpg)
§ Let’sbuildacausalgraphicalmodel!§ Variables
§ T:Traffic§ R:Itrains§ L:Lowpressure§ D:Roofdrips§ B:Ballgame§ C:Cavity
Example:TrafficII
![Page 22: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/22.jpg)
Example:AlarmNetwork§ Variables
§ B:Burglary§ A:Alarmgoesoff§ M:Marycalls§ J:Johncalls§ E:Earthquake!
![Page 23: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/23.jpg)
Bayes’NetSeman+cs
![Page 24: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/24.jpg)
Bayes’NetSeman+cs
§ Asetofnodes,onepervariableX
§ Adirected,acyclicgraph
§ Acondi+onaldistribu+onforeachnode§ Acollec+onofdistribu+onsoverX,oneforeach
combina+onofparents’values
§ CPT:condi+onalprobabilitytable
§ Descrip+onofanoisy“causal”process
A1
X
An
ABayesnet=Topology(graph)+LocalCondi2onalProbabili2es
![Page 25: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/25.jpg)
Probabili+esinBNs
§ Bayes’netsimplicitlyencodejointdistribu+ons
§ Asaproductoflocalcondi+onaldistribu+ons
§ ToseewhatprobabilityaBNgivestoafullassignment,mul+plyalltherelevantcondi+onalstogether:
§ Example:
![Page 26: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/26.jpg)
Probabili+esinBNs
§ Whyareweguaranteedthatseyng
resultsinaproperjointdistribu+on?
§ Chainrule(validforalldistribu+ons):
§ Assumecondi+onalindependences:
àConsequence:
§ NoteveryBNcanrepresenteveryjointdistribu+on§ Thetopologyenforcescertaincondi+onalindependencies
![Page 27: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/27.jpg)
Onlydistribu2onswhosevariablesareabsolutelyindependentcanberepresentedbyaBayes’netwithnoarcs.
Example:CoinFlips
h 0.5
t 0.5
h 0.5
t 0.5
h 0.5
t 0.5
X1 X2 Xn
![Page 28: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/28.jpg)
Example:Traffic
R
T
+r 1/4
-r 3/4
+r +t 3/4
-t 1/4
-r +t 1/2
-t 1/2
![Page 29: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/29.jpg)
Example:Traffic
§ Causaldirec+on
R
T
+r 1/4
-r 3/4
+r +t 3/4
-t 1/4
-r +t 1/2
-t 1/2
+r +t 3/16
+r -t 1/16
-r +t 6/16
-r -t 6/16
![Page 30: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/30.jpg)
Example:ReverseTraffic
§ Reversecausality?
T
R
+t 9/16
-t 7/16
+t +r 1/3
-r 2/3
-t +r 1/7
-r 6/7
+r +t 3/16
+r -t 1/16
-r +t 6/16
-r -t 6/16
![Page 31: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/31.jpg)
Causality?
§ WhenBayes’netsreflectthetruecausalpa?erns:§ O{ensimpler(nodeshavefewerparents)§ O{eneasiertothinkabout§ O{eneasiertoelicitfromexperts
§ BNsneednotactuallybecausal§ Some+mesnocausalnetexistsoverthedomain
(especiallyifvariablesaremissing)§ E.g.considerthevariablesTrafficandDrips§ Endupwitharrowsthatreflectcorrela+on,notcausa+on
§ Whatdothearrowsreallymean?§ Topologymayhappentoencodecausalstructure§ Topologyreallyencodescondi+onalindependence
![Page 32: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/32.jpg)
Example:AlarmNetworkB P(B)
+b 0.001
-b 0.999
E P(E)
+e 0.002
-e 0.998
B E A P(A|B,E)
+b +e +a 0.95
+b +e -a 0.05
+b -e +a 0.94
+b -e -a 0.06
-b +e +a 0.29
-b +e -a 0.71
-b -e +a 0.001
-b -e -a 0.999
A J P(J|A)
+a +j 0.9
+a -j 0.1
-a +j 0.05
-a -j 0.95
A M P(M|A)
+a +m 0.7
+a -m 0.3
-a +m 0.01
-a -m 0.99
B E
A
MJ
![Page 33: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/33.jpg)
Example:AlarmNetworkB P(B)
+b 0.001
-b 0.999
E P(E)
+e 0.002
-e 0.998
B E A P(A|B,E)
+b +e +a 0.95
+b +e -a 0.05
+b -e +a 0.94
+b -e -a 0.06
-b +e +a 0.29
-b +e -a 0.71
-b -e +a 0.001
-b -e -a 0.999
A J P(J|A)
+a +j 0.9
+a -j 0.1
-a +j 0.05
-a -j 0.95
A M P(M|A)
+a +m 0.7
+a -m 0.3
-a +m 0.01
-a -m 0.99
B E
A
MJ
![Page 34: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/34.jpg)
SizeofaBayes’Net
§ Howbigisajointdistribu+onoverNBooleanvariables?
2N
§ HowbigisanN-nodenetifnodeshaveuptokparents?
O(N*2k+1)
§ Bothgiveyouthepowertocalculate
§ BNs:Hugespacesavings!
§ AlsoeasiertoelicitlocalCPTs
§ Alsofastertoanswerqueries(coming)
![Page 35: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/35.jpg)
Bayes’Nets
§ Sofar:howaBayes’netencodesajointdistribu+on
§ Next:howtoanswerqueriesaboutthatdistribu+on§ Today:
§ FirstassembledBNsusinganintui+veno+onofcondi+onalindependenceascausality
§ Thensawthatkeypropertyiscondi+onalindependence§ Maingoal:answerqueriesaboutcondi+onal
independenceandinfluence
§ A{erthat:howtoanswernumericalqueries(inference)
![Page 36: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/36.jpg)
Bayes’Nets
§ Representa+on
§ Condi+onalIndependences
§ Probabilis+cInference
§ LearningBayes’NetsfromData
![Page 37: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/37.jpg)
Condi+onalIndependence
§ XandYareindependentif
§ XandYarecondi+onallyindependentgivenZ
§ (Condi+onal)independenceisapropertyofadistribu+on
§ Example:
![Page 38: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/38.jpg)
BayesNets:Assump+ons
§ Assump+onswearerequiredtomaketodefinetheBayesnetwhengiventhegraph:
§ Beyondabove“chainruleàBayesnet”condi+onalindependenceassump+ons
§ O{enaddi+onalcondi+onalindependences
§ Theycanbereadoffthegraph
§ Importantformodeling:understandassump+onsmadewhenchoosingaBayesnetgraph
P (xi|x1 · · ·xi�1) = P (xi|parents(Xi))
![Page 39: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/39.jpg)
Example
§ Condi+onalindependenceassump+onsdirectlyfromsimplifica+onsinchainrule:
§ Addi+onalimpliedcondi+onalindependenceassump+ons?
X Y Z W
![Page 40: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/40.jpg)
IndependenceinaBN
§ Importantques+onaboutaBN:§ Aretwonodesindependentgivencertainevidence?§ Ifyes,canproveusingalgebra(tediousingeneral)§ Ifno,canprovewithacounterexample§ Example:
§ Ques+on:areXandZnecessarilyindependent?§ Answer:no.Example:lowpressurecausesrain,whichcausestraffic.§ XcaninfluenceZ,ZcaninfluenceX(viaY)§ Addendum:theycouldbeindependent:how?
X Y Z
![Page 41: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/41.jpg)
D-separa+on:Outline
![Page 42: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/42.jpg)
D-separa+on:Outline
§ Studyindependenceproper+esfortriples
§ Analyzecomplexcasesintermsofmembertriples
§ D-separa+on:acondi+on/algorithmforansweringsuchqueries
![Page 43: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/43.jpg)
CausalChains
§ Thisconfigura+onisa“causalchain”
X:LowpressureY:RainZ:Traffic
§ GuaranteedXindependentofZ?No!
§ OneexamplesetofCPTsforwhichXisnotindependentofZissufficienttoshowthisindependenceisnotguaranteed.
§ Example:
§ Lowpressurecausesraincausestraffic,highpressurecausesnoraincausesnotraffic
§ Innumbers:P(+y|+x)=1,P(-y|-x)=1,P(+z|+y)=1,P(-z|-y)=1
![Page 44: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/44.jpg)
CausalChains
§ Thisconfigura+onisa“causalchain” § GuaranteedXindependentofZgivenY?
§ Evidencealongthechain“blocks”theinfluence
Yes!
X:LowpressureY:RainZ:Traffic
![Page 45: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/45.jpg)
CommonCause
§ Thisconfigura+onisa“commoncause” § GuaranteedXindependentofZ?No!
§ OneexamplesetofCPTsforwhichXisnotindependentofZissufficienttoshowthisindependenceisnotguaranteed.
§ Example:
§ Projectduecausesbothforumsbusyandlabfull
§ Innumbers:P(+x|+y)=1,P(-x|-y)=1,P(+z|+y)=1,P(-z|-y)=1
Y:Projectdue
X:Forumsbusy Z:Labfull
![Page 46: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/46.jpg)
CommonCause
§ Thisconfigura+onisa“commoncause” § GuaranteedXandZindependentgivenY?
§ Observingthecauseblocksinfluencebetweeneffects.
Yes!
Y:Projectdue
X:Forumsbusy Z:Labfull
![Page 47: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/47.jpg)
CommonEffect§ Lastconfigura+on:twocausesofone
effect(v-structures)
Z:Traffic
§ AreXandYindependent?§ Yes:theballgameandtheraincausetraffic,but
theyarenotcorrelated
§ S+llneedtoprovetheymustbe(tryit!)
§ AreXandYindependentgivenZ?
§ No:seeingtrafficputstherainandtheballgameincompe++onasexplana+on.
§ Thisisbackwardsfromtheothercases
§ Observinganeffectac+vatesinfluencebetweenpossiblecauses.
X:Raining Y:Ballgame
![Page 48: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/48.jpg)
TheGeneralCase
![Page 49: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/49.jpg)
TheGeneralCase
§ Generalques+on:inagivenBN,aretwovariablesindependent(givenevidence)?
§ Solu+on:analyzethegraph
§ Anycomplexexamplecanbebrokenintorepe++onsofthethreecanonicalcases
![Page 50: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/50.jpg)
Reachability
§ Recipe:shadeevidencenodes,lookforpathsintheresul+nggraph
§ A?empt1:iftwonodesareconnectedbyanundirectedpathnotblockedbyashadednode,theyarecondi+onallyindependent
§ Almostworks,butnotquite§ Wheredoesitbreak?§ Answer:thev-structureatTdoesn’tcount
asalinkinapathunless“ac+ve”
R
T
B
D
L
![Page 51: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/51.jpg)
Ac+ve/Inac+vePaths
§ Ques+on:AreXandYcondi+onallyindependentgivenevidencevariables{Z}?§ Yes,ifXandY“d-separated”byZ§ Considerall(undirected)pathsfromXtoY
§ Noac+vepaths=independence!
§ Apathisac+veifeachtripleisac+ve:§ CausalchainA→B→CwhereBisunobserved(eitherdirec+on)§ CommoncauseA←B→CwhereBisunobserved§ Commoneffect(akav-structure)
A→B←CwhereBoroneofitsdescendentsisobserved
§ Allittakestoblockapathisasingleinac+vesegment
Ac+veTriples Inac+veTriples
![Page 52: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/52.jpg)
§ Query:
§ Checkall(undirected!)pathsbetweenand§ Ifoneormoreac+ve,thenindependencenotguaranteed
§ Otherwise(i.e.ifallpathsareinac+ve),thenindependenceisguaranteed
D-Separa+on
Xi �� Xj |{Xk1 , ..., Xkn}
Xi �� Xj |{Xk1 , ..., Xkn}
?
Xi �� Xj |{Xk1 , ..., Xkn}
![Page 53: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/53.jpg)
Example
Yes R
T
B
T’
![Page 54: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/54.jpg)
Example
R
T
B
D
L
T’
Yes
Yes
Yes
![Page 55: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/55.jpg)
Example
§ Variables:§ R:Raining§ T:Traffic§ D:Roofdrips§ S:I’msad
§ Ques+ons:T
S
D
R
Yes
![Page 56: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/56.jpg)
StructureImplica+ons
§ GivenaBayesnetstructure,canrund-separa+onalgorithmtobuildacompletelistofcondi+onalindependencesthatarenecessarilytrueoftheform
§ Thislistdeterminesthesetofprobabilitydistribu+onsthatcanberepresented
Xi �� Xj |{Xk1 , ..., Xkn}
![Page 57: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/57.jpg)
Compu+ngAllIndependences
X
Y
Z
X
Y
Z
X
Y
Z
X
Y
Z
![Page 58: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/58.jpg)
XY
Z
{X �� Y,X �� Z, Y �� Z,
X �� Z | Y,X �� Y | Z, Y �� Z | X}
TopologyLimitsDistribu+ons
§ GivensomegraphtopologyG,onlycertainjointdistribu+onscanbeencoded
§ Thegraphstructureguaranteescertain(condi+onal)independences
§ (Theremightbemoreindependence)
§ Addingarcsincreasesthesetofdistribu+ons,buthasseveralcosts
§ Fullcondi+oningcanencodeanydistribu+on
X
Y
Z
X
Y
Z
X
Y
Z
{X �� Z | Y }
X
Y
Z X
Y
Z X
Y
Z
X
Y
Z X
Y
Z X
Y
Z
{}
![Page 59: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/59.jpg)
BayesNetsRepresenta+onSummary
§ Bayesnetscompactlyencodejointdistribu+ons
§ Guaranteedindependenciesofdistribu+onscanbededucedfromBNgraphstructure
§ D-separa+ongivesprecisecondi+onalindependenceguaranteesfromgraphalone
§ ABayes’net’sjointdistribu+onmayhavefurther(condi+onal)independencethatisnotdetectableun+lyouinspectitsspecificdistribu+on
![Page 60: Bayes’ Nets - University of Washington · § When Bayes’ nets reflect the true causal paerns: § O{en simpler (nodes have fewer parents) § O{en easier to think about § O{en](https://reader033.fdocuments.net/reader033/viewer/2022050212/5f5ea031a910622bc44ec03b/html5/thumbnails/60.jpg)
Bayes’Nets
§ Representa+on§ Condi+onalIndependences§ Probabilis+cInference
§ Enumera+on(exact,exponen+alcomplexity)§ Variableelimina+on(exact,worst-case
exponen+alcomplexity,o{enbe?er)§ Probabilis+cinferenceisNP-complete§ Sampling(approximate)
§ LearningBayes’NetsfromData