Probabilistic Resolution. Logical reasoning Absolute implications office meeting office talk office...
-
Upload
kaylee-mcginnis -
Category
Documents
-
view
215 -
download
0
Transcript of Probabilistic Resolution. Logical reasoning Absolute implications office meeting office talk office...
Probabilistic Resolution
Logical reasoning
• Absolute implications
• office meeting
• office talk
• office pick_book
• But what if my rules are not absolute?
Migrating to Probabilities:Graphical Models
noisy_office
meeting
talk
pick_bookmeeting talk pick_book P(noisy_office| m,p,t)
T T T 0.9992T T F 0.998T F T 0.996T F F 0.99F T T 0.92F T F 0.8F F T 0.6F F F 0
Actually, the original model does not justify the
last row
Migrating to Probabilities:Graphical Models
noisy_office meeting
talk
pick_book
meeting talk pick_book f(noisy_office = T)T T T 0.9992T T F 0.998T F T 0.996T F F 0.99F T T 0.92F T F 0.8F F T 0.6F F F 0
Variable Elimination (VE)
noisy_office meeting
talk
pick_book
Variable Elimination (VE)
noisy_office
talk
pick_book
meeting(noisy_office, pick_book, talk, meeting) (meeting)
meeting
Variable Elimination (VE)
noisy_office
talk
pick_book
Variable Elimination (VE)
noisy_office
pick_book
Variable Elimination (VE)
noisy_office
meeting talk pick_book f(office = T)T T T 1T T F 1T F T 1T F F 1F T T 1F T F 1F F T 1F F F 0
Graphical Models generalize Logic
office meeting
talk
pick_book
VE generalizes Resolution
Resolution
A or BB or C
A or C
A B C
A C
Variable Elimination
There is still an important difference, though.
Story so far
• Logic uses absolute rules;
• Probabilistic models can deal with noise, and generalize logic;
• But...
Logical reasoning “ends early”
• office meeting
• office talk
• office pick_book
• ...
• Given evidence ‘meeting’, we are done after considering first rule alone.
Ending early in deterministic graphical model
Variable Elimination uses all nodes to calculate P(office | meeting)
office meeting
talk
pick_book
meeting talk pick_book f(office = T)T T T 1T T F 1T F T 1T F F 1F T T 1F T F 1F F T 1F F F 0
Ending early in deterministic graphical model
But if ‘meeting’ is observed, we don’t need to look beyond it
office
talk
pick_book
talk pick_book f(office = T)T T 1T F 1F T 1F F 1
Ending early in deterministic graphical model
We can use “smarter” algorithms to end early here as well
office
talk
pick_book
talk pick_book f(office = T)T T 1T F 1F T 1F F 1
Ending early in non-deterministic graphical models
Calculating P(noisy_office | meeting)
noisy_office meeting
talk
pick_book
meeting talk pick_book f(noisy_office = T)T T T 0.9992T T F 0.998T F T 0.996T F F 0.99F T T 0.92F T F 0.8F F T 0.6F F F 0
Ending early in non-deterministic graphical models
P(noisy_office | meeting) depends on all nodes
noisy_office
talk
pick_book
talk pick_book f(noisy_office = T)T T 0.9992T F 0.998F T 0.996F F 0.99
Ending early in non-deterministic graphical models
noisy_office
talk
pick_book
talk pick_book f(noisy_office = T)T T 0.9992T F 0.998F T 0.996F F 0.99
But we already know P(noisy_office | meeting) [0.99, 0.9992]Can we take advantage of this?
Goal
• A graphical model inference algorithm that derives a bound on solution so far;
• Ends as soon as bound is “good enough”;
• An anytime algorithm.
Probabilistic Resolution
Resolution
A or BB or C
A or C
A B C
A C
Variable Elimination
•Variable Elimination generalizes Resolution, but neither provides intermediate results nor ends early.
•Probabilistic Resolution = VE + “ending early”
Story so far
• Logic uses absolute rules;
• Probabilistic models can deal with noise, and generalize logic;
• Logic ends as soon as possible, graphical models do not;
• They can if we are willing to use bounds;
• But how to calculate bounds?
But how to get bounds?
Q N2
N1
N4
N3
...
...
......
But how to get bounds?
Q N2
N1
N4
N3
...
...
......
But how to get bounds?
Q N2
N1
N4
N3
But how to get bounds?
Q N
But how to get bounds?
Q Nf1 f2
P(Q) N f1(Q,N) f2(N)
P(Q) N f1(Q,N) P2(N)
P(Q) f ( P2(N) )
But how to get bounds?
Q N
P(Q) f ( P2(N) )
0 1 0 1
f
P(Q) P2(N)
But how to get bounds?
Q N
P(Q) P2(N)
(0,0,1)
(1,0,0)
(0,1,0)
(0,0,1)
(1,0,0)
(0,1,0)
f
P(Q) f ( P2(N) )
But how to get bounds?
Q N
P(Q) P2(N)
(0,0,1)
(1,0,0)
(0,1,0)
(0,0,1)
(1,0,0)
(0,1,0) f
P(Q) f ( P2(N) )
bound
Infinite number of points!
Justify inner shape to be equal to outter one
But how to get bounds?
Q N
P(Q) P2(N)
(0,0,1)
(1,0,0)
(0,1,0)
(0,0,1)
(1,0,0)
(0,1,0)
f
P(Q) f ( P2(N) )Vertices are
enough
But how to get bounds?
Q N
P(Q)
(0,0,1)
(1,0,0)
(0,1,0)
(0,0,1)
(1,0,0)
(0,1,0)
f
P(Q) f ( P2(N) )
P2(N)
No necessary correspondence
But how to get bounds?
Q N
(0,0,1)
(1,0,0)
(0,1,0)
f
P(Q) f ( P2(N) )
P2(N)0 1P(Q)
Correspondence would be
impossible in this case
Make slide with opposite: segment to
triangle
But how to get bounds?
Q N
0 1 0 1
f
P(Q)
P(Q) f ( P2(N) )
P2(N)
Example I
Q N
Q N f(Q,N)1 1 0.61 0 0.40 1 0.30 0 0.7
[0,1][0.36, 0.67]
P(Q) f ( P2(N) )
P(Q) N f(Q,N) P2(N)
P(Q) f(Q,0)P2(N=0) + f(Q,1)P2(N=1)
For P2(N=0) = 1:
P(Q) f(Q,0) 1 + f(Q,1) 0
P(Q) f(Q,0)
P(Q=1) = f(1,0) / (f(0,0) + f(1,0))
P(Q=1) = 0.4 / (0.7 + 0.4) = 0.36
For P2(N=1) = 1:
P(Q) f(Q,0) 0 + f(Q,1) 1
P(Q) f(Q,1)
P(Q=1) = f(1,1) / (f(0,1) + f(1,1))
P(Q=1) = 0.6 / (0.3 + 0.6) = 0.67
P2(N)
Example II
Q N
[0,1][0.5]
Q N f(Q,N)1 1 11 0 10 1 10 0 1
P(Q)
(0,0,1)
(1,0,0)
(0,1,0)
(0,0,1)
(0,1,0)
f
(1,0,0)
0 1 0 1
f
P(Q) P2(N)
Example III
Q N
[0,1][0,1]
Q N f(Q,N)1 1 11 0 00 1 00 0 1
P2(N)P(Q)
(0,0,1)
(1,0,0)
(0,1,0)
(0,0,1)
(0,1,0)
f
(1,0,0)
0 1 0 1
f
P(Q) P2(N)
Example IV
noisy_office meeting
talk
pick_book
meeting talk pick_book f(noisy_office = T)T T T 0.9992T T F 0.998T F T 0.996T F F 0.99F T T 0.92F T F 0.8F F T 0.6F F F 0
Example IV
noisy_office
talk
pick_book
talk pick_book f(noisy_office = T)T T 0.9992T F 0.998F T 0.996F F 0.99
Example IV
noisy_office
talk
pick_book
talk pick_book f(noisy_office = T)T T 0.9992T F 0.998F T 0.996F F 0.99
0.4
Example IV
noisy_office
pick_book
pick_book f(noisy_office = T)T 0.9976F 0.994
Example IV
noisy_office
pick_book
pick_book f(noisy_office = T)T 0.9976F 0.994 1
Example IV
noisy_office
f(noisy_office = T)0.9976
Algorithm
• Same as Variable Elimination, but update bounds every time a neighbor is eliminated;
• Bounds always improve at each neighbor elimination;
• Trade-off between granularity of bound updates (explain granularity) and ordering efficiency.
Complexity Issues
• Calculating bound is exponential on the size of neighborhood component, so complexity is exponential on largest neighborhood component during execution;
• This can be larger than tree-width;
• But finding tree-width is hard anyway.
Preliminary Tests
0
0.2
0.4
0.6
0.8
1
1.2
0 0 1 2 2 4 8 9 92 102 105 105 108 125
% of exact computation time (not in scale)
Bo
un
d i
nte
rval
wid
th strokevolume=high
hrekg=low
pcwp=normal
Conclusions
• Making Probabilistic Inference more like Logic Inference;
• Getting an anytime algorithm in the process;
• Preparing ground for First-order Probabilistic Resolution.