Inductive Logic Programming: The Problem Specification
description
Transcript of Inductive Logic Programming: The Problem Specification
![Page 1: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/1.jpg)
Inductive Logic Programming: The Problem Specification
• Given:– Examples: first-order atoms or definite clauses,
each labeled positive or negative.– Background knowledge: in the form of a
definite clause theory.– Language bias: constraints on the form of
interesting new clauses.
![Page 2: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/2.jpg)
ILP Specification (Continued)
• Find:– A hypothesis h that meets the language constraints
and that, when conjoined with B, entails (implies) all of the positive examples but none of the negative examples.
• To handle real-world issues such as noise, we often relax the requirements, so that h need only entail significantly more positive examples than negative examples.
![Page 3: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/3.jpg)
A Common Approach
• Use a greedy covering algorithm.– Repeat while some positive examples remain
uncovered (not entailed):• Find a good clause (one that covers as many positive
examples as possible but no/few negatives).• Add that clause to the current theory, and remove the
positive examples that it covers.
• ILP algorithms use this approach but vary in their method for finding a good clause.
![Page 4: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/4.jpg)
A Difficulty
• Problem: It is undecidable in general whether one definite clause implies another, or whether a definite clause together with a logical theory implies a ground atom.
• Approach: Use subsumption rather than implication.
![Page 5: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/5.jpg)
Subsumption for Literals
b).p(f(a),notbut a)p(f(a), subsumes X)p(f(X), :Example
.L Lsuch that on substituti aexists thereifonly and if L subsumes L Literal
21
21
![Page 6: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/6.jpg)
Subsumption for Clauses
c}.{Xon substituti theusingb)p(Y, c)p(a, subsumes X)p(a, W}.ZW,Y W,{X on substituti theusing
W),p(W, subsumes Z)p(Y, Y)p(X, :Examples
literals). its ofset theas viewedis clause a(where C Csuch that on substituti a exists
thereifonly and if C clause subsumes C Clause21
21
![Page 7: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/7.jpg)
![Page 8: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/8.jpg)
Least Generalization of Terms
).(return ,)).lgg(),...,(lgg(Return terms.are and and 0 is f ofarity thewhere
),( and )( then f, symbolfunctionprimary same thefrombuilt are tand t
or in appear not do that variablesto termsof pairs (ordered) frombijection a be φLet
).,lgg(tion generalizaleast :. and termsTwo :
21
11
11
1211
21
21
21
21
,ttφ,su,suf
,...,ss,...,uun,...,ssft,...,uuft
.tt
tttt
nn
nn
nn
Otherwise
If
OutputInput
![Page 9: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/9.jpg)
Least Generalization of Terms (Continued)
• Examples:– lgg(a,a) = a– lgg(X,a) = Y– lgg(f(a,b),g(a)) = Z– lgg(f(a,g(a)),f(b,g(b))) = f(X,g(X))
• lgg(t1,t2,t3) = lgg(t1,lgg(t2,t3)) = lgg(lgg(t1,t2),t3): justifies finding the lgg of a set of terms using the pairwise algorithm.
![Page 10: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/10.jpg)
Least Generalization of Literals
.)lgg(),...,lgg(Return . form thehas and
form thehas Otherwise, TOP.return then unnegated)other theand negated is (one signs
differentor predicatesdifferent have and If).,lgg(tion generalizaLeast :
. and Literals :
11
121
1
21
21
21
),su,sup(),...,sp(sL),...,up(u
L
LLLL
LL
nn
nn
OutputInput
![Page 11: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/11.jpg)
Lattice of Literals
• Consider the following partially ordered set.• Each member of the set is an equivalence class of
literals, equivalent under variance.• One member of the set is greater than another if
and only if one member of the first set subsumes one member of the second (can be shown equivalent to saying: if and only if every member of the first set subsumes every member of the second).
![Page 12: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/12.jpg)
Lattice of Literals (Continued)
• For simplicity, we now will identify each equivalence class with one (arbitrary) representative literal.
• Add elements TOP and BOTTOM to this set, where TOP is greater than every literal, and every literal is greater than BOTTOM.
• Every pair of literals has a least upper bound, which is their lgg.
![Page 13: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/13.jpg)
Lattice of Literals (Continued)
• Every pair of literals has a greatest lower bound, which is their greatest common instance (the result of applying their most general unifier to either literal, or BOTTOM if no most general unifier exists.)
• Therefore, this partially ordered set satisfies the definition of a lattice.
![Page 14: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/14.jpg)
![Page 15: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/15.jpg)
![Page 16: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/16.jpg)
![Page 17: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/17.jpg)
Least Generalization of Clauses
clause. resulting theReturn ).C,lgg(C of literal a as lgg thisadd
thenTOPnot is lgg their if ,C from one and Cfrom one literals, ofpair every For set.empty
the to)C,lgg(Cin literals ofset theInitialize).C,lgg(Ction generalizaLeast
l...lCand l...lC clauses, Two :
21
21
21
21
m.2,2,12
n1,1,11
:Output
Input
![Page 18: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/18.jpg)
Example
T he lg g o f the fo llo w in g tw o c lau ses and
is:
p a f a p b b p b f bp f a f a p f a b p a f a
p X f a p X Y p Z Z p Z b p U f U
( , ( )) ( , ) ~ ( , ( ))( ( ), ( )) ( ( ), ) ~ ( , ( ))
( , ( )) ( , ) ( , ) ( , ) ~ ( , ( ))
![Page 19: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/19.jpg)
Lattice of Clauses
• We can construct a lattice of clauses in a manner analogous to our construction of literals.
• Again, the ordering is subsumption; again we group clauses into variants; and again we add TOP and BOTTOM elements.
• Again the least upper bound is the lgg, but the greatest lower bound is just the union (clause containing all literals from each).
![Page 20: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/20.jpg)
Lattice of Clauses for the Given Hypothesis Language
active(X)
active(X) :- active(X) :- active(X) :- has-hydrophobic(X,A) has-donor(X,A) has-acceptor(X,A)
active(X) :- active(X) :- active(X) :- has-hydrophobic(X,A), has-donor(X,A), has-acceptor(X,A), has-donor(X,B), has-donor(X,B), has-donor(X,B), distance(X,A,B,5.0) distance(X,A,B,4.0) distance(X,A,B,6.0) . . .
![Page 21: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/21.jpg)
Incorporating Background Knowledge: Saturation
• Recall that we wish to find a hypothesis clause h that together with the background knowledge B will entail the positive examples but not the negative examples.
• Consider an arbitrary positive example e. Our hypothesis h together with B should entail e: Bh ⊨ e. We can also write this as h ⊨ B e.
![Page 22: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/22.jpg)
Saturation (Continued)
• If e is an atom (atomic formula), and we only use atoms from B, then B e is a definite clause.
• We call B e the saturation of e with respect to B.
![Page 23: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/23.jpg)
Saturation (Continued)
• Recall that we approximate entailment by subsumption.
• Our hypothesis h must be in that part of the lattice of clauses above (subsuming) B e.
![Page 24: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/24.jpg)
Alternative Derivation of Saturation
• From Bh ⊨ e by contraposition: B {e} ⊨ h.
• Again by contraposition: h ⊨ (B e)• So by DeMorgan’s Law: h ⊨ B e• If e is an atom (atomic formula), and we
only use atoms from B, then B e is a definite clause.
![Page 25: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/25.jpg)
![Page 26: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/26.jpg)
![Page 27: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/27.jpg)
![Page 28: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/28.jpg)
![Page 29: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/29.jpg)
![Page 30: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/30.jpg)
![Page 31: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/31.jpg)
![Page 32: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/32.jpg)
Overview of Some ILP Algorithms
• GOLEM (bottom-up): saturates every positive example and then repeatedly takes lggs as long as the result does not cover a negative example.
• PROGOL, ALEPH (top-down): saturates first uncovered positive example, and then performs top-down admissible search of the lattice above this saturated example.
![Page 33: Inductive Logic Programming: The Problem Specification](https://reader035.fdocuments.net/reader035/viewer/2022062305/56815d4a550346895dcb51c4/html5/thumbnails/33.jpg)
Algorithms (Continued)
• FOIL (top-down): performs greedy top-down search of the lattice of clauses (does not use saturation).
• LINUS/DINUS: strictly limit the representation language, convert the task to propositional logic, and use a propositional (single-table) learning algorithm.