Predicate Calculus Syntax

• Countable set of predicate symbols, each with specified arity 0. For example: clinical data with multiple tables of patient information. Each such table can be a predicate and each record (tuple) is an atomic formula.

Table – Patient

patient(1,3-3-03,f) – a predicate with arity = 3.

ID DOB Gender

1 3-3-03 f

-- -- --

-- -- --

Further Notation…. • Variables – start with upper-case letter• Predicate/atomic symbols – start with lower-case letter• Constant – start with lower-case letter• Logical connectives: ,,,,,• We will also use negation by failure (please see

Skolemization slides)• Limiting ourselves to atomic formulas and (if -then)

rules (clauses)• Learn Non-recursive rules – predicate in consequent

cannot appear in antecedent• Can have recursive rules elsewhere (background

knowledge)

Examples…

1. p(a,b,c)^p(d,e,f) true if both atomic formulas are true

2. p(a,b,c) p(d,e,f) true if first atomic formula is false and/or second atomic formula is true

3. XYZ (q(X,Y)^q(Y,Z) p(X,Z)) non-recursive rule

4. If a and b are formulas then so are a, ab, ab, ab, ab

5. If X is a variable and a is a formula then Xa and Xa are formulas. We say that X is quantified in the formulas Xa and Xa.

Some Notes

• Predicates of arity 0 are also called propositions, the only atomic formulas allowed in propositional logic

• An expression is an atomic formula

• A sentence is any formula in which all variables are quantified

Skolemization

• Process is applied to one sentence at a time and applied only to the entire sentence (so outermost quantifier first). Each sentence initially has empty vector of free variables.

• Replace X A(X) with A(X), and add X to vector of free variables.

• Replace X A(X) with A(x(V)) where x is a new function symbol and V is the current vector of free variables.

• If no function symbols, don’t need second bullet, and x is just a constant (nullary function) in third bullet

Learning …

To learn, summarizing all information (eg. clinical data) and coming up with 1000-10,000 features could be one approach; however, we lose information in this process. Another option is a join, but the frequency of subjects in database alters the algorithm results.

Inductive logic programming (ILP) can handle these issues by using predicate calculus to represent the database, and by learning rules.

Inductive Logic Programming: The Problem Specification

• Given:– Examples: first-order atomic formulas (atoms),

each labeled positive or negative.

– Background knowledge: definite clause (if-then rules) theory.

– Language bias: constraints on the form of interesting new rules (clauses).

ILP Specification (Continued)

• Find:– A hypothesis h that meets the language constraints

and that, when conjoined with B, implies (lets us prove) all of the positive examples but none of the negative examples.

• To handle real-world issues such as noise, we often relax the requirements, so that h need only entail significantly more positive examples than negative examples.

A Common Approach

• Use a greedy covering algorithm.– Repeat while some positive examples remain

uncovered (not entailed):• Find a good clause (one that covers as many positive

examples as possible but no/few negatives).

• Add that clause to the current theory, and remove the positive examples that it covers.

• ILP algorithms use this approach but vary in their method for finding a good clause.

Scoring function

MDL – minimum description lengthScore(R) = PositiveR - NegativeR -A (# atomic formulas)

FOIL – uses information gain as scoring function. Example Label predicate+ mother(john,sue)+ mother(jane,sue)+ mother(joe,jane)- mother(sue,jane)- mother(joe,john)- mother(fred,jane)- mother(jane,fred)

Background knowledge

parent(john, fred)parent(john,sue)parent(jane,fred)parent(joel,jane)parent(joel,jack)parent(jane,sue)male(john)male(jack)male(fred)male(joel)female(sue)female(jane)

We want to learn predicate ‘mother’ with scoring function S_MDL = S(P,N,A) = P – N - A

mother(X,Y) P N A S 3 4 1 -2 meaning S(3,4,2)=-2

mother(X,Y)parent(X,Y) mother(X,Y)female(X) mother(X,Y)female(Y)P N A S P N A S P N A S3 1 2 0 1 2 2 -3 3 2 2-1

Lattice can grow further by including substitution of variables but generalization is limited in this case and can be misleading.mother(X,sue)P N A S2 0 1 1

Now see if we can cover all +examples combination of clauses

Building Bottom clause (refinement)

Start substituting variables in predicate to generate all ‘+ examples’ and write down atomic formulas from background knowledge which explains the predicate.

For example: mother(X,Y)X = john, Y = sueparent(X,Y)^ male(X)^female(Y)parent(X,Z), Z = fred etc

Generate lattice to get these bottom clause for each ‘+ example’. Each example will provide a sub-lattice. Just search the sub-lattice above this bottom clause (only use atoms in this bottom clause).