Opmaker2: Efficient Action Schema Acquisition T.L.McCluskey, S.N.Cresswell, N. E. Richardson and...
-
Upload
jermaine-cashmore -
Category
Documents
-
view
217 -
download
0
Transcript of Opmaker2: Efficient Action Schema Acquisition T.L.McCluskey, S.N.Cresswell, N. E. Richardson and...
Opmaker2: Efficient Action Schema Acquisition
T.L.McCluskey, S.N.Cresswell, N. E. Richardson and M.M.West
The University of Huddersfield,UK
Email [email protected]
Knowledge Engineering and Intelligent Interfaces
Motivation – Intelligent Agents that can Plan?
The availability of knowledge via ontologies is growing it is not unreasonable that an agent will be able to acquire and refine
such knowledge with some degree of autonomy. A necessary precondition of the use of (current) automated planning
technology is that there exists detailed knowledge: action specifications and heuristics
the amount of effort needed to encode bug free, accurate action specifications and planning heuristics, and to maintain them, is significant.
for every agent that can perform planning, must we hand code and hand maintain its action descriptions and heuristics? No, if agents are to achieve this kind of
autonomy, then they should be capable of learning and refining action knowledge and heuristics.
Knowledge Engineering and Intelligent Interfaces
Vision
Agent
ApplicationOntologies
PlanEngines
Learningphase
Agent capable of deliberative planning
Knowledge Engineering and Intelligent Interfaces
Planning and Machine Learning – moved on?
Eg my work in 1987 (IJCAI-87, IWML-87) deriving planning heuristics (macros, operator choice, goal-orders) offline
YES – must derive heuristics + dynamic domain model
Knowledge Engineering and Intelligent Interfaces
General Idea of Opmaker2 A knowledge base for planning can be divided into 3:- the easy (static) part of a domain model- The (hard) dynamic part- Planning heuristics
The idea of Opmaker is to learn the (hard) dynamic part and the planning heuristics from “training sequences” within the context of an existing (static) part of a domain model
Knowledge Engineering and Intelligent Interfaces
Learning Problem - INPUT Input:
(a) knowledge of objects, and collections of similar objects making up distinct classes, possible states of a typical object of each class, and state invariants
(b) Knowledge of existing plans that other agents, or a trainer, has used [in terms of verbs and affected objects] with initial state and goal state for affected objects
the agent does not have an explicit specification of actions in such a way that it can reason about their synthesis (or the agent does have such a specification but needs to refine, maintain or evolve it).
Knowledge Engineering and Intelligent Interfaces
Example of (b)Example training sequence from “Extended Tyre Domain”1. name: do_up; unchanging: wrench0,jack0, trim1; changing: hub1,nuts1
2. name: jack_down; unchanging:: nilchanging: hub1,jack0
3. name: tighten; unchanging: wrench0,hub1,trim1; changing: nuts1
4. name: apply_trim; unchanging: hub1; changing: trim1,wheel5
Knowledge Engineering and Intelligent Interfaces
Learning Problem - OUTPUT
1. A full parameterised specification of actions which can be used to do planning
2. Planning heuristics.
3. Refinement of any existing parameterised specification of actions, and heuristics, that it currently holds.
Results:
Opmaker2 outputs 1, plus 2 in the form of HTN methods. Trying to avoid 3.
Knowledge Engineering and Intelligent Interfaces
Opmaker vs Opmaker2
Opmaker2INPUT:Partial (static) Domain TheoryTraining sequences (.. extra Invariants)
OUTPUT:Primitive Operator SchemaHTN Schema
Opmaker (in GIPO)INPUT:Partial (static) Domain TheoryTraining sequencesIntermediate State Details
OUTPUT:Primitive Operator Schema
But hasn’t this been done before – by Opmaker(1) (2002) for example?
Painful part!
Knowledge Engineering and Intelligent Interfaces
Method Overview
(1).use a set of heuristics and inferences to track the changing states (‘object trace’) of each changing object referred to within a training example; produce a set of candidate operators
(2). use domain invariants to reduce the set of candidate operators (to a singleton)
(3). use the techniques of the original Opmaker algorithm to generalise object references and create parameterised operator schema from the specific object transitions extracted in (i) from the training examples.
Knowledge Engineering and Intelligent Interfaces
Method Overview for (1)
Track an object O in a
training sequence
Id1,Id2,…,Idk,…, Idn:
Where Idk includes the
last mention of
changing O
O.i
O.f
Method seeks to fill in state transitiondetails..
KNOWN
UNKNOWN
Id1
Idk
Knowledge Engineering and Intelligent Interfaces
Method Overview for (1)
Track an object O in a
training sequence
Id1,Id2,…,Idk,…, Idn:
Where Idk includes the
last mention of
changing O
O.i
O.f
..by finding theObject’s trace includingany associations itmakes
Idk
Id1
Knowledge Engineering and Intelligent Interfaces
Method Overview for (1) Considering a changing object O at training action with id
‘Idx’ .. If Idx is not the last change of O then we potentially have a choice of target state for O …
O current state = O.c Idx
Same stateState that has parameter of object type NOT in Idx
Potential new states
Knowledge Engineering and Intelligent Interfaces
Back to Example id1: do_up; unchanging: wrench0,jack0, trim1; changing: hub1,nuts1 id2: jack_down; unchanging:: nilchanging: hub1,jack0 id3: tighten; unchanging: wrench0,hub1,trim1; changing: nuts1 Id4: apply_trim; unchanging: hub1; changing: trim1,wheel5
Knowledge Engineering and Intelligent Interfaces
Method Overview for (1) - example Considering a changing object hub1 at training action ‘do_up’ ..
If do_up is not the last change of hub1 then we potentially have a choice of target state for hub1 …
hub1 current state = unfastened(hub1), jacked_up(hub1,jack0)
do_up
Same state
Potential new states (see PAPER for definitions)
S1 S2 S3 S4
Knowledge Engineering and Intelligent Interfaces
Method Overview – result of step (1) Form sets of candidate operators for each Id, of form
- Id - x- Set of parameters- Set of prevail conditions (unchanging objects)- Set of object transitions
Knowledge Engineering and Intelligent Interfaces
Method Overview – step (2)
use domain invariants to reduce the set of candidate operators (to a singleton)
Eliminate a candidate that is logically consistent with domain invariants
Knowledge Engineering and Intelligent Interfaces
Method Overview – step (3)
use the techniques of the original Opmaker algorithm to generalise object references and create parameterised operator schema from the specific object transitions extracted in (i) from the training examples.
Generalise those left using OpMaker to create operator schema
Knowledge Engineering and Intelligent Interfaces
Producing Heuristics- Our earlier paper (2002) on Opmaker indicated how the
training sequences can be made into ‘canned plans’ and used as components in planning using the Hybrid planner HyHTN
- Even earlier work (1997) showed how invariants are key to producing goal ordering heuristics
- In Opmaker2 we have implemented the former method and the canned plans are produced with the primitive operator schema
Knowledge Engineering and Intelligent Interfaces
Experiments 3 Domains considered : Extended Tyre Domain, Hiking Domain, Blocks DomainEnough training sequences to include instances of all required
operator schemaExperimental Criteria:- A. ‘How many’ axioms have to be added to the static
domain model to make sure the operator set is unique?- B. Are the operator sets induced capable of being used in
solving planning problems?- C. With the canned plans introduced, is plan generation
time reduced?- D. Does the performance of the learning program vary a
great deal with different training sequences
Knowledge Engineering and Intelligent Interfaces
Experimental Results
A. ‘How many’ axioms have to be added to the static domain model to make sure the operator set is unique?
- Tyre Domain - 8, Hiking Domain - 0, Blocks Domain - 4B. Are the operator sets induced capable of being used in solving planning
problems? Yes, in all 3 casesC. With the canned plans introduced, is plan generation time reduced?- Tyre Domain, Hiking Domain, Blocks Domain Yes, in all 3 cases (no graphs to show as yet..)D. Still much to do BUT LONG training sequences do not seem to cause a
problem – we tried a 22 element solution in the Blocks World and it produced a single set of operator schema
BUT result with blocks domain threw up some problems because of the multiple object references of the same class within one operator schema
Knowledge Engineering and Intelligent Interfaces
Conclusions and Future Work Essentially, we have traded off effort to produce a detailed
“static” domain model vs effort to produce operator schema modelling actions because
(a) Static knowledge is more easily available
(b) Our experience of operator building in GIPO II showed that creating the parameterised specs to be the hardest part of domain encoding
Open Question for future work:
How to induce operator schema which require constraints mot appearing in the input object state definitions
How to guarantee that a set of invariants is strong enough to deliver a working set of operator schema?