atai17 4 conditional planning-1 · *hudog 6whlqedxhu,qvwlwxwh iru 6riwzduh 7hfkqrorj\ $gydqfhg...
Transcript of atai17 4 conditional planning-1 · *hudog 6whlqedxhu,qvwlwxwh iru 6riwzduh 7hfkqrorj\ $gydqfhg...
1
Gerald Steinbauer
Institute for Software Technology
1
Advanced Topics of AI – Conditional Planning
Gerald Steinbauer
Institut für Softwaretechnologie
Inffeldgasse 16b/2
A-8010 Graz
Austria
Advanced Topics of Artificial Intelligence- Conditional Planning -
Gerald Steinbauer
Institute for Software Technology
2
Advanced Topics of AI – Conditional Planning
Motivation• so far: fully observable, static and deterministic domains
• agent can plan first and then execute plan with eyes closed
• uncertain environment: • incomplete (partially observable and/or nondeterministic)• incorrect (differences between world and model) information• use sensing• adapt plan when necessary
• degree of uncertainty defined by indeterminacy• bounded: actions can have unpredictable effects, yet can be listed in
action description axioms• unbounded: preconditions and effects unknown or too large to
enumerate
Gerald Steinbauer
Institute for Software Technology
3
Advanced Topics of AI – Conditional Planning
Handling indeterminacy
• sensorless planning (conformant planning)• find plan that achieves goal in all possible circumstances (regardless
of initial state and action effects)
• conditional planning (contingency planning)• Construct conditional plan with different branches for possible
contingencies
• execution monitoring and replanning• while executing plan judge whether plan requires revision
• continuous planning • planning active for a life time: adapt to changed circumstances and
reformulate goals if necessary
Gerald Steinbauer
Institute for Software Technology
4
Advanced Topics of AI – Conditional Planning
Abstract example• initial state = <chair, table, cans of paint, unknown colors>
goal state = <color(table) = color(chair)>
• sensorless planning (conformant planning)• open any can of paint and apply it to both chair and table
• conditional planning (contingency planning)• sense color of table and chair, if they are the same then finish;
else sense label’s paint; if color(label) = color(Furniture) then apply color to other piece; else apply color to both
• execution monitoring and replanning• same as conditional and can fix errors (missed spots)
• continuous planning • can revise goal when we want to first eat before painting the table and the chair
2
Gerald Steinbauer
Institute for Software Technology
5
Advanced Topics of AI – Conditional Planning
Vacuum Cleaner
3 state variables (at_r, at_l, clean_r, clean_l)
3 actions (right, left, suck)
Gerald Steinbauer
Institute for Software Technology
6
Advanced Topics of AI – Conditional Planning
Murphy Vacuum Cleaner
• single murphy vacuum cleaner• non-deterministically the cleaner dumps dust on the location it
moves to if location is clean
• double murphy vacuum cleaner+ non-deterministically the cleaner dumps dust on the location if it
sucks and location is clean
• triple murphy vacuum cleaner+ non-deterministically the cleaner does not move if it is command to
Gerald Steinbauer
Institute for Software Technology
7
Advanced Topics of AI – Conditional Planning
Sensorless Planning (1)
• assume• partial observability/incomplete knowledge
• deterministic actions
• state space becomes a belief space• 8 states
• 2 beliefs (collections of possible states)
• in general 2
Gerald Steinbauer
Institute for Software Technology
8
Advanced Topics of AI – Conditional Planning
Sensorless Planning (2)
brave vs. cautious execution
3
Gerald Steinbauer
Institute for Software Technology
9
Advanced Topics of AI – Conditional Planning
Prediction of Beliefs
• still no sensing
• deterministic vs. non-deterministic actions
• belief space allows to represent both
deterministic right non-deterministic rightGerald Steinbauer
Institute for Software Technology
10
Advanced Topics of AI – Conditional Planning
Different Plans
• weak plan (optimistic)• the plan has a chance of success, i.e. some of its executions
achieve the goal
• strong plan (pessimistic)• the plan is guaranteed to achieve the goal for all possible
executions
• strong cyclic plan (reasonably optimistic)• the plan reaches the goal with an iterative trial-and-error strategy,
such that all the associated executions always have a possibility of terminating and, when they do, they achieve the goal
• sensorless (conformant) planning looks for a strong plan without observability
Gerald Steinbauer
Institute for Software Technology
11
Advanced Topics of AI – Conditional Planning
Conditional Planning (1)
• deal with uncertainty by checking the environment to see what is really happening
• used in fully observable and nondeterministic environments: • the outcome of an action is non-deterministic• conditional steps will check the state of the environment• how to construct a conditional plan?
Gerald Steinbauer
Institute for Software Technology
12
Advanced Topics of AI – Conditional Planning
Conditional Planning (2)
• actions: left, right, suck• propositions to define states: 𝑎𝑡_𝑟, 𝑎𝑡_𝑙, 𝑐𝑙𝑒𝑎𝑛_𝑙,
𝑐𝑙𝑒𝑎𝑛_𝑟
• how to include indeterminism?• actions can have more than one effect
• e.g., moving left sometimes fails• Action(Left, PRECOND: AtR, EFFECT: AtL)• Becomes: Action(Left, PRECOND: AtR, EFFECT: AtL AtR)
• actions can have conditional effects (vacuum cleaner sometimes dumps dirt when moved)
• Action(Left, PRECOND: AtR,EFFECT: AtL (AtL when cleanL: ¬cleanL)
• both disjunctive and conditional (“when cleanL” refers to before the action is taken, “¬cleanL” refers to after action)
4
Gerald Steinbauer
Institute for Software Technology
13
Advanced Topics of AI – Conditional Planning
Conditional planning (3)
• conditional plans require conditional steps:• if <test> then plan_A else plan_B
• if 𝑎𝑡_𝑙𝑐𝑙𝑒𝑎𝑛_𝑙 then 𝑟𝑖𝑔ℎ𝑡 else 𝑠𝑢𝑐𝑘
• plans become trees
• games against nature:• find conditional plans that work regardless of which action
outcomes actually occur
• assume vacuum-world• initial state = 𝑎𝑡 𝑐𝑙𝑒𝑎𝑛_𝑙 𝑐𝑙𝑒𝑎𝑛_𝑟
• would be easy, except that we have a double murphy vacuum cleaner: possibility of depositing dirt when moving to clean other square, and possibility of depositing dirt when action is suck in a clean square
Gerald Steinbauer
Institute for Software Technology
14
Advanced Topics of AI – Conditional Planning
Game tree
state node
chance node
[left, if at_l clean_l clean_r then [ ] else suck]
Gerald Steinbauer
Institute for Software Technology
15
Advanced Topics of AI – Conditional Planning
Solution of games against Nature
• solution is a subtree that• has a goal node at every leaf• specifies one action at each of its state nodes• oncludes every outcome branch at each of the chance nodes
• in previous example:• [left, if at_l clean_l clean_r then [ ] else suck]
• for exact solutions: use minimax algorithm with 2 modifications:• max and min nodes become OR and AND nodes• algorithm returns conditional plan instead of single move
• at an OR node, the plan is the action selected• at an AND node, the plan is a nested series of if-then-
else steps (or “cases”) specifying subplans for each outcome (with tests being complete state descriptions)
Gerald Steinbauer
Institute for Software Technology
16
Advanced Topics of AI – Conditional Planning
Search And-Or Graph Search (1)
“discards” repeated states in graph, like the “Loop” state in vacuum cleaner game tree
5
Gerald Steinbauer
Institute for Software Technology
17
Advanced Topics of AI – Conditional Planning
• how does it deal with cycles?• when a state already on the path appears, return failure
• if there is a non-cyclic solution, it is reachable from the earlier incarnation of the state
• ensures algorithm termination in every finite state space, since every path must reach a goal, a dead end, or a repeated state
• but the algorithm does not check whether some state is already on some other path from the root
Search And-Or Graph Search (2)
Gerald Steinbauer
Institute for Software Technology
18
Advanced Topics of AI – Conditional Planning
• sometimes only a cyclic solution exists • e.g., triple Murphy: sometimes the move is not performed
• [Left, if CleanL then [] else Suck] is not a solution
• our And-Or-search algorithm would fail• modification: use label to repeat parts of plan (but now there may be infinite
loops)• [L1: left, if at_r then L1 else if clean_l then [] else suck]
search graph for the “triple murphy” world,
left doesn’t always move the vacuum
cleaner
Search And-Or Graph Search (1)
Gerald Steinbauer
Institute for Software Technology
19
Advanced Topics of AI – Conditional Planning
Conditional Planning and Partially Observable Environments
• fully observable: conditional tests can ask any question and get an answer
• partially observable• the agent has limited information about the environment• modeled by a state-set = belief states• e.g., assume vacuum agent which cannot sense presence or absence
of dirt in squares other than the one it’s on• alternate double murphy: dirt can be left behind when agent leaves a clean
square• solution in fully observable world: keep moving left and right, sucking dirt
whenever it appears until both squares are clean and I’m in square left• but “both squares a clean” can’t be tested now…
Gerald Steinbauer
Institute for Software Technology
20
Advanced Topics of AI – Conditional Planning
Incomplete Knowledge
• the agent knows: 𝑐𝑙𝑒𝑎𝑟 𝐴 , 𝑐𝑙𝑒𝑎𝑟 𝐶 , 𝑐𝑙𝑒𝑎𝑟 𝐷 , 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝐶 , ¬𝑜𝑛 𝐴, 𝐶 ,
¬𝑜𝑛 𝐴, 𝐷 , 𝑜𝑛 𝐴, 𝐵 ∨ 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝐴 ,∀𝑥. ¬𝑜𝑛 𝑥, 𝐵 → 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝑥
• the agent doesn’t know: 𝑐𝑙𝑒𝑎𝑟 𝐵 , ¬𝑐𝑙𝑒𝑎𝑟 𝐵 , 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝐴 , ¬𝑜𝑛𝑇𝑎𝑏𝑙𝑒(𝐴)
6
Gerald Steinbauer
Institute for Software Technology
21
Advanced Topics of AI – Conditional Planning
Multiple Worlds
• the agent believes to be in multiple worlds
• all words evolve separately
• the agent knows at the end: 𝑜𝑛 𝐷, 𝐶 , 𝑜𝑛 𝐴, 𝐵 ∨𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝐴 , ∀𝑥. 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝑥 → 𝑥 = 𝐴 ∨ 𝑥 = 𝐵
Gerald Steinbauer
Institute for Software Technology
22
Advanced Topics of AI – Conditional Planning
Defining Knowledge
• we define a new predicate 𝐾 𝑠, 𝑠• called accessibility relation
• expresses that for an agent it is the same to be in situation 𝑠 or 𝑠’
• we can express that an agent knows 𝜙 in situation 𝑆• ∀𝑠. 𝐾 𝑠, 𝑆 → 𝜙[𝑠]
• e.g. ∀𝑠. 𝐾(𝑠, 𝑆 ) → ∀𝑥. 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝑥, 𝑠 → 𝑐𝑙𝑒𝑎𝑟 𝑥, 𝑠
• logical omniscience:• if ∀𝑠. 𝜙(𝑠) → 𝜓(𝑠) and ∀𝑠, 𝑠 . 𝐾 𝑠, 𝑠 → 𝜙 𝑠
• then ∀𝑠, 𝑠 . 𝐾 𝑠, 𝑠 → 𝜓 𝑠
• we can now define what an agent knows• 𝐾𝑛𝑜𝑤𝑠 𝜙, 𝑠 = ∀𝑠 𝐾(𝑠 , 𝑠) → 𝜙[𝑠′]
• 𝐾𝑊ℎ𝑒𝑡ℎ𝑒𝑟 𝜙, 𝑠 = 𝐾𝑛𝑜𝑤𝑠 𝜙, 𝑠 ∨ 𝐾𝑛𝑜𝑤𝑠(¬𝜙, 𝑠)
Gerald Steinbauer
Institute for Software Technology
23
Advanced Topics of AI – Conditional Planning
Incomplete Knowledge and Sensing
• situations are discarded that does not match the sensing result, e.g. 𝐹(𝑠)
Gerald Steinbauer
Institute for Software Technology
24
Advanced Topics of AI – Conditional Planning
Partially Observable:Alternate Double Murphy
this relates back to our
possible worlds formalization of
knowledge; e.g., at top,
agent might be in two possible
worlds, because all it knows is 𝑎𝑡_𝑟and 𝑐𝑙𝑒𝑎𝑛_𝑟
brave vs. cautious execution
7
Gerald Steinbauer
Institute for Software Technology
25
Advanced Topics of AI – Conditional Planning
Belief states
• three choices for representation• sets of full state descriptions
• {{at_r,clean_r,clean_l},{{at_r,clean_r}}
• logical sentences that capture the set of possible worlds in the belief state (Open World Assumption, unknown propositions are left unspecified)
• 𝑎𝑡 ∨ 𝑐𝑙𝑒𝑎𝑛_𝑟
• knowledge propositions describing the agent’s knowledge (Closed World Assumption, unknown knowledge propositions are assumed false)
• 𝐾(𝑎𝑡_𝑟) 𝐾(𝑐𝑙𝑒𝑎𝑛_𝑟)
Gerald Steinbauer
Institute for Software Technology
26
Advanced Topics of AI – Conditional Planning
Belief states• choices 2 and 3 are roughly equivalent (let’s continue with
3)• symbols can appear in three ways: positive, negative or
unknown: 3𝑛 possible belief states for 𝑛 proposition symbols• Yet, set of belief sets is a powerset (set of all subsets) of the
physical states: 2𝑛 physical states, so 22𝑛 belief states --- which is
much larger than 3𝑛
• hence 3 is restricted as representation• any scheme capable of representing every possible belief state will
require 𝑂(2𝑛) bits to represent each one in the worst case • the current scheme only requires 𝑂(𝑛) --- trading expressiveness for
compactness• STRIPS closure
Gerald Steinbauer
Institute for Software Technology
27
Advanced Topics of AI – Conditional Planning
Sensing in Conditional Planning
• how does it work?• automatic sensing
• At every time step the agent gets all available percepts• active sensing
• percepts are obtained through the execution of specific sensory actions.• checkDirt and checkLocation
• given the representation and the sensing, action descriptions for checkDirt and checkLocation can now be formulated
Gerald Steinbauer
Institute for Software Technology
28
Advanced Topics of AI – Conditional Planning
Contingent-FF (1)
• implementation of a conditional planner
• by Jörg Hoffmann and Ronen Brafman
• supports incomplete knowledge and conditional/nondeterministic effects
• understands PDDL plus neccesary extensions
• is limited to acyclic conditional plans and restricts the use of nondeterminism
8
Gerald Steinbauer
Institute for Software Technology
29
Advanced Topics of AI – Conditional Planning
Contingent-FF (2)
• extensions to PDDL• (nondet literal)
• the statement can be used to declare an effect non-deterministic
• supports only one literal
• supports only one non-deterministic effect part per action
• (unknown atom)• declares an atom to be unknown in the initial state
• both possibility will be consider in planning
• (oneof literal_1 literal_2 … literal_n)• allows to specify that exactly one literal holds in the initial state
• (or literal_1 literal_2 … literal_n)• allows to specify that some literals holds in the initial state
• statements are conjunctive -> allows CNF
Gerald Steinbauer
Institute for Software Technology
30
Advanced Topics of AI – Conditional Planning
Literature
• Stuart J. Russell and Peter Norvig. 2003. Artificial Intelligence: A Modern Approach. Pearson Education.
• Malik Ghallab, Dana Nau and Paolo Traverso. Automated Planning – Theory and Practice. Morgan Kaufman. 2005.
• Malik Ghallab, Dana Nau, and Paolo Traverso. 2016. Automated Planning and Acting. Cambridge University Press, New York, NY, USA.
• Jörg Hoffmann and Ronen I. Brafman. 2005. Contingent planning via heuristic forward search with implicit belief states. In Proceedings of the Fifteenth International Conference on International Conference on Automated Planning and Scheduling (ICAPS'05). AAAI Press 71-88.
• This slides reuse material from publicly available lectures by Dana Nau, Jeff Rosenschein and Jörg Hoffmann.
Gerald Steinbauer
Institute for Software Technology
31
Advanced Topics of AI – Conditional Planning
Thank you!