PrinciplesofAr.ﬁcialIntelligence · PennsylvaniaStateUniversity **** College of Information...

Pennsylvania State University

College of Information Sciences and Technology Artificial Intelligence Research Laboratory

Principles of Ar.ficial Intelligence

Vasant Honavar Ar3ficial Intelligence Research Laboratory

College of Informa3on Sciences and Technology Bioinforma3cs and Genomics Graduate Program

The Huck Ins3tutes of the Life Sciences Pennsylvania State University

[email protected] hHp://vhonavar.ist.psu.edu

hHp://faculty.ist.psu.edu/vhonavar

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar




Summary of uninformed (blind) search algorithms Criterion Breadth

-First Depth-First

Iterative deepening

Bi-directional

Complete? YES NO YES YES*

Time bd+1 bm bd bd/2

Space bd+1 bm bd bd/2

Admissible?

Optimal?

YES NO

NO NO

YES YES

YES* NO

Assuming all arc costs are equal m – max depth of search d – depth of solu7on, b – finite branching factor * Assuming forward and backward search are BFS




Finding an op3mal solu3on

•  All operator applica3ons may not be equally expensive •  Suppose we have a cost func3on c: S x O à ℜ+

•  c (q,o,r) = cost of applying operator o in state q to reach state r •  Path cost is typically assumed to be the sum of costs of operator

applica3ons along the path •  An op3mal solu3on is one with the lowest cost path from the

specified start state s to a goal g ∈ G




Finding op3mal (minimum cost) solu3ons

•  Branch and bound search (BBS) with dynamic programming •  Maintain a list of nodes sorted by cost g(n) of cheapest known par3al

paths to the respec3ve nodes •  Terminate when a node picked from the open list happens to be a goal

node A

B

D

C

G E F

I H

1

3 8

2

31

2 1

((A,0))

((AB, 1) (AC, 2)

((AC, 2) (ABD, 4)(ABG,9))

((ACE, 3) (ABD, 4)(ACF, 5) (ABG,9))

((ACEI, 4) (ABD, 4)(ACEH, 5)(ACF, 5) (ABG,9))




Finding op3mal solu3ons

•  Branch and bound search (BBS) with dynamic programming •  Maintain a list of nodes sorted by cost g(n) of cheapest known par3al paths to

the respec3ve nodes •  Terminate when a node picked from the open list happens to be a goal node •  Ques3ons:

–  Is BBS complete? •  Yes

–  Is BBS admissible? •  Yes

–  Under the assump3on that each arc cost is bounded from below by some posi3ve constant δ




Informed search

•  Informed search uses problem-‐specific knowledge •  Which search strategies?

–  Best-‐first search and its variants •  Heuris3c func3ons

–  How to design them •  Local search and op3miza3on

–  Hill climbing, local beam search, gene3c algorithms,… •  Local search in con3nuous spaces •  Online search agents




A heuris3c func3on

•  [dic3onary]“A rule of thumb, simplifica7on, or educated guess that reduces or limits the search for solu7ons in domains that are difficult and poorly understood.”

Heuris3cs, those rules of thumb, Oaen scorned as sloppy, dumb, Yet slowly commonsense become!

–  Judea Pearl, in Heuris7cs




Why we need heuris3cs •  Combinatorial explosion •  Uninformed search runs out of 3me, space, or both quickly •  Solu3on – incorporate heuris3cs to guide search •  Perfect heuris3c would guide search directly from start state

to a goal state yielding the least expensive solu3on –  No need to search!

•  In prac3ce, we have imperfect but useful heuris3cs •  A heuris3c func3on h(n), roughly speaking, es3mates the cost

of comple3ng a par3al path (s..n) to obtain a solu3on, i.e., a path (s..n…g) where g is a goal state




Why we need heuris3cs •  Combinatorial explosion •  Uninformed search runs out of 3me, space, or both quickly •  Perfect ra3onality unaffordable •  Can we come up with a compromise?

–  Use a heuris3c func3on to guide choice of ac3ons •  Suppose we could es3mate the cost of the cheapest solu3on

obtainable by expanding each node that could be chosen

A

s1 s2 s3

( )( )

( ) ( ) ( ) 6.1 0.2 8.00 ,

0 ,:

321 ===

=∈∀

≥∈∀

ℜ→ +

shshshghGgnhSn

Sh




A heuris3c func3on

In problems requiring the cheapest path from start state to a goal state

–  h(n) = es3mated cost of the cheapest path from node n to a goal node

–  h(g1)=h(g2)=0 s

n

g2

m

g1

h(m) h(n)




Hill-‐climbing search

•  Choose “locally best” moves, guided by the heuris3c func3on, with random choice to break 3es

•  Terminates when a goal state is reached •  Does not look beyond the immediate successors of the

current state in deciding which move to make •  Essen3ally DFS, where at each step, successors are

ordered by heuris3c evalua3on •  a.k.a. greedy local search




Heuris3c func3on -‐ Example

•  E.g for the 8-‐puzzle –  Avg. solu3on cost is about 22 steps –  Exhaus3ve search to depth 22 ≈ 3.1 x 1010 states –  A good heuris3c func3on can reduce search




Heuris3c func3ons

Two commonly used heuris3cs •  h1 = the number of misplaced 3les (not coun3ng blank) rela3ve

to the goal (why?) h1(s)=8

•  h2 = the sum of the distances of the 3les (not coun3ng blank) from their desired posi3ons (ManhaHan distance) (why?) h2(s)=3+1+2+2+2+3+3+2=18




Drawbacks of hill-‐climbing search

•  Ridge –  sequence of local maxima difficult for greedy algorithms to navigate

•  Plateau –  an area of the state space where the evalua3on func3on is flat




Hill-‐climbing varia3ons

•  Stochas3c hill-‐climbing –  Random selec3on among the uphill moves –  The selec3on probability can vary with the steepness of the uphill move

•  First-‐choice hill-‐climbing –  stochas3c hill climbing by genera3ng successors randomly un3l a beHer one is found

•  Random-‐restart hill-‐climbing –  Tries to avoid geqng stuck in local maxima




Best-‐first search

•  General approach to informed search: –  Best-‐first search: node is selected for expansion based on an evalua7on func7on f(n)

•  Idea: evalua3on func3on measures es3mated cost of comple3ng the par3al solu3on (s..n) –  Choose node which appears best

•  Implementa3on: –  Open-list is a queue sorted in decreasing order of desirability –  Special cases: greedy search, A* search




Examples of Best-‐first search

•  Best first search –  List of par3al paths is ordered by h values of the nodes

•  A* search –  BBS-‐like search with dynamic programming –  Open list nodes ordered by

( ) ( ) ( )nhngnf +=




A* search

s

n

g2

m

g1

h(m)=6 h(n)=8

g(n) =2 g(m)=3

3 4

2

3 5 6 8

( ) ( ) ( )nhngnf +=




Romania with step costs in km

•  hSLD= straight-‐line distance heuris3c.

•  hSLD cannot be computed from the problem descrip3on, requires addi3onal informa3on or knowledge on the part of the agent

•  In this example f(n)=h(n) –  Expand node that is closest to goal

= Greedy best-‐first search




A* search

•  Best-‐known form of best-‐first search •  Idea: avoid expanding paths that are already expensive •  Evalua3on func3on f(n)=g(n) + h(n)

–  g(n) the cost (so far) to reach the node. –  h(n) es3mated cost to get from the node to the goal –  f(n) es3mated total cost of path through n to goal




A* search

•  A* search using an admissible heuris3c –  A heuris3c is admissible if it never overes7mates the cost to reach the goal

–  Op3mis3c

Formally: 1. h(n) <= h*(n) where h*(n) is the true cost from n 2. h(n) >= 0 so h(G)=0 for any goal G.

e.g. hSLD(n) never overes3mates the actual road distance




Romania example




Admissible heuris3cs •  A heuris3c h(n) is admissible if for every node n,

h(n) ≤ h*(n), where h*(n) is the true cost to reach the goal state from n. •  An admissible heuris3c never overes3mates the cost to reach the goal, i.e., it

is op3mis3c •  Example: hSLD(n) (never overes3mates the actual road distance) •  Theorem: If h(n) is admissible, A* using TREE-SEARCH is admissible •  (provided each arc cost is bounded from below by a posi3ve constant δ) •  Proof Sketch:

–  Show that A* terminates with a solu3on –  Show that A* terminates with an op3mal solu3on




Completeness of A*: A* Terminates with a solu3on

•  Assump3on: Cost of each arc is bounded from below by a posi3ve constant

•  Any 3me a path gets extended, its cost increases by at least •  There is a solu3on, i.e., a bounded cost path to a goal (by assump3on) •  Termina3on condi3on: a path termina3ng in a goal is picked off the front of

the list of par3al paths L •  There are two kinds of paths other than those that terminate in a goal:

–  Paths that lead to dead ends – discovered and discarded (if their cost is lower than that of a path to a goal)

–  Paths that are infinite (demoted behind a path leading to a goal because sooner or later their cost exceeds the cost of a path leading to a goal)

–  See notes for a formal proof.

€

∀ni,n j ,c ni,n j( ) ≥ δ > 0




Admissibility of A*: A* cannot terminate with a subop3mal goal

Suppose some subop3mal goal G2 has been generated and is in the fringe. Let n be an unexpanded node in the fringe such that n is on a shortest path to an op3mal goal G.

f(G2) = g(G2) since h(G2) = 0 g(G2) > g(G) since G2 is subop3mal f(G) = g(G) since h(G) = 0 f(G2) > f(G) from above




Admissibility of A*: A* cannot terminate with a subop3mal goal

Suppose some subop3mal goal G2 has been generated and is in the fringe. Let n be an unexpanded node in the fringe such that n is on a shortest path to an op3mal goal G.

f(G2) > f(G) from above h(n) ≤ h*(n) since h is admissible g(n) + h(n) ≤ g(n) + h*(n) f(n) ≤ f(G) Hence f(G2) > f(n), and A* will never select G2 for expansion That is, A* tree search is admissible provided the heuris3c func3on

is admissible See notes for a more formal proof.




What if the search space is not a tree?




Graph search •  Previous proof breaks down •  Possible solu3ons:

–  Discard more expensive paths when there are mul3ple paths to a node • Adds messy book-‐keeping • Admissible if h is admissible

–  Ensure that op3mal path to any repeated state is always first followed • Extra requirement on h(n): consistency

∀n,∀n ' where n ' is a child of n, h(n) ≤ h(n ')+ c(n,n ')




Consistency If h is consistent, we have

f(n) is non-‐decreasing along any path à First goal selected for expansion must be an op3mal goal à A* using graph search is admissible provided the heuris3c used is

consistent

)(),()( mhmncnh +≤

)()()(

)(),()()()()(

nfnhng

mhmncngmhmgmf

>

++≥

++=

+=

δ

c ( n ; m ) c ( n ; m )

n

m

G

h ( n )

h ( m )




Proper3es of A* search

•  Complete?: YES –  Since bands of increasing f are added –  Unless there are infinitely many nodes with f < f(G)

• Not possible when the branching factor is finite and arc costs are bounded from below by δ

•  Time complexity: –  exponen3al in the length of the solu3on (in the worst case)

•  Space complexity: –  Need to maintain all generated nodes in memory –  Space is a more serious problem than 3me




Memory-‐bounded heuris3c search

•  Some solu3ons to A*’s space problems –  Itera3ve-‐deepening A* (IDA*)

• Cutoff informa3on is the f-‐cost (g+h) instead of depth • Can expand too many nodes when every node has a different f value

–  (simple) Memory-‐bounded A* ((S)MA*) • Drop the worst-‐leaf node when memory is full

–  Recursive best-‐first search (RBFS) • Recursive algorithm that aHempts to mimic standard best-‐first search with linear space




(simple) memory-‐bounded A*

•  Use all available memory. –  i.e. expand best leafs un3l available memory is full –  When full, SMA* drops worst leaf node (highest f-‐value) –  Backup the f-‐value of the forgoHen node to its parent

•  What if all leafs have the same f-‐value? –  Same node could be selected for expansion and dele3on –  SMA* solves this by expanding newest best leaf and dele3ng oldest worst leaf.

•  SMA* is complete if solu3on is reachable, admissible if op3mal solu3on is reachable within the available memory bound




Recursive best-‐first search (RBFS)

•  Similar to IDA* but keeps track of the f value of the best-‐alterna3ve path available from any ancestor of the current node –  If current f values exceeds this alterna3ve f value then backtrack to the best alterna3ve path

–  During backtracking (unwinding of recursion) change f value of each node to best f-‐value of its children

–  Remembers the f value of the best leaf in the “forgoHen” sub-‐tree and hence can revisit that sub-‐tree, if warranted, at some point in the future




Recursive best-‐first search func.on RECURSIVE-‐BEST-‐FIRST-‐SEARCH(problem) return a solu3on or failure

return RFBS(problem,MAKE-‐NODE(INITIAL-‐STATE[problem]),∞) func.on RFBS( problem, node, f_limit) return a solu3on or failure and a new f-‐

cost limit if GOAL-‐TEST[problem](STATE[node]) then return node successors ← EXPAND(node, problem) if successors is empty then return failure, ∞ for each s in successors do f [s] ← max(g(s) + h(s), f [node]) repeat best ← the lowest f-‐value node in successors if f [best] > f_limit then return failure, f [best] alterna7ve ← the second lowest f-‐value among successors result, f [best] ← RBFS(problem, best, min(f_limit, alterna7ve)) if result ≠ failure then return result




Recursive best-‐first search: Example

•  Path un3l Rumnicu Vilcea is already expanded •  f-‐limit for every recursive call is shown above the node •  f value is shown below the node •  The path is followed un3l Pites3 which has a f value worse than

the f-‐limit.




Recursive best-‐first search: Example

•  Unwind recursion and store best f-‐value for current best leaf Pites3 result, f [best] ← RBFS(problem, best, min(f_limit, alterna7ve))

•  best is now Fagaras. Call RBFS for new best –  best value is now 450




Recursive best-‐first search: example.

•  Unwind recursion and store best f-‐value for current best leaf Fagaras

•  best is now Rimnicu Viclea (again). Call RBFS for new best –  Subtree is again expanded. –  Best alterna7ve subtree is now through Timisoara.

•  Solu3on is found because 447 > 417.




RBFS

•  RBFS is a bit more efficient than IDA* –  S3ll excessive node genera3on –  Like A*, admissible if h(n) is admissible

•  Space complexity is O(bd). •  Time complexity difficult to characterize

–  Depends on accuracy if h(n) and how oaen best path changes •  IDA* and RBFS suffer from too li&le memory •  Other variants of A* aim to address these limita3ons




Learning to search beHer

•  All previous algorithms use fixed strategies. •  Agents can learn to improve their search by exploi3ng

experience gained during the search (e.g., by analyzing missteps) •  We will see examples of this when we consider learning agents

PrinciplesofAr.ﬁcialIntelligence · PennsylvaniaStateUniversity **** College of Information...

Documents

Transcript of PrinciplesofAr.ﬁcialIntelligence · PennsylvaniaStateUniversity **** College of Information...

Principles*of*Ar.ﬁcial*Intelligence* · PennsylvaniaState*University* **** College of Information...

Documents

Transcript of Principles*of*Ar.ﬁcial*Intelligence* · PennsylvaniaState*University* **** College of Information...

PrinciplesofAr.ﬁcialIntelligence · PennsylvaniaStateUniversity **** College of Information...

Transcript of PrinciplesofAr.ﬁcialIntelligence · PennsylvaniaStateUniversity **** College of Information...