CS 416 Artificial Intelligence Lecture 5 Finish Uninformed Searches Begin Informed Searches Lecture...

53
CS 416 Artificial Intelligence Lecture 5 Lecture 5 Finish Uninformed Searches Finish Uninformed Searches Begin Informed Searches Begin Informed Searches

Transcript of CS 416 Artificial Intelligence Lecture 5 Finish Uninformed Searches Begin Informed Searches Lecture...

CS 416Artificial Intelligence

Lecture 5Lecture 5

Finish Uninformed SearchesFinish Uninformed Searches

Begin Informed SearchesBegin Informed Searches

Lecture 5Lecture 5

Finish Uninformed SearchesFinish Uninformed Searches

Begin Informed SearchesBegin Informed Searches

Uniform-cost search (review)

Always expand the lowest-path-cost nodeAlways expand the lowest-path-cost node

Don’t evaluate the node until it is “expanded”… Don’t evaluate the node until it is “expanded”… not when it is the result of expansionnot when it is the result of expansion

Always expand the lowest-path-cost nodeAlways expand the lowest-path-cost node

Don’t evaluate the node until it is “expanded”… Don’t evaluate the node until it is “expanded”… not when it is the result of expansionnot when it is the result of expansion

Uniform-cost search (review)

Fringe = [SFringe = [S00]]

Expand(S) Expand(S) {A {A11, B, B55, C, C1515}}

Fringe = [SFringe = [S00]]

Expand(S) Expand(S) {A {A11, B, B55, C, C1515}}

B

Uniform-cost search (review)

Fringe = [AFringe = [A11, B, B55, C, C1515]]

Expand(A) = {GExpand(A) = {G1111}}

Fringe = [AFringe = [A11, B, B55, C, C1515]]

Expand(A) = {GExpand(A) = {G1111}}

Uniform-cost search (review)

Fringe = [BFringe = [B55, G, G1111, C, C1515]]

Expand(B) Expand(B) {G {G1010}}

Fringe = [BFringe = [B55, G, G1111, C, C1515]]

Expand(B) Expand(B) {G {G1010}}

Uniform-cost search (review)

Fringe = [GFringe = [G1010, C, C1515]]

Expand(G) Expand(G) Goal Goal

Fringe = [GFringe = [G1010, C, C1515]]

Expand(G) Expand(G) Goal Goal

Bidirectional search

Search from goal to startSearch from goal to start

Search from start to goalSearch from start to goal

Search from goal to startSearch from goal to start

Search from start to goalSearch from start to goal

Two bTwo bd/2d/2 searches searches instead of one binstead of one bdd

Two bTwo bd/2d/2 searches searches instead of one binstead of one bdd

Bidirectional search

ImplementationImplementation

• Each search checks nodes before expansion to see if they are Each search checks nodes before expansion to see if they are on the fringe of the other searchon the fringe of the other search

ImplementationImplementation

• Each search checks nodes before expansion to see if they are Each search checks nodes before expansion to see if they are on the fringe of the other searchon the fringe of the other search

Bidirectional search

• Example: Bidirectional BFS search with d=6 & b=10Example: Bidirectional BFS search with d=6 & b=10

– Worst case: both search trees must expand all but one Worst case: both search trees must expand all but one element of the third level of the treeelement of the third level of the tree

2* (1 + 10 + 100 + 1000 + 10000 - 10) node 2* (1 + 10 + 100 + 1000 + 10000 - 10) node expansionsexpansions

Versus 1 * (1 + 10 + 100 + Versus 1 * (1 + 10 + 100 + …… + 10000000) expansions + 10000000) expansions

• Example: Bidirectional BFS search with d=6 & b=10Example: Bidirectional BFS search with d=6 & b=10

– Worst case: both search trees must expand all but one Worst case: both search trees must expand all but one element of the third level of the treeelement of the third level of the tree

2* (1 + 10 + 100 + 1000 + 10000 - 10) node 2* (1 + 10 + 100 + 1000 + 10000 - 10) node expansionsexpansions

Versus 1 * (1 + 10 + 100 + Versus 1 * (1 + 10 + 100 + …… + 10000000) expansions + 10000000) expansions

Bidirectional search

ImplementationImplementation

• Checking fringe of other treeChecking fringe of other tree

– At least one search tree must be kept in memoryAt least one search tree must be kept in memory

– Checking can be done in constant time (hash table)Checking can be done in constant time (hash table)

• Searching back from goalSearching back from goal

– Must be able to compute predecessors to node n: Must be able to compute predecessors to node n: Pred (n)Pred (n)

– Easy with 15-puzzle, but how about chess?Easy with 15-puzzle, but how about chess?

ImplementationImplementation

• Checking fringe of other treeChecking fringe of other tree

– At least one search tree must be kept in memoryAt least one search tree must be kept in memory

– Checking can be done in constant time (hash table)Checking can be done in constant time (hash table)

• Searching back from goalSearching back from goal

– Must be able to compute predecessors to node n: Must be able to compute predecessors to node n: Pred (n)Pred (n)

– Easy with 15-puzzle, but how about chess?Easy with 15-puzzle, but how about chess?

Avoid repeated states

Search algorithms that forget their history are Search algorithms that forget their history are doomed to repeat it.doomed to repeat it. Russell and NorvigRussell and Norvig

• So remember where you’ve been… on a listSo remember where you’ve been… on a list

• If you come upon a node you’ve visited before, don’t expand itIf you come upon a node you’ve visited before, don’t expand it

• Let’s call this Let’s call this GRAPH-SEARCHGRAPH-SEARCH

Search algorithms that forget their history are Search algorithms that forget their history are doomed to repeat it.doomed to repeat it. Russell and NorvigRussell and Norvig

• So remember where you’ve been… on a listSo remember where you’ve been… on a list

• If you come upon a node you’ve visited before, don’t expand itIf you come upon a node you’ve visited before, don’t expand it

• Let’s call this Let’s call this GRAPH-SEARCHGRAPH-SEARCH

GRAPH-SEARCH

Faster and smaller space requirements if many Faster and smaller space requirements if many repeating statesrepeating states

• Time-space requirements are a function of state space not Time-space requirements are a function of state space not depth/branching of tree to goaldepth/branching of tree to goal

• At this point in class, repeated states will be thrown away – At this point in class, repeated states will be thrown away – even if the new path to the state is better than the one even if the new path to the state is better than the one explored previously explored previously

Faster and smaller space requirements if many Faster and smaller space requirements if many repeating statesrepeating states

• Time-space requirements are a function of state space not Time-space requirements are a function of state space not depth/branching of tree to goaldepth/branching of tree to goal

• At this point in class, repeated states will be thrown away – At this point in class, repeated states will be thrown away – even if the new path to the state is better than the one even if the new path to the state is better than the one explored previously explored previously

Interesting problems

Exercise 3.9:Exercise 3.9:

• 3 cannibals and 3 missionaries and a boat that can hold one 3 cannibals and 3 missionaries and a boat that can hold one or two people are on one side of the river. Get everyone or two people are on one side of the river. Get everyone across the river (early AI problem, 1968)across the river (early AI problem, 1968)

• 8-puzzle and 15-puzzle, invented by Sam Loyd in good ol’ 8-puzzle and 15-puzzle, invented by Sam Loyd in good ol’ USA in 1870s. Think about search space.USA in 1870s. Think about search space.

• Rubik’s cubeRubik’s cube

• Traveling Salesman Problem (TSP)Traveling Salesman Problem (TSP)

Exercise 3.9:Exercise 3.9:

• 3 cannibals and 3 missionaries and a boat that can hold one 3 cannibals and 3 missionaries and a boat that can hold one or two people are on one side of the river. Get everyone or two people are on one side of the river. Get everyone across the river (early AI problem, 1968)across the river (early AI problem, 1968)

• 8-puzzle and 15-puzzle, invented by Sam Loyd in good ol’ 8-puzzle and 15-puzzle, invented by Sam Loyd in good ol’ USA in 1870s. Think about search space.USA in 1870s. Think about search space.

• Rubik’s cubeRubik’s cube

• Traveling Salesman Problem (TSP)Traveling Salesman Problem (TSP)

Chapter 4 – Informed Search

INFORMED?INFORMED?• Uses problem-specific knowledge Uses problem-specific knowledge

beyond the definition of the problem itselfbeyond the definition of the problem itself

– selecting best selecting best lane in trafficlane in traffic

– playing a sportplaying a sport

what’s the what’s the heuristic heuristic (or evaluation (or evaluation function)?function)?

INFORMED?INFORMED?• Uses problem-specific knowledge Uses problem-specific knowledge

beyond the definition of the problem itselfbeyond the definition of the problem itself

– selecting best selecting best lane in trafficlane in traffic

– playing a sportplaying a sport

what’s the what’s the heuristic heuristic (or evaluation (or evaluation function)?function)?

www.curling.ca

Best-first search

BFS/DFS/UCS differ in how they select a node to BFS/DFS/UCS differ in how they select a node to pull off the fringepull off the fringe

• We want to pull off the fringe the node that’s on the optimal We want to pull off the fringe the node that’s on the optimal pathpath

– But if we know the “best” node to explore, we don’t have to But if we know the “best” node to explore, we don’t have to search!!!search!!!

BFS/DFS/UCS differ in how they select a node to BFS/DFS/UCS differ in how they select a node to pull off the fringepull off the fringe

• We want to pull off the fringe the node that’s on the optimal We want to pull off the fringe the node that’s on the optimal pathpath

– But if we know the “best” node to explore, we don’t have to But if we know the “best” node to explore, we don’t have to search!!!search!!!

We’re not certain we know the best node

Best-first search

Use an evaluation function to select node to Use an evaluation function to select node to expandexpand

• f(n) = evaluation function = expected “cost” for a path from f(n) = evaluation function = expected “cost” for a path from root to goal that goes through node root to goal that goes through node nn

• Select the node Select the node nn on fringe that minimizes on fringe that minimizes f(n)f(n)

• How do we build How do we build f(n)f(n)??

Use an evaluation function to select node to Use an evaluation function to select node to expandexpand

• f(n) = evaluation function = expected “cost” for a path from f(n) = evaluation function = expected “cost” for a path from root to goal that goes through node root to goal that goes through node nn

• Select the node Select the node nn on fringe that minimizes on fringe that minimizes f(n)f(n)

• How do we build How do we build f(n)f(n)??

Evaluation function: f(n)

Combine two costsCombine two costs

• f(n) = g(n) + h(n)f(n) = g(n) + h(n)

– g(n)g(n) = cost to get to = cost to get to nn from from start start We know this! We know this!

– h(n)h(n) = cost to get to from = cost to get to from n n to to goalgoal

Combine two costsCombine two costs

• f(n) = g(n) + h(n)f(n) = g(n) + h(n)

– g(n)g(n) = cost to get to = cost to get to nn from from start start We know this! We know this!

– h(n)h(n) = cost to get to from = cost to get to from n n to to goalgoal

Heuristics

A function, h(n), that estimates cost of cheapest A function, h(n), that estimates cost of cheapest path from node n to the goalpath from node n to the goal

• h(n) = 0 if n == goal nodeh(n) = 0 if n == goal node

A function, h(n), that estimates cost of cheapest A function, h(n), that estimates cost of cheapest path from node n to the goalpath from node n to the goal

• h(n) = 0 if n == goal nodeh(n) = 0 if n == goal node

Greedy best-first search

Trust your heuristic and ignore path costsTrust your heuristic and ignore path costs

• evaluate node that minimizes evaluate node that minimizes h(n)h(n)

• f(n) = h(n)f(n) = h(n)

Example: getting from A to BExample: getting from A to B

• Explore nodes with shortest straight distance to BExplore nodes with shortest straight distance to B

• Shortcomings of heuristic?Shortcomings of heuristic?

– Greedy can be badGreedy can be bad

Trust your heuristic and ignore path costsTrust your heuristic and ignore path costs

• evaluate node that minimizes evaluate node that minimizes h(n)h(n)

• f(n) = h(n)f(n) = h(n)

Example: getting from A to BExample: getting from A to B

• Explore nodes with shortest straight distance to BExplore nodes with shortest straight distance to B

• Shortcomings of heuristic?Shortcomings of heuristic?

– Greedy can be badGreedy can be bad

A* (A-star) Search

Don’t simply minimize the cost to goal…Don’t simply minimize the cost to goal…minimize the cost from start to goal…minimize the cost from start to goal…

• f(n) = g(n) + h(n)f(n) = g(n) + h(n)

– g(n)g(n) = cost to get to = cost to get to n n from from startstart

– h(n)h(n) = cost to get from = cost to get from nn to to goalgoal

Select node from fringe that minimizes f(n)Select node from fringe that minimizes f(n)

Don’t simply minimize the cost to goal…Don’t simply minimize the cost to goal…minimize the cost from start to goal…minimize the cost from start to goal…

• f(n) = g(n) + h(n)f(n) = g(n) + h(n)

– g(n)g(n) = cost to get to = cost to get to n n from from startstart

– h(n)h(n) = cost to get from = cost to get from nn to to goalgoal

Select node from fringe that minimizes f(n)Select node from fringe that minimizes f(n)

A* is Optimal?

A* can be optimal if h(n) satisfies conditionsA* can be optimal if h(n) satisfies conditions

• h(n) never overestimates cost to reach the goalh(n) never overestimates cost to reach the goal

– it is eternally optimisticit is eternally optimistic

– called ancalled an admissible heuristicadmissible heuristic

– f(n) never overestimates cost of a solution through nf(n) never overestimates cost of a solution through n

• Proof of optimality?Proof of optimality?

A* can be optimal if h(n) satisfies conditionsA* can be optimal if h(n) satisfies conditions

• h(n) never overestimates cost to reach the goalh(n) never overestimates cost to reach the goal

– it is eternally optimisticit is eternally optimistic

– called ancalled an admissible heuristicadmissible heuristic

– f(n) never overestimates cost of a solution through nf(n) never overestimates cost of a solution through n

• Proof of optimality?Proof of optimality?

A* is Optimal

We must prove that A* will not return a suboptimal We must prove that A* will not return a suboptimal goal or a suboptimal path to a goalgoal or a suboptimal path to a goal

• Let G be a suboptimal goal nodeLet G be a suboptimal goal node

– f(G) = g(G) + h(G)f(G) = g(G) + h(G)

– h(G) = 0 because G is a goal node and we cannot h(G) = 0 because G is a goal node and we cannot overestimate actual cost to reach Goverestimate actual cost to reach G

– f(G) = g(G) > C* (because G is suboptimal)f(G) = g(G) > C* (because G is suboptimal)

We must prove that A* will not return a suboptimal We must prove that A* will not return a suboptimal goal or a suboptimal path to a goalgoal or a suboptimal path to a goal

• Let G be a suboptimal goal nodeLet G be a suboptimal goal node

– f(G) = g(G) + h(G)f(G) = g(G) + h(G)

– h(G) = 0 because G is a goal node and we cannot h(G) = 0 because G is a goal node and we cannot overestimate actual cost to reach Goverestimate actual cost to reach G

– f(G) = g(G) > C* (because G is suboptimal)f(G) = g(G) > C* (because G is suboptimal)

A* is Optimal (cont.)

• Let n be a node on the optimal pathLet n be a node on the optimal path

– because h(n) does not overestimatebecause h(n) does not overestimate

– f(n) = g(n) + h(n) <= C*f(n) = g(n) + h(n) <= C*

• Therefore f(n) <= C* < f(G)Therefore f(n) <= C* < f(G)

– node n will be selected before node Gnode n will be selected before node G

• Let n be a node on the optimal pathLet n be a node on the optimal path

– because h(n) does not overestimatebecause h(n) does not overestimate

– f(n) = g(n) + h(n) <= C*f(n) = g(n) + h(n) <= C*

• Therefore f(n) <= C* < f(G)Therefore f(n) <= C* < f(G)

– node n will be selected before node Gnode n will be selected before node G

A* is optimal if h() is admissible

Repeated States and GRAPH-SEARCH

GRAPH-SEARCH always ignores all but the first GRAPH-SEARCH always ignores all but the first occurrence of a state during searchoccurrence of a state during search

• Lower cost path may be tossedLower cost path may be tossed

– So, don’t throw away subsequent occurrencesSo, don’t throw away subsequent occurrences

– Or, ensure that the optimal path to any repeated state is Or, ensure that the optimal path to any repeated state is always the first one followedalways the first one followed

• Additional constraint on heurisitic, Additional constraint on heurisitic, consistencyconsistency

GRAPH-SEARCH always ignores all but the first GRAPH-SEARCH always ignores all but the first occurrence of a state during searchoccurrence of a state during search

• Lower cost path may be tossedLower cost path may be tossed

– So, don’t throw away subsequent occurrencesSo, don’t throw away subsequent occurrences

– Or, ensure that the optimal path to any repeated state is Or, ensure that the optimal path to any repeated state is always the first one followedalways the first one followed

• Additional constraint on heurisitic, Additional constraint on heurisitic, consistencyconsistency

Consistent heuristic: h(n)

Heuristic function must be monotonicHeuristic function must be monotonic

• for every node, for every node, nn, and successor,, and successor, n’n’, obtained with action , obtained with action aa

– estimated cost of reaching goal from estimated cost of reaching goal from nn is no greater than is no greater than cost of getting to cost of getting to n’n’ plus estimated cost of reaching goal plus estimated cost of reaching goal from from n’n’

– h(n) <= c(n, a, n’) + h(n’)h(n) <= c(n, a, n’) + h(n’)

• This implies f(n) along any path are nondecreasingThis implies f(n) along any path are nondecreasing

Heuristic function must be monotonicHeuristic function must be monotonic

• for every node, for every node, nn, and successor,, and successor, n’n’, obtained with action , obtained with action aa

– estimated cost of reaching goal from estimated cost of reaching goal from nn is no greater than is no greater than cost of getting to cost of getting to n’n’ plus estimated cost of reaching goal plus estimated cost of reaching goal from from n’n’

– h(n) <= c(n, a, n’) + h(n’)h(n) <= c(n, a, n’) + h(n’)

• This implies f(n) along any path are nondecreasingThis implies f(n) along any path are nondecreasing

Examples of consistent h(n)

h(n) <= c(n, a, nh(n) <= c(n, a, nsuccsucc) + h(n) + h(nsuccsucc))

• recall recall h(n)h(n) is admissible is admissible

– The quickest you can get there from here is 10 minutesThe quickest you can get there from here is 10 minutes

It may take more than 10 minutes, but not fewerIt may take more than 10 minutes, but not fewer

• After taking an action and learning the After taking an action and learning the costcost

– It took you two minutes to get here and you still have nine minutes to It took you two minutes to get here and you still have nine minutes to gogo

– We cannot learn… it took you two minutes to get here and you have We cannot learn… it took you two minutes to get here and you have seven minutes to goseven minutes to go

h(n) <= c(n, a, nh(n) <= c(n, a, nsuccsucc) + h(n) + h(nsuccsucc))

• recall recall h(n)h(n) is admissible is admissible

– The quickest you can get there from here is 10 minutesThe quickest you can get there from here is 10 minutes

It may take more than 10 minutes, but not fewerIt may take more than 10 minutes, but not fewer

• After taking an action and learning the After taking an action and learning the costcost

– It took you two minutes to get here and you still have nine minutes to It took you two minutes to get here and you still have nine minutes to gogo

– We cannot learn… it took you two minutes to get here and you have We cannot learn… it took you two minutes to get here and you have seven minutes to goseven minutes to go

2109

010

Example of inconsistent h(n)

As a thought exercise for after classAs a thought exercise for after class

• Consider what happens when a heuristic is inconsistentConsider what happens when a heuristic is inconsistent

• Consider how one could have a consistent but non-Consider how one could have a consistent but non-admissible heuristicadmissible heuristic

As a thought exercise for after classAs a thought exercise for after class

• Consider what happens when a heuristic is inconsistentConsider what happens when a heuristic is inconsistent

• Consider how one could have a consistent but non-Consider how one could have a consistent but non-admissible heuristicadmissible heuristic

Proof of monotonicity of f(n)

If h(n) is consistent (monotonic)If h(n) is consistent (monotonic)

then f(n) along any path is nondecreasingthen f(n) along any path is nondecreasing

• let n’ be a successor of nlet n’ be a successor of n

– g(n’) = g(n) + c (n, a, n’) for some g(n’) = g(n) + c (n, a, n’) for some aa

– f(n’) = g(n’) + h(n’) f(n’) = g(n’) + h(n’) = g(n) + c(n, a, n’) + h(n’) >= g(n) + h(n) = f(n) = g(n) + c(n, a, n’) + h(n’) >= g(n) + h(n) = f(n)

If h(n) is consistent (monotonic)If h(n) is consistent (monotonic)

then f(n) along any path is nondecreasingthen f(n) along any path is nondecreasing

• let n’ be a successor of nlet n’ be a successor of n

– g(n’) = g(n) + c (n, a, n’) for some g(n’) = g(n) + c (n, a, n’) for some aa

– f(n’) = g(n’) + h(n’) f(n’) = g(n’) + h(n’) = g(n) + c(n, a, n’) + h(n’) >= g(n) + h(n) = f(n) = g(n) + c(n, a, n’) + h(n’) >= g(n) + h(n) = f(n)

monotonicity impliesh(n) <= c(n, a, n’) + h(n’)

Contours

Because f(n) is Because f(n) is nondecreasingnondecreasing

we can draw we can draw contourscontours

• If we know C*If we know C*

• We only need to We only need to explore contours explore contours less than C*less than C*

Because f(n) is Because f(n) is nondecreasingnondecreasing

we can draw we can draw contourscontours

• If we know C*If we know C*

• We only need to We only need to explore contours explore contours less than C*less than C*

Properties of A*

• A* expands all nodes with f(n) < C*A* expands all nodes with f(n) < C*

• A* expands some (at least one) of the nodes on the C* A* expands some (at least one) of the nodes on the C* contour before finding the goalcontour before finding the goal

• A* expands no nodes with f(n) > C*A* expands no nodes with f(n) > C*

– these unexpanded nodes can be these unexpanded nodes can be prunedpruned

• A* expands all nodes with f(n) < C*A* expands all nodes with f(n) < C*

• A* expands some (at least one) of the nodes on the C* A* expands some (at least one) of the nodes on the C* contour before finding the goalcontour before finding the goal

• A* expands no nodes with f(n) > C*A* expands no nodes with f(n) > C*

– these unexpanded nodes can be these unexpanded nodes can be prunedpruned

A* is Optimally Efficient

Compared to other algorithms that search from rootCompared to other algorithms that search from root

Compared to other algorithms using same heuristicCompared to other algorithms using same heuristic

No other optimal algorithm is guaranteed to expand No other optimal algorithm is guaranteed to expand fewer nodes than A*fewer nodes than A*

(except perhaps eliminating consideration of ties at f(n) = C*)(except perhaps eliminating consideration of ties at f(n) = C*)

Compared to other algorithms that search from rootCompared to other algorithms that search from root

Compared to other algorithms using same heuristicCompared to other algorithms using same heuristic

No other optimal algorithm is guaranteed to expand No other optimal algorithm is guaranteed to expand fewer nodes than A*fewer nodes than A*

(except perhaps eliminating consideration of ties at f(n) = C*)(except perhaps eliminating consideration of ties at f(n) = C*)

Pros and Cons of A*

A* is optimal and optimally efficientA* is optimal and optimally efficient

A* is still slow and bulky (space kills first)A* is still slow and bulky (space kills first)• Number of nodes grows exponentially with the length to goalNumber of nodes grows exponentially with the length to goal

– This is actually a function of heuristic, but they all have This is actually a function of heuristic, but they all have errorserrors

• A* must search all nodes within this goal contourA* must search all nodes within this goal contour

• Finding suboptimal goals is sometimes only feasible solutionFinding suboptimal goals is sometimes only feasible solution

• Sometimes, better heuristics are non-admissibleSometimes, better heuristics are non-admissible

A* is optimal and optimally efficientA* is optimal and optimally efficient

A* is still slow and bulky (space kills first)A* is still slow and bulky (space kills first)• Number of nodes grows exponentially with the length to goalNumber of nodes grows exponentially with the length to goal

– This is actually a function of heuristic, but they all have This is actually a function of heuristic, but they all have errorserrors

• A* must search all nodes within this goal contourA* must search all nodes within this goal contour

• Finding suboptimal goals is sometimes only feasible solutionFinding suboptimal goals is sometimes only feasible solution

• Sometimes, better heuristics are non-admissibleSometimes, better heuristics are non-admissible

Memory-bounded Heuristic Search

Try to reduce memory needsTry to reduce memory needs

Take advantage of heuristic to improve performanceTake advantage of heuristic to improve performance

• Iterative-deepening A* (IDA*)Iterative-deepening A* (IDA*)

• Recursive best-first search (RBFS)Recursive best-first search (RBFS)

• SMA*SMA*

Try to reduce memory needsTry to reduce memory needs

Take advantage of heuristic to improve performanceTake advantage of heuristic to improve performance

• Iterative-deepening A* (IDA*)Iterative-deepening A* (IDA*)

• Recursive best-first search (RBFS)Recursive best-first search (RBFS)

• SMA*SMA*

Iterative Deepening A*

Iterative DeepeningIterative Deepening

• Remember from uninformed search, this was a depth-first Remember from uninformed search, this was a depth-first search where the max depth was iteratively increasedsearch where the max depth was iteratively increased

• As an informed search, we again perform depth-first search, As an informed search, we again perform depth-first search, but only nodes with f-cost less than or equal to smallest f-but only nodes with f-cost less than or equal to smallest f-cost of nodes expanded at last iterationcost of nodes expanded at last iteration

– What happens when f-cost is real-valued?What happens when f-cost is real-valued?

Iterative DeepeningIterative Deepening

• Remember from uninformed search, this was a depth-first Remember from uninformed search, this was a depth-first search where the max depth was iteratively increasedsearch where the max depth was iteratively increased

• As an informed search, we again perform depth-first search, As an informed search, we again perform depth-first search, but only nodes with f-cost less than or equal to smallest f-but only nodes with f-cost less than or equal to smallest f-cost of nodes expanded at last iterationcost of nodes expanded at last iteration

– What happens when f-cost is real-valued?What happens when f-cost is real-valued?

Recursive best-first search

Depth-first combined with best alternativeDepth-first combined with best alternative

• Keep track of options along fringeKeep track of options along fringe

• As soon as current depth-first exploration becomes more As soon as current depth-first exploration becomes more expensive of best fringe optionexpensive of best fringe option

– back up to fringe, but update node costs along the wayback up to fringe, but update node costs along the way

Depth-first combined with best alternativeDepth-first combined with best alternative

• Keep track of options along fringeKeep track of options along fringe

• As soon as current depth-first exploration becomes more As soon as current depth-first exploration becomes more expensive of best fringe optionexpensive of best fringe option

– back up to fringe, but update node costs along the wayback up to fringe, but update node costs along the way

Recursive best-first search

Recursive best-first search

Recursive best-first search

Quality of Iterative Deepening A* and Recursive best-first search

RBFSRBFS• O(bd) space complexity [if h(n) is admissible]O(bd) space complexity [if h(n) is admissible]

• Time complexity is hard to describeTime complexity is hard to describe

– efficiency is heavily dependent on quality of h(n)efficiency is heavily dependent on quality of h(n)

– same states may be explored many timessame states may be explored many times

• IDA* and RBFS use too little memoryIDA* and RBFS use too little memory

– even if you wanted to use more than O(bd) memory, these even if you wanted to use more than O(bd) memory, these two could not provide any advantagetwo could not provide any advantage

RBFSRBFS• O(bd) space complexity [if h(n) is admissible]O(bd) space complexity [if h(n) is admissible]

• Time complexity is hard to describeTime complexity is hard to describe

– efficiency is heavily dependent on quality of h(n)efficiency is heavily dependent on quality of h(n)

– same states may be explored many timessame states may be explored many times

• IDA* and RBFS use too little memoryIDA* and RBFS use too little memory

– even if you wanted to use more than O(bd) memory, these even if you wanted to use more than O(bd) memory, these two could not provide any advantagetwo could not provide any advantage

Simple Memory-bounded A*

Use all available memoryUse all available memory• Follow A* algorithm and fill memory with new expanded nodesFollow A* algorithm and fill memory with new expanded nodes

• If new node does not fitIf new node does not fit

– free() stored node with worst f-valuefree() stored node with worst f-value

– propagate f-value of freed node to parentpropagate f-value of freed node to parent

• SMA* will regenerate a subtree only when it is neededSMA* will regenerate a subtree only when it is needed

– the path through deleted subtree is unknown, but cost is the path through deleted subtree is unknown, but cost is knownknown

Use all available memoryUse all available memory• Follow A* algorithm and fill memory with new expanded nodesFollow A* algorithm and fill memory with new expanded nodes

• If new node does not fitIf new node does not fit

– free() stored node with worst f-valuefree() stored node with worst f-value

– propagate f-value of freed node to parentpropagate f-value of freed node to parent

• SMA* will regenerate a subtree only when it is neededSMA* will regenerate a subtree only when it is needed

– the path through deleted subtree is unknown, but cost is the path through deleted subtree is unknown, but cost is knownknown

Thrashing

Typically discussed in OS w.r.t. memoryTypically discussed in OS w.r.t. memory

• The cost of repeatedly freeing and regenerating parts of the The cost of repeatedly freeing and regenerating parts of the search tree dominate the cost of actual searchsearch tree dominate the cost of actual search

• time complexity will scale significantly if thrashingtime complexity will scale significantly if thrashing

– So we saved space with SMA*, but if the problem is large, So we saved space with SMA*, but if the problem is large, it will be intractable from the point of view of computation it will be intractable from the point of view of computation timetime

Typically discussed in OS w.r.t. memoryTypically discussed in OS w.r.t. memory

• The cost of repeatedly freeing and regenerating parts of the The cost of repeatedly freeing and regenerating parts of the search tree dominate the cost of actual searchsearch tree dominate the cost of actual search

• time complexity will scale significantly if thrashingtime complexity will scale significantly if thrashing

– So we saved space with SMA*, but if the problem is large, So we saved space with SMA*, but if the problem is large, it will be intractable from the point of view of computation it will be intractable from the point of view of computation timetime

Meta-foo

What does meta mean in AI?What does meta mean in AI?

• Frequently it means step back a level from fooFrequently it means step back a level from foo

• Metareasoning = reasoning about reasoningMetareasoning = reasoning about reasoning

• These informed search algorithms have pros and cons These informed search algorithms have pros and cons regarding how they choose to explore new levelsregarding how they choose to explore new levels

– a metalevel learning algorithm may combine learn how to a metalevel learning algorithm may combine learn how to combine techniques and parameterize searchcombine techniques and parameterize search

What does meta mean in AI?What does meta mean in AI?

• Frequently it means step back a level from fooFrequently it means step back a level from foo

• Metareasoning = reasoning about reasoningMetareasoning = reasoning about reasoning

• These informed search algorithms have pros and cons These informed search algorithms have pros and cons regarding how they choose to explore new levelsregarding how they choose to explore new levels

– a metalevel learning algorithm may combine learn how to a metalevel learning algorithm may combine learn how to combine techniques and parameterize searchcombine techniques and parameterize search

Heuristic Functions

8-puzzle problem8-puzzle problem8-puzzle problem8-puzzle problemAvg Depth=22

Branching = approx 3

322 states

170,000 repeated

Heuristics

The number of misplaced tiles (hThe number of misplaced tiles (h11))

• Admissible because at least n moves required to solve n Admissible because at least n moves required to solve n misplaced tilesmisplaced tiles

The distance from each tile to its goal position (hThe distance from each tile to its goal position (h22))

• No diagonals, so use No diagonals, so use Manhattan DistanceManhattan Distance

– As if walking around rectilinear city blocksAs if walking around rectilinear city blocks

• also admissiblealso admissible

The number of misplaced tiles (hThe number of misplaced tiles (h11))

• Admissible because at least n moves required to solve n Admissible because at least n moves required to solve n misplaced tilesmisplaced tiles

The distance from each tile to its goal position (hThe distance from each tile to its goal position (h22))

• No diagonals, so use No diagonals, so use Manhattan DistanceManhattan Distance

– As if walking around rectilinear city blocksAs if walking around rectilinear city blocks

• also admissiblealso admissible

Compare these two heuristics

Effective Branching Factor, b*Effective Branching Factor, b*• If A* creates If A* creates NN nodes to find the goal at depth nodes to find the goal at depth dd

– b* = branching factor such that a uniform tree of depth d contains b* = branching factor such that a uniform tree of depth d contains N+1 nodes (add 1 to N to account for root)N+1 nodes (add 1 to N to account for root)

N+1 = 1 + b* + (b*)N+1 = 1 + b* + (b*)22 + … + (b*) + … + (b*)dd

• b* close to 1 is ideal b* close to 1 is ideal

– because this means the heuristic guided the A* search linearlybecause this means the heuristic guided the A* search linearly

– If b* were 100, on average, the heuristic had to consider 100 children If b* were 100, on average, the heuristic had to consider 100 children for each nodefor each node

– Compare heuristics based on their b*Compare heuristics based on their b*

Effective Branching Factor, b*Effective Branching Factor, b*• If A* creates If A* creates NN nodes to find the goal at depth nodes to find the goal at depth dd

– b* = branching factor such that a uniform tree of depth d contains b* = branching factor such that a uniform tree of depth d contains N+1 nodes (add 1 to N to account for root)N+1 nodes (add 1 to N to account for root)

N+1 = 1 + b* + (b*)N+1 = 1 + b* + (b*)22 + … + (b*) + … + (b*)dd

• b* close to 1 is ideal b* close to 1 is ideal

– because this means the heuristic guided the A* search linearlybecause this means the heuristic guided the A* search linearly

– If b* were 100, on average, the heuristic had to consider 100 children If b* were 100, on average, the heuristic had to consider 100 children for each nodefor each node

– Compare heuristics based on their b*Compare heuristics based on their b*

Compare these two heuristics

Compare these two heuristics

hh22 is always better than h is always better than h11

• for any node, n, for any node, n, hh22(n) >= h(n) >= h11(n)(n)

• hh22 dominatesdominates h h11

• Recall all nodes with f(n) < C* will be expanded?Recall all nodes with f(n) < C* will be expanded?

– This means all nodes, h(n) + g(n) < C*, will be expandedThis means all nodes, h(n) + g(n) < C*, will be expanded

All nodes where h(n) < C* - g(n) will be expandedAll nodes where h(n) < C* - g(n) will be expanded

– All nodes hAll nodes h22 expands will also be expanded by h expands will also be expanded by h11 and and

because hbecause h11 is smaller, others will be expanded as well is smaller, others will be expanded as well

hh22 is always better than h is always better than h11

• for any node, n, for any node, n, hh22(n) >= h(n) >= h11(n)(n)

• hh22 dominatesdominates h h11

• Recall all nodes with f(n) < C* will be expanded?Recall all nodes with f(n) < C* will be expanded?

– This means all nodes, h(n) + g(n) < C*, will be expandedThis means all nodes, h(n) + g(n) < C*, will be expanded

All nodes where h(n) < C* - g(n) will be expandedAll nodes where h(n) < C* - g(n) will be expanded

– All nodes hAll nodes h22 expands will also be expanded by h expands will also be expanded by h11 and and

because hbecause h11 is smaller, others will be expanded as well is smaller, others will be expanded as well

Inventing admissible heuristic funcs

How can you create h(n)?How can you create h(n)?

• Simplify problem by reducing restrictions on actionsSimplify problem by reducing restrictions on actions

– Allow 8-puzzle pieces to sit atop on anotherAllow 8-puzzle pieces to sit atop on another

– Call this a Call this a relaxed problemrelaxed problem

– The cost of optimal solution to relaxed problem is The cost of optimal solution to relaxed problem is admissible heuristic for original problemadmissible heuristic for original problem

It is at least as expensive for the original problemIt is at least as expensive for the original problem

How can you create h(n)?How can you create h(n)?

• Simplify problem by reducing restrictions on actionsSimplify problem by reducing restrictions on actions

– Allow 8-puzzle pieces to sit atop on anotherAllow 8-puzzle pieces to sit atop on another

– Call this a Call this a relaxed problemrelaxed problem

– The cost of optimal solution to relaxed problem is The cost of optimal solution to relaxed problem is admissible heuristic for original problemadmissible heuristic for original problem

It is at least as expensive for the original problemIt is at least as expensive for the original problem

Examples of relaxed problems

A tile can move from square A tile can move from square AA to square to square BB if if

AA is horizontally or vertically adjacent to is horizontally or vertically adjacent to BB

and and BB is blank is blank

• A tile can move from A to B if A is adjacent to B (overlap)A tile can move from A to B if A is adjacent to B (overlap)

• A tile can move from A to B if B is blank (teleport)A tile can move from A to B if B is blank (teleport)

• A tile can move from A to B (teleport and overlap)A tile can move from A to B (teleport and overlap)

Solutions to these relaxed problems can be computed Solutions to these relaxed problems can be computed without search and therefore heuristic is easy to computewithout search and therefore heuristic is easy to compute

A tile can move from square A tile can move from square AA to square to square BB if if

AA is horizontally or vertically adjacent to is horizontally or vertically adjacent to BB

and and BB is blank is blank

• A tile can move from A to B if A is adjacent to B (overlap)A tile can move from A to B if A is adjacent to B (overlap)

• A tile can move from A to B if B is blank (teleport)A tile can move from A to B if B is blank (teleport)

• A tile can move from A to B (teleport and overlap)A tile can move from A to B (teleport and overlap)

Solutions to these relaxed problems can be computed Solutions to these relaxed problems can be computed without search and therefore heuristic is easy to computewithout search and therefore heuristic is easy to compute

Multiple Heuristics

If multiple heuristics available:If multiple heuristics available:

• h(n) = max {hh(n) = max {h11(n), h(n), h22(n), …, h(n), …, hmm(n)}(n)}

If multiple heuristics available:If multiple heuristics available:

• h(n) = max {hh(n) = max {h11(n), h(n), h22(n), …, h(n), …, hmm(n)}(n)}

Use solution to subproblem as heuristic

What is optimal cost of solving some portion of What is optimal cost of solving some portion of original problem?original problem?

• subproblem solution is heuristic of original problemsubproblem solution is heuristic of original problem

What is optimal cost of solving some portion of What is optimal cost of solving some portion of original problem?original problem?

• subproblem solution is heuristic of original problemsubproblem solution is heuristic of original problem

Pattern Databases

Store optimal solutions to subproblems in databaseStore optimal solutions to subproblems in database

• We use an exhaustive search to solve every permutation of the We use an exhaustive search to solve every permutation of the 1,2,3,4-piece subproblem of the 8-puzzle1,2,3,4-piece subproblem of the 8-puzzle

• During solution of 8-puzzle, look up optimal cost to solve the During solution of 8-puzzle, look up optimal cost to solve the 1,2,3,4-piece subproblem and use as heuristic1,2,3,4-piece subproblem and use as heuristic

Store optimal solutions to subproblems in databaseStore optimal solutions to subproblems in database

• We use an exhaustive search to solve every permutation of the We use an exhaustive search to solve every permutation of the 1,2,3,4-piece subproblem of the 8-puzzle1,2,3,4-piece subproblem of the 8-puzzle

• During solution of 8-puzzle, look up optimal cost to solve the During solution of 8-puzzle, look up optimal cost to solve the 1,2,3,4-piece subproblem and use as heuristic1,2,3,4-piece subproblem and use as heuristic

Learning

Could also build pattern database while solving Could also build pattern database while solving cases of the 8-puzzlecases of the 8-puzzle

• Must keep track of intermediate states and true final cost of Must keep track of intermediate states and true final cost of solutionsolution

• Inductive learningInductive learning builds mapping of state -> cost builds mapping of state -> cost

• Because too many permutations of actual statesBecause too many permutations of actual states

– Construct important Construct important features features to reduce size of spaceto reduce size of space

Could also build pattern database while solving Could also build pattern database while solving cases of the 8-puzzlecases of the 8-puzzle

• Must keep track of intermediate states and true final cost of Must keep track of intermediate states and true final cost of solutionsolution

• Inductive learningInductive learning builds mapping of state -> cost builds mapping of state -> cost

• Because too many permutations of actual statesBecause too many permutations of actual states

– Construct important Construct important features features to reduce size of spaceto reduce size of space