Heuristics & Informed Search

Heuristics & Informed Search

Hill-climbingbold & informed, heuristics, potential dangers, simulated annealing

Best first searchtentative & uninformed, BFS + hill-climbing

Algorithm Aoptimal cost solutions, admissibility, A*

Search strategies so far…Search strategies so far…bold & uninformed: STUPID

tentative & uninformed: DFS (and variants), BFS

bold & informed: hill-climbing essentially, DFS utilizing a "measure of closeness" to the goal

(a.k.a. a heuristic function)

from a given state, pick the best successor statei.e., the state with highest heuristic value

stop if no successor is better than the current state

EXAMPLE: travel problem (loc(omaha) loc(los_angeles))

heuristic(State) = -(crow-flies distance from L.A.)

Hill-climbing exampleHill-climbing examplemove(loc(omaha), loc(chicago)).move(loc(omaha), loc(denver)).move(loc(chicago), loc(denver)).move(loc(chicago), loc(los_angeles)).move(loc(chicago), loc(omaha)).move(loc(denver), loc(los_angeles)).move(loc(denver), loc(omaha)).move(loc(los_angeles), loc(chicago)).move(loc(los_angeles), loc(denver)).

heuristic(loc(omaha)) = -1700heuristic(loc(chicago)) = -2200heuristic(loc(denver)) = -1400heuristic(loc(los_angeles)) = 0

omaha

chicago denver

chicago los_angeles

heuristic value = -1700

hval= -2200 hval= -1400

hval= -2200 hval= 0

the heuristic guides the search, always moving closer to the goal

what if the flight from Denver to L.A. were cancelled?

8-puzzle example8-puzzle example

start state has 5 tiles in place1 2 3

8 6

7 5 4

hval = 5

457

368

21

457

68

321

57

468

321

hval = 4 hval = 6 hval = 6

457

628

31

457

68

321

47

658

321

hval = 4 hval = 4 hval = 5

of the 3 possible moves, 2 improve the situation

if assume left-to-right preference, move leads to a dead-end (no successor state improves the situation) so STOP!

heuristic(State) = # of tiles in place, including the space

Intuition behind hill-climbingIntuition behind hill-climbingif you think of the state space as a topographical map (heuristic value = elevation),

then hill-climbing selects the steepest gradient to move up towards the goal

potential dangers

plateau: successor states have same values, no way to choose

foothill: local maximum, can get stuck on minor peak

ridge: foothill where N-step lookahead might help

Hill-climbing variantsHill-climbing variantscould generalize hill-climbing to continue even if the successor states

look worse always choose best successor don't stop unless reach the goal or no successors

dead-ends are still possible (and likely if the heuristic is not perfect)

simulated annealing allow moves in the wrong direction on a probabilistic basis decrease the probability of a backward move as the search continues

idea: early in the search, when far from the goal, heuristic may not be good heuristic should improve as you get closer to the goal

approach is based on a metallurgical technique

Best first searchBest first searchhill-climbing is dependent on a near-perfect heuristic since bold

tentative & informed: best first search like breadth first search, keep track of all paths searched like hill-climbing, use heuristics to guide the search

always expand the "most promising" pathi.e., the path ending in a state with highest heuristic value

S1

S2 S3

hval = 0

hval = 20 hval =10

S4 S5hval = 5 hval = 8

S7hval = infinityGOAL

S6 hval = 3

Best first exampleBest first example1 2 3

8 6

7 5 4

hval = 5

457

628

31

457

68

321

47

658

321

hval = 4 hval = 4 hval = 5457

68

321

57

468

321

hval = 5 hval = 7

567

48

321

57

468

321

hval = infinity hval = 4

hval = 4 hval = 6 hval = 6457

368

21

457

68

321

57

468

321

Breadth first search vs. best first searchBreadth first search vs. best first search

procedural description of breadth first search

must keep track of all current paths, initially: [ [Start] ]

look at first path in the list of current pathsif the path ends in a goal, then DONEotherwise,

remove the first path generate all paths by extending to each possible move add the newly extended paths to the end of the list recurse

procedural description of best first search

must keep track of all current paths w/ hvals, initially: [ Hval:[Start] ]

look at first path in the list of current pathsif the path ends in a goal, then DONEotherwise,

remove the first path generate all paths by extending to each possible move add the newly extended paths to the list, sorted by (decreasing) hval recurse

Best first implementationBest first implementation%%% best.pro Dave Reed 3/15/02%%% Best first search with cycle checking%%% best(Current, Goal, Path): Path is a list of states that%%% lead from Current to Goal with no duplicate states.%%%%%% best_help(ListOfPaths, Goal, Path): Path is a list%%% of states (with associated hval) that lead from one of %%% the paths in ListOfPaths (a list of lists) to Goal with %%% no duplicate states.%%%%%% extend(Path, Goal, ListOfPaths): ListOfPaths is the list%%% of all possible paths (with associated hvals) obtainable %%% by extending Path (at the head) with no duplicate states.%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

best(State, Goal, Path) :- best_help([0:[State]], Goal, RevPath), reverse(RevPath, Path).

best_help([_:[Goal|Path]|_], Goal, [Goal|Path]).best_help([_:Path|RestPaths], Goal, SolnPath) :- extend(Path, Goal, NewPaths), append(RestPaths, NewPaths, TotalPaths), sort(TotalPaths, SortedPaths), reverse(SortedPaths,RevPaths), best_help(RevPaths, Goal, SolnPath).

extend([State|Path], Goal, NewPaths) :- bagof(H:[NextState,State|Path], (move(State, NextState), not(member(NextState, [State|Path])), heuristic(NextState,Goal,H)), NewPaths), !.extend(_, _, []).

differences from BFS

associate heuristic value with each path

H:Path

since extend needs to know heuristic values for paths, must pass Goal to extend

once the new paths have been appended, they must be sorted (in decreasing order by heuristic value)

Travel exampleTravel example%%% heuristic(Loc, Goal, Value) : Value is -distance from Goal%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

heuristic(loc(omaha), loc(los_angeles), -1700).heuristic(loc(chicago), loc(los_angeles), -2200).heuristic(loc(denver), loc(los_angeles), -1400).heuristic(loc(los_angeles), loc(los_angeles), 0).

must define the heuristic function for this problem

?- best(loc(omaha), loc(los_angeles), Path).

Path = [loc(omaha), loc(denver), loc(los_angeles)]

Yes

best first search is able to find a path

8-puzzle example8-puzzle example%%% heuristic(Board, Goal, Value) : Value is the # of tiles in place%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

heuristic(tiles(Row1,Row2,Row3),tiles(Goal1,Goal2,Goal3), Value) :- same_count(Row1, Goal1, V1), same_count(Row2, Goal2, V2), same_count(Row3, Goal3, V3), Value is V1 + V2 + V3.

same_count([], [], 0).same_count([H1|T1], [H2|T2], Count) :- same_count(T1, T2, TCount), (H1 = H2, Count is TCount + 1 ; H1 \= H2, Count is TCount).

must define the heuristic function for this problem

?- best(tiles([1,2,3],[8,6,space],[7,5,4]), tiles([1,2,3],[8,space,4],[7,6,5]), Path).

Path = [tiles([1, 2, 3], [8, 6, space], [7, 5, 4]), tiles([1, 2, 3], [8, 6, 4], [7, 5, space]), tiles([1, 2, 3], [8, 6, 4], [7, space, 5]), tiles([1, 2, 3], [8, space, 4], [7, 6, 5])]

Yes

?- best(tiles([2,3,4],[1,8,space],[7,6,5]), tiles([1,2,3],[8,space,4],[7,6,5]), Path).

Path = [tiles([2, 3, 4], [1, 8, space], [7, 6, 5]), tiles([2, 3, space], [1, 8, 4], [7, 6, 5]), tiles([2, space, 3], [1, 8, 4], [7, 6, 5]), tiles([space, 2, 3], [1, 8, 4], [7, 6, 5]), tiles([1, 2, 3], [space, 8, 4], [7, 6, 5]), tiles([1, 2|...], [8, space|...], [7, 6|...])]

Yes

best first search is able to find a path

Other examplesOther examplesmove(jugs(J1,J2), jugs(4,J2)) :- J1 < 4.move(jugs(J1,J2), jugs(J1,3)) :- J2 < 3.move(jugs(J1,J2), jugs(0,J2)) :- J1 > 0.move(jugs(J1,J2), jugs(J1,0)) :- J2 > 0.move(jugs(J1,J2), jugs(4,N)) :- J2 > 0, J1 + J2 >= 4, N is J2 - (4 - J1).move(jugs(J1,J2), jugs(N,3)) :- J1 > 0, J1 + J2 >= 3, N is J1 - (3 - J2).move(jugs(J1,J2), jugs(N,0)) :- J2 > 0, J1 + J2 < 4, N is J1 + J2.move(jugs(J1,J2), jugs(0,N)) :- J1 > 0, J1 + J2 < 3, N is J1 + J2.

heuristic function for the Water Jug Problem?

heuristic function for Missionaries & Cannibals?

Comparing best first searchComparing best first searchunlike hill-climbing, best first search can handle "dips" in the search

not so dependent on a perfect heuristic

depth first search and breadth first search may be seen as special cases of best first searchDFS: heuristic value is distance (number of moves) from start state

BFS: heuristic value is inverse of distance (number of moves) from start state

or, procedurally:DFS: assign all states the same heuristic value

when adding new paths, add equal heuristic values at the front

BFS: assign all states the same heuristic value when adding new paths, add equal heuristic values at the end

Optimizing costsOptimizing costsconsider a related problem:

instead of finding the shortest path (i.e., fewest moves) to a solution, suppose we want to minimize some cost

EXAMPLE: airline travel problem• could associate costs with each flight, try to find the cheapest route• could associate distances with each flight, try to find the shortest route

we could use a strategy similar to breadth first search repeatedly extend the minimal cost path

search is guided by the cost of the path so far

but such a strategy ignores heuristic information would like to utilize a best first approach, but not directly applicable

search is guided by the remaining cost of the path

IDEAL: combine the intelligence of both strategies cost-so-far component of breadth first search (to optimize actual cost) cost-remaining component of best first search (to make use of heuristics)

Travel problem revisitedTravel problem revisited%%% travelcost.pro Dave Reed 3/15/02%%%%%% This file contains the state space definition%%% for air travel, with costs (distances) associated%%% with each flight and heuristics providing %%% as-the-crow-flies distance.%%%%%% loc(City): defines the state where the person%%% is in the specified City.%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

move(loc(omaha), loc(chicago), 500).move(loc(omaha), loc(denver), 600).move(loc(chicago), loc(denver), 1000).move(loc(chicago), loc(los_angeles), 2200).move(loc(chicago), loc(omaha), 500).move(loc(denver), loc(los_angeles), 1400).move(loc(denver), loc(omaha), 600).move(loc(los_angeles), loc(chicago), 2200).move(loc(los_angeles), loc(denver), 1400).

heuristic(loc(omaha), loc(los_angeles), 1700).heuristic(loc(chicago), loc(los_angeles), 2200).heuristic(loc(denver), loc(los_angeles), 1400).heuristic(loc(los_angeles), loc(los_angeles), 0).

add the actual cost (number of miles per flight) to each move

heuristic is crow-flies distance (note: no longer negative, since want estimate of remaining cost)

want to minimize the total flight distance from Omaha to L.A.

omaha

denver

chicago

los_angeles

1000

1400 600

500

2200

Algorithm AAlgorithm Aassociate 2 costs with a path

g actual cost of the path so farh heuristic estimate of the remaining cost to the goalf = g + h combined heuristic cost estimate

Algorithm A (for trees – note that usual formulation is for graphs) best first search using f as the heuristic

S1

S2 S3

h = 24

h = 19f = 29

h = 4f = 24

S4 S5h = 5f = 27

h = 5f = 26

S6h = 2f = 33

10 20

2 1

10

S7

6

h = 0 GOALf = 28

Algorithm A implementationAlgorithm A implementation%%% Algorithm A (for trees)%%% atree(Current, Goal, Path): Path is a list of states%%% that lead from Current to Goal with no duplicate states.%%%%%% atree_help(ListOfPaths, Goal, Path): Path is a list of%%% states (with associated F and G values) that lead from one%%% of the paths in ListOfPaths (a list of lists) to Goal with%%% no duplicate states.%%%%%% extend(G:Path, Goal, ListOfPaths): ListOfPaths is the list%%% of all possible paths (with associated F and G values) obtainable %%% by extending Path (at the head) with no duplicate states.%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

atree(State, Goal, G:Path) :- atree_help([0:0:[State]], Goal, G:RevPath), reverse(RevPath, Path).

atree_help([_:G:[Goal|Path]|_], Goal, G:[Goal|Path]).atree_help([_:G:Path|RestPaths], Goal, SolnPath) :- extend(G:Path, Goal, NewPaths), append(RestPaths, NewPaths, TotalPaths), sort(TotalPaths, SortedPaths), atree_help(SortedPaths, Goal, SolnPath).

extend(G:[State|Path], Goal, NewPaths) :- bagof(NewF:NewG:[NextState,State|Path], Cost^H^(move(State, NextState, Cost), not(member(NextState, [State|Path])), heuristic(NextState,Goal,H), NewG is G+Cost, NewF is NewG+H), NewPaths), !.extend(_, _, []).

differences from best

associate two values with each path (F is total estimated cost, G is actual cost so far)

F:G:Path

since extend needs to know of current path, must pass G

new feature of bagof:if a variable appears only in the 2nd arg, must identify it as backtrackable

Travel exampleTravel example?- atree(loc(omaha), loc(los_angeles), Path).

Path = 2000:[loc(omaha), loc(denver), loc(los_angeles)] ;

Path = 2700:[loc(omaha), loc(chicago), loc(los_angeles)] ;

Path = 2900:[loc(omaha), loc(chicago), loc(denver), loc(los_angeles)] ;

No

note: Algorithm A finds the path with least cost (here, distance)not necessarily the path with fewest steps

suppose the flight from Chicago to L.A. was 2500 miles (instead of 2200) then OmahaChicagoDenverL.A. is shorter than OmahaChicagoL.A.

can use semi-colon to see alternative paths in order

8-puzzle example8-puzzle example

?- atree(tiles([1,2,3],[8,6,space],[7,5,4]), tiles([1,2,3],[8,space,4],[7,6,5]), Path).

Path = 3:[tiles([1, 2, 3], [8, 6, space], [7, 5, 4]), tiles([1, 2, 3], [8, 6, 4], [7, 5, space]), tiles([1, 2, 3], [8, 6, 4], [7, space, 5]), tiles([1, 2, 3], [8, space, 4], [7, 6, 5])]

Yes

?- atree(tiles([2,3,4],[1,8,space],[7,6,5]), tiles([1,2,3],[8,space,4],[7,6,5]), Path).

Path = 5:[tiles([2, 3, 4], [1, 8, space], [7, 6, 5]), tiles([2, 3, space], [1, 8, 4], [7, 6, 5]), tiles([2, space, 3], [1, 8, 4], [7, 6, 5]), tiles([space, 2, 3], [1, 8, 4], [7, 6, 5]), tiles([1, 2|...], [space, 8|...], [7, 6|...]), tiles([1|...], [8|...], [7|...])]

Yes

here, Algorithm A finds the same paths as best first search not surprising since actual cost component (g) is trivial still, not guaranteed to be the case (recall: best only cares about the future)

could apply Alg. A to the 8-puzzle problem actual cost of each move (g) is 1 remaining cost estimate (h) is # of tiles out of place

Algorithm A vs. hill-climbingAlgorithm A vs. hill-climbingif the cost estimate function h is perfect, then f is a perfect heuristic

Algorithm A is deterministic

S1

S2 S3

h = 24

h = 20f = 30

h = 8f = 28

S4 S5h = 6f = 28

h = 12f = 33

Xh = 2f = 33

10 20

2 1

10

S5

6

h = 0 GOALf = 28

X

20

X

2

h = 0 GOALf = 30

h = 0 GOALf = 33

if know actual costs for each state, Alg. A reduces to hill-climbing

AdmissibilityAdmissibilityin general, actual costs are unknown at start – must rely on heuristics

if the heuristic is imperfect, Alg. A is NOT guaranteed to find an optimal solution

S1

S2 S3

h = 2f = 2

h = 4f = 5

h = 1f = 4

1 3

X

1

h = 0 GOALf = 2

S4

1

h = 0 GOALf = 4

if a control strategy is guaranteed to find an optimal solution (when a solution exists), we say it is admissibleif cost estimate h is never overestimates actual cost, then Alg. A is admissible

(when admissible, Alg. A is commonly referred to as Alg. A*)

Admissible examplesAdmissible examplesis our heuristic for the travel problem admissible?

h (State) = crow-flies distance from L.A.

is our heuristic for the 8-puzzle admissible?

h (State) = number of tiles out of place, including the space

is our heuristic for the Missionaries & Cannibals admissible?

Cost of the searchCost of the searchthe closer h is to the actual cost, the fewer states considered

however, the cost of computing h tends to go up as it improves

also, admissibility is not always needed or desired

Graceful Decay of Admissibility:If h rarely overestimates the actual cost by more than D, then Alg. A will

rarely find a solution whose cost is more than D greater than optimal.

computationcost

closeness of h toactual cost

cost of solving problem

cost of computing h

cost of search using h

the best algorithm is one that minimizes the total cost of the solution