Search and Optimization Strategies

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)


Search and Optimization Strategies Topics: Definitions Branch & Bound Greedy Local Search Teaching material for the course of "Tecniche di Programmazione" at Politecnico di Torino in year 2012/2013. More information:

Transcript of Search and Optimization Strategies

  • 1. Search and Optimization StrategiesTecniche di Programmazione A.A. 2012/2013
  • 2. SummaryA.A. 2012/2013Tecniche di programmazione2 Definitions Branch & Bound Greedy Local Search
  • 3. DefinitionsSearch and Optimization Strategies
  • 4. Search vs OptimizationA.A. 2012/2013Tecniche di programmazione4 A search algorithm is an algorithm for finding an itemwith specified properties among a collection of items Specified properties: usually local properties, defined by somefeasibility criteria Collection: explicit, or virtual (implicit) An optimization algorithm aims ad finding the bestpossible solution, according to an objective function,among all possible feasible solutions for a givenproblem
  • 5. Objective functionA.A. 2012/2013Tecniche di programmazione5 Must be computable for all feasible solutions May be undefined for unfeasible (or incomplete) solutions Maximization problems Objective function = value, goodness Must find the maximum value of objective function Minimization problems Objective function = cost Must find the minimum value of objective function
  • 6. A.A. 2012/2013Tecniche di programmazione6Types of problems Search problems: is there (at least one) feasible solution? Ex: magical square Ex: Hamiltonian cycle Optimization problems: what is the best solution? In these problems finding feasible solutions is usually easy Ex: Hamiltonian cycle on a complete graph (Hamiltonan bee) Hybrid problems (search + optimization): are there any validsolutions? And what is the best? Ex: travelling salesman problem
  • 7. A.A. 2012/2013Tecniche di programmazione7RepresentationS: SolutionsV:Valid (feasible) solutionsM: best solutionssssf(s)Objectivefunction
  • 8. A.A. 2012/2013Tecniche di programmazione8Classification Search problems S V |V| = 0 ? Find a s V Optimization problems S =V find max(f(s)) Hybrid problems S V find max(f(s)) such that s V
  • 9. Exact techniquesA.A. 2012/2013Tecniche di programmazione9 Exhaustive generation of all possible solutions Iteratively Recursively Optimizing recursive techniques Visit order Generate non-useless solutions: Branch and Bound Recognize equivalent sub-problems: Dynamic Programming1 23 45 67 8
  • 10. Approximate TecniquesA.A. 2012/2013Tecniche di programmazione10 Greedy Pseudo-random Hill climber123
  • 11. Approximate solutionsA.A. 2012/2013Tecniche di programmazione11 Optimum solution: the absolute best Optimal solution: a solution that approximates the optimum one (might be coincident, but wedont know) can no longer be improved with the chosen optimizationtechnique Locally optimum solution: optimum solution in acontinuous domain
  • 12. IntuitivelyA.A. 2012/2013Tecniche di programmazione12xf(x)OptimumsolutionOptimalsolutionLocallyoptimumsolutions
  • 13. Computational IntelligenceA.A. 2012/2013Tecniche di programmazione13 Evolutionary Algorithms Genetic Algorithms Genetic Programming Simulated annealing Tabu-Search
  • 14. Branch & BoundSearch and Optimization Strategies
  • 15. A.A. 2012/2013Tecniche di programmazione15Introduction Branch and Bound is an exact method for exploring allsolutions based on their implicit enumeration All solutions are considered, but not one-by-one. In factB&B: Considers disjoints subsets of solutions (branching) Evaluates them according to an estimate of the objectivefunction (bounding), by eliminating (pruning) those subsets thatmay not contain the optimum solution
  • 16. A.A. 2012/2013Tecniche di programmazione16Definition z*: a sub-optimal estimate of the final result Initialized to + for minimization problems Initialized to - for maximization problems z(r)E : the exact solution of sub-problem P(r) z(r)B: a super-optimal estimate of solutions of sub-problemP(r) lower bound for minimization problems upper bound for maximization problems
  • 17. A.A. 2012/2013Tecniche di programmazione17General algorithm (I) Branch-and-bound is a divide-et-impera method: Divide a given problem (and its set of feasible solutions) in sub-problems to be analyzed separately. Keep a list P of open sub-problems At each step, select one current problem P(r)
  • 18. General algorithm (II)A.A. 2012/2013Tecniche di programmazione18 If P(r) has no feasible solutions, it is closed If P(r) has (at least one) solution, and may be solved up tothe optimum z(r)E P(r) is closed Its solution, with value z(r)E, may possibly replace z*, if itimproves it
  • 19. General algorithm (III)A.A. 2012/2013Tecniche di programmazione19 Se P(r) has solutions, but its difficult to be solved up tothe optimum, we try to show that it may not yield abetter solution than the already known ones. We compute a super-optimal estimate z(r)B of its solutions: If z(r)B is not better than z*, all solutions in P(r) are dominatedby the best known-solution, and P(r) may be closed If z(r)B is better than z*, P(r) is broken into sub-problems thatare inserted in the list P. P(r) is closed, and the sub-problemswill be opened, eventually.
  • 20. General algorithm (IV)A.A. 2012/2013Tecniche di programmazione20 Proceed until P is empty When P is empty, then z* is the exact optimum solutionfor the original problem.
  • 21. The solution treeA.A. 2012/2013Tecniche di programmazione21P0P1 P2P21 P22P11 P12P111 P112P12Node: sub-problemLeaf node: closed sub-problem
  • 22. Ingredients of a B&B algorithmA.A. 2012/2013Tecniche di programmazione22 Strategy of visit of the search tree: criteria to choose thenext sub-problem to be analyzed out of the list P Bounding technique: how to evaluate the super-optimalestimate z(r)B Branching rule: how to split the current problem P(r) intosmaller sub-problems.
  • 23. Main featuresA.A. 2012/2013Tecniche di programmazione23 Advantages: Finds optimum solution May limit the number of visited solutions May give a partial (approximate) solution, if stopped earlier Disadvantages: Computation time for visiting the solution tree depends on thestrategy The worst case is to visit all possible nodes, anyway Time needed to compute z(r)B.
  • 24. GreedySearch and Optimization Strategies
  • 25. A.A. 2012/2013Tecniche di programmazione55Introduction A greedy algorithm searches for a globally optimalsolution, by choosing locally optimum ones. The adopted technique is shortsighted: Every choice made at any step is never re-considered again No back-tracking Basic principle: define an attractiveness for every partialsolution Attractiveness (it: appetibilit) = estimate of how much, probably,we are close to the optimum.
  • 26. General structure (I)A.A. 2012/2013Tecniche di programmazione56 If the attractiveness {ai} of partial solutions are knownsince the beginning, and may not be modifiedGreedy ({a1, a2, an})S sort {ai} in decreasing attractiveness orderfor each ai doif ai may be added to Sthen S S {ai}return S
  • 27. General structure (II)A.A. 2012/2013Tecniche di programmazione57 If attractiveness may be modified by the effects ofprevious choicesGreedy2 ({a1, a2, an})S compute attractiveness of aiwhile there are element to choose dochoose the most-attractive aiif ai may be added to Sthen S S {ai}update the remaining aireturn S
  • 28. ExampleA.A. 2012/2013Tecniche di programmazione58 Find the minimum number of coins to give change to acustomer. We assume we have a list of possible coin values, andtheir availability is infinite
  • 29. A.A. 2012/2013Tecniche di programmazione59Example Available coins 20, 10, 5, 2, 1 Change to give 55
  • 30. A.A. 2012/2013Tecniche di pr