Introduction to Algorithms: Brute-Force...
Transcript of Introduction to Algorithms: Brute-Force...
Introduction to Algorithms:Brute-Force Algorithms
Introduction to Algorithms
Brute Force• Powering a Number• Selection Sort• Exhaustive Search
• 0/1 Knapsack Problem• Assignment Problem
CS 421 - Analysis of Algorithms2
Brute-Force Algorithm Design
CS 421 - Analysis of Algorithms3
• Straightforward, usually based on problem definition.
• Rarely the most efficient but can be applied to wide range of problems.
• For some elementary problems, almost as good as most efficient. May not be worth cost.
• May work just as well on smaller data sets.• Used to measure other algorithms against.
Calculating Powers of a Number
CS 421 - Analysis of Algorithms
Problem: Compute a n, where n ∈N.
Naive algorithm: Θ(n).
a n = n * n * n * … * n
4
a times
Selection Sort
CS 421 - Analysis of Algorithms
Given a list of n orderable items, rearrange them in non-decreasing order.1. Scan entire list to find smallest item.2. Exchange it with first item. First element now in its
final, sorted position.3. Scan remaining n – 1 items, starting with second
element, and find smallest item.4. Exchange it with second item. Second element now in
its final, sorted position. 5. Repeat for a total of n – 1 times.
5
Selection Sort -Example
CS 421 - Analysis of Algorithms
Selection sort on the list: 89, 45, 68, 90, 29, 34, 17. Each line corresponds to an iteration of the algorithm. The values in bold are the smallest item for that iterations. Elements to the left of the vertical bar are in their final positions.
6
| 89 45 68 90 29 34 1717 |45 68 90 29 34 8917 29 | 68 90 45 34 8917 29 34 | 90 45 68 8917 29 34 45 | 90 68 8917 29 34 45 68 | 90 8917 29 34 45 68 89 | 90
Selection Sort -Analysis
CS 421 - Analysis of Algorithms
Selection sort is implemented using nested forloops:
Outer loop: iterates from 0 to n – 2 – don’t have to visit element because already in final sorted position
Inner loop: finds smallest value remaining the list. Even though gets smaller each iteration, still on order of n.Therefore:
T(n) = Θ(n2)7
Sequential search
In a list No better than linear timeWe can do better if not a listWorst case O(n)
CS 421 - Analysis of Algorithms 8
Exhaustive Search
CS 421 - Analysis of Algorithms9
Brute-force approach to combinatorial problems (i.e. permutations, combinations, subsets of a given set).1. Generate all elements of problem domain (all
possible solutions).2. Select those that satisfy the constraints of the
problem.3. Chose one or more that are most desirable (i.e.
optimize the objective function).
0/1 Knapsack ProblemAnalysis
CS 421 - Analysis of Algorithms10
The most costly operation is generating all of the subsets of n items. Since there are 2n subsets,
Brute-force Approach to Knapsack problem: Ω(2n).
Not feasible for any but the smallest values of n.
0-1 knapsack Algorithmpublic boolean knapsack (double [] v, double V, double [] w, double W,
double counterV, double counterW, int i){if(i<v.length){
boolean b1=knapsack(v,V,w,W, counterV,counterW, i+1);boolean b2=knapsack(v,V,w,W,counterV+v[i],counterW+w[i], i+1);return b1||b2;
}else{return (counterV>= V) && (counterW<= W)
}}
T(n)=2T(n-1) +1T(n-1)<=C*2^{n-1}T(n)<=C*2*2^(n-1)+1T(n)<=C*2*2^(n-1)+1<=C*2^n
0-1 knapsack Algorithmpublic boolean knapsack (double [] v, double V, double [] w, double W,
double counterV, double counterW, int i){if(i<v.length){
boolean b1=knapsack(v,V,w,W, counterV,counterW, i+1);boolean b2=knapsack(v,V,w,W,counterV+v[i],counterW+w[i], i+1);return b1||b2;
}else{return (counterV>= V) && (counterW<= W)
}}
T(n)=2T(n-1) +1T(n-1)<=C*2^{n-1}T(n)<=C*2*2^(n-1)+1T(n)<=C*2*2^(n-1)+1<=C*2^n
T(n)=2T(n-1) +1T(n-1)<=C*(n-1)2^{n-1}T(n)<=C*2*2^(n-1)*(n-1)+1T(n)<=C*2*2^(n-1)*(n-1)+1<=C*n*2^n
0/1 Knapsack Problem
CS 421 - Analysis of Algorithms13
Given n items of known weights w1, w2, …, wn and values v1, v2, …, vn , and a knapsack of capacity W, find most valuable subset of items that will fit.
1. Generate all subsets of n items.2. Calculate the weights of each and eliminate all
the infeasible solutions.3. Find the subset with the maximum value.
Assignment Problem
CS 421 - Analysis of Algorithms 14
There are n jobs that need to be completed, and n people that need to be assigned to a job, one person per job. The cost for the i th person to perform job j is known C[i, j]. Find the assignment with the lowest cost.
1. Generate all permutations of n people assigned to n jobs.
2. Calculate the cost of each permutation/solution.3. Find the solution with the minimum value.
Assignment ProblemAnalysis
CS 421 - Analysis of Algorithms15
The most costly operation is generating all of the permutations. Since there are n! permutations,
Brute-force Approach to Assignment problem: Θ(n!).
Not feasible for any but the smallest values of n.
String Matching
CS 421 - Analysis of Algorithms 16
String Matching
CS 421 - Analysis of Algorithms 17
String Matching
CS 421 - Analysis of Algorithms 18
Complexity
m(n-m+1) O(nm)
CS 421 - Analysis of Algorithms 19
Travelling salesman problem
CS 421 - Analysis of Algorithms 20
Travelling salesman problem
CS 421 - Analysis of Algorithms 21
Path={(A,B),(B,C),(C,D),(D,A)}
Travelling salesman problem
CS 421 - Analysis of Algorithms 22
Path={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}
Hamiltonian Path
CS 421 - Analysis of Algorithms 23
Path={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}
Hamiltonian Path
CS 421 - Analysis of Algorithms 24
Path={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}
A Hamiltonian path or traceable path is a path that visits each vertex exactly once
Hamiltonian Path
CS 421 - Analysis of Algorithms 25
Path={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}
A Hamiltonian path or traceable path is a path that visits each vertex exactly once
Hamiltonian Path
CS 421 - Analysis of Algorithms 26
Path*={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}
A Hamiltonian path or traceable path is a path that visits each vertex exactly once
Optimal
Graph Coloring
CS 421 - Analysis of Algorithms 27
Binary counter
CS 421 - Introduction to Algorithms 28
When increment up to n, change is at most 1+floor(log n) bits -number of bits in the binary representation of n.
Thus, the cost of counting up to n, a sequence of nincrements, is O(n log n).
Counter value
A[k] A[k-1] … A[2] A[1] A[0] Cost
0 0 0 … 0 0 0 0
1 0 0 … 0 0 1 1
2 0 0 … 0 1 0 2
3 0 0 … 0 1 1 1
4 0 0 … 1 0 0 3
5 0 0 … 1 0 1 1
6 0 0 … 1 1 0 2
7 0 0 … 1 1 1 1
8 0 0 … 0 0 0 4
… … … … … … …
2k - 1 1 1 … 1 1 1 k
Binary counter
CS 421 - Analysis of Algorithms 29
454 Chapter 17 Amortized Analysis
analysis, we assign the amortized cost of each operation to be the average cost. Inthis example, therefore, all three stack operations have an amortized cost of O.1/.
We emphasize again that although we have just shown that the average cost, andhence the running time, of a stack operation is O.1/, we did not use probabilisticreasoning. We actually showed a worst-case bound of O.n/ on a sequence of noperations. Dividing this total cost by n yielded the average cost per operation, orthe amortized cost.
Incrementing a binary counterAs another example of aggregate analysis, consider the problem of implementinga k-bit binary counter that counts upward from 0. We use an array AŒ0 : : k ! 1! ofbits, where A: length D k, as the counter. A binary number x that is stored in thecounter has its lowest-order bit in AŒ0! and its highest-order bit in AŒk ! 1!, so thatx D
Pk!1iD0 AŒi ! " 2i . Initially, x D 0, and thus AŒi ! D 0 for i D 0; 1; : : : ; k ! 1. To
add 1 (modulo 2k) to the value in the counter, we use the following procedure.
INCREMENT.A/
1 i D 02 while i < A: length and AŒi ! == 13 AŒi ! D 04 i D i C 15 if i < A: length6 AŒi ! D 1
Figure 17.2 shows what happens to a binary counter as we increment it 16 times,starting with the initial value 0 and ending with the value 16. At the start ofeach iteration of the while loop in lines 2–4, we wish to add a 1 into position i .If AŒi ! D 1, then adding 1 flips the bit to 0 in position i and yields a carry of 1,to be added into position i C 1 on the next iteration of the loop. Otherwise, theloop ends, and then, if i < k, we know that AŒi ! D 0, so that line 6 adds a 1 intoposition i , flipping the 0 to a 1. The cost of each INCREMENT operation is linearin the number of bits flipped.
As with the stack example, a cursory analysis yields a bound that is correct butnot tight. A single execution of INCREMENT takes time ‚.k/ in the worst case, inwhich array A contains all 1s. Thus, a sequence of n INCREMENT operations onan initially zero counter takes time O.nk/ in the worst case.
We can tighten our analysis to yield a worst-case cost of O.n/ for a sequence of nINCREMENT operations by observing that not all bits flip each time INCREMENTis called. As Figure 17.2 shows, AŒ0! does flip each time INCREMENT is called.The next bit up, AŒ1!, flips only every other time: a sequence of n INCREMENT
NP-Hard
NP-HardOptimized approach:n Cut n Branch and Bound
CS 421 - Analysis of Algorithms 30
Conclusion
CS 421 - Analysis of Algorithms
The Brute Force approach:• Straightforward approach to solving
problem, usually based directly on problem description.
• Wide applicability and simple.• But in general, has subpar performance.
31