Post on 29-Mar-2015
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Chapter 2 Chapter 2
Fundamentals of the Analysis Fundamentals of the Analysis of Algorithm Efficiencyof Algorithm Efficiency
Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Towers of Hanoi (from Wikipedia)Towers of Hanoi (from Wikipedia)
If n>1, then somewhere along the sequence of moves, the If n>1, then somewhere along the sequence of moves, the largest disk must be moved from peg f to another peg, largest disk must be moved from peg f to another peg, preferably to peg t. preferably to peg t.
The only situation that allows this move is when all smaller The only situation that allows this move is when all smaller n-1 disks are on peg r. n-1 disks are on peg r. • First all h-1 smaller disks must go from f to r. First all h-1 smaller disks must go from f to r.
• Move the largest disk Move the largest disk
• Finally move the h-1 smaller disks from peg r to peg t. Finally move the h-1 smaller disks from peg r to peg t.
Now the problem is reduced to moving h-1 disks from one Now the problem is reduced to moving h-1 disks from one peg to another one, first from f to r and subsequently from peg to another one, first from f to r and subsequently from r to t r to t • The same method can be used both times by renaming the pegs. The same method can be used both times by renaming the pegs.
• The same strategy can be used to reduce the h-1 problem to h-2, h-The same strategy can be used to reduce the h-1 problem to h-2, h-3, and so on until only one disk is left. This is called 3, and so on until only one disk is left. This is called recursion..
1-2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Towers of HanoiTowers of Hanoi
The following is a procedure for moving a tower of h disks The following is a procedure for moving a tower of h disks from a peg f onto a peg t, with r being the remaining third from a peg f onto a peg t, with r being the remaining third peg:peg:• Step 1: If h>1 then first move the h-1 smaller disks from peg f to Step 1: If h>1 then first move the h-1 smaller disks from peg f to
peg r.peg r.
• Step 2: Now the largest disk, i.e. disk h-1 can be moved from peg f Step 2: Now the largest disk, i.e. disk h-1 can be moved from peg f to peg t.to peg t.
• Step 3: If h>1 then again use this procedure to move the h-1 smaller Step 3: If h>1 then again use this procedure to move the h-1 smaller disks from peg r to peg t.disks from peg r to peg t.
1-3
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Example 2: The Tower of Hanoi PuzzleExample 2: The Tower of Hanoi Puzzle
http://en.wikipedia.org/wiki/Tower_of_Hanoi http://www.cut-the-knot.org/recurrence/hanoi.shtml Recursive Solution (Move n>1 disks from P1 to P3
• Move recursively n-1 disks from P1 to P2 (P3 is Aux)
• Then move the largest disk from P1 to P3
• Move recursively n-1 disks from P2 to P3 (P1 is Aux)
Recursive Algorithm Analysis• M(n) = M(n-1) + 1 + M(n-1) for n>1 , M(1) = 1
• M(n) = 2M(n-1)+1, M(1) = 1
1-4
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Example 2: The Tower of Hanoi PuzzleExample 2: The Tower of Hanoi Puzzle
1
2
3
Recurrence for number of moves:Recurrence for number of moves:
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Solving recurrence for number of movesSolving recurrence for number of moves
M(M(nn) = 2M() = 2M(nn-1) + 1, M(1) = 1-1) + 1, M(1) = 1
Use Backward SubstitutionUse Backward Substitution
M(n) = 2M(n-1)+1M(n) = 2M(n-1)+1
=2[2M(n-2)+1]+1 = 2=2[2M(n-2)+1]+1 = 222M(n-2)+2+1M(n-2)+2+1
=2=222[2M(n-3)+1]+2+1 [2M(n-3)+1]+2+1 = 2= 233M(n-3)+2M(n-3)+222+2+1+2+1
PatternPattern
M(n) = 2M(n) = 2iiM(n-i)+2M(n-i)+2i-1i-1+2+2i-2i-2+…+2+1+…+2+1
= 2= 2iiM(n-i)+2M(n-i)+2ii-1-1
Initial condition: n=1 or i = n-1 Initial condition: n=1 or i = n-1
= 2= 2nn-1 -1 exponential algorithm : still optimal exponential algorithm : still optimal
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Tree of calls for the Tower of Hanoi PuzzleTree of calls for the Tower of Hanoi Puzzle
n
n-1 n-1
n-2 n-2 n-2 n-2
1 1
... ... ...2
1 1
2
1 1
2
1 1
2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Example 3: Counting #bitsExample 3: Counting #bits
Basic operation: additions Recurrence relation:
A(n) = A(floor(n/2)) + 1 for n>1, A(1) = 0
Problem: floor(n/2) makes backward substitution not work when n is not a power of 2Estimate: for only cases where n is a power of 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Slightly Updated RecurrenceSlightly Updated Recurrence
A(2A(2kk) = A(2) = A(2k-1k-1) + 1 for k>0, A(2) + 1 for k>0, A(200) = 0) = 0
Use backward substitutionUse backward substitutionA(2A(2kk) = A(2) = A(2k-1k-1) + 1) + 1
= [A(2= [A(2k-2k-2) + 1]+1 = A(2) + 1]+1 = A(2k-2k-2) + 2) + 2
= [A(2= [A(2k-3k-3) + 1]+2 = A(2) + 1]+2 = A(2k-3k-3) + 3) + 3
……
= A(2= A(2k-ik-i) + i) + i
……
= A(2= A(2k-kk-k) + k) + k
Thus,Thus,
A(2A(2kk) = A(1) + k = k) = A(1) + k = k
n=2n=2kk and hence k = log and hence k = log22nn A(n) = log2n = A(n) = log2n = ΘΘ(log n)(log n) 1-9
Chapter 3Chapter 3
Brute ForceBrute Force
Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Brute ForceBrute Force
A straightforward approach, usually based directly on the A straightforward approach, usually based directly on the problem’s statement and definitions of the concepts involvedproblem’s statement and definitions of the concepts involved
Just do it, and don’t be fancyJust do it, and don’t be fancy
Examples:Examples: Computing Computing aan n ((a a > 0, > 0, nn a nonnegative integer) a nonnegative integer)
Computing Computing nn!!
Multiplying two matricesMultiplying two matrices
Searching for a key of a given value in a listSearching for a key of a given value in a list
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
So, When and Why Brute ForceSo, When and Why Brute Force
Why not brute force?Why not brute force?• You’ll never get an elegant algorithmYou’ll never get an elegant algorithm
• You’ll never give yourself a chance to see the hidden patterns You’ll never give yourself a chance to see the hidden patterns necessary for getting an optimal solutionnecessary for getting an optimal solution
• For problems where solutions that are better than all pairs exist, For problems where solutions that are better than all pairs exist, brute force will not be optimalbrute force will not be optimal
Why brute force?Why brute force?• Exceedingly easy to conceptualizeExceedingly easy to conceptualize
• The method can be applied to a large range of problems (not true The method can be applied to a large range of problems (not true for a lot of the other strategies we’ll see)for a lot of the other strategies we’ll see)
• Works well for small values (input sizes) and its easy to implementWorks well for small values (input sizes) and its easy to implement
• Expense of a more advanced algorithm may not be justifiedExpense of a more advanced algorithm may not be justified
– Only a few difficult instances to solveOnly a few difficult instances to solve• Makes a good yard stickMakes a good yard stick
– i.e. We can do at least this well very easilyi.e. We can do at least this well very easily
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Brute-Force Sorting AlgorithmBrute-Force Sorting Algorithm
Selection SortSelection Sort Scan the array to find its smallest element and Scan the array to find its smallest element and swap it with the first element. Then, starting with the second swap it with the first element. Then, starting with the second element, scan the elements to the right of it to find the element, scan the elements to the right of it to find the smallest among them and swap it with the second elements. smallest among them and swap it with the second elements. Generally, on pass Generally, on pass i i (0 (0 i i n-n-2), find the smallest element in 2), find the smallest element in AA[[i..n-i..n-1] and swap it with 1] and swap it with AA[[ii]:]:
AA[0] [0] . . . . . . AA[[ii-1] | -1] | AA[[ii], . . . , ], . . . , AA[[minmin], . . ., ], . . ., AA[[nn--1] 1]
in their final positionsin their final positions
Example: 7 3 2 5Example: 7 3 2 5
Example: 89 45 68 90 29 34 17Example: 89 45 68 90 29 34 17
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Analysis of Selection SortAnalysis of Selection Sort
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
AnalysisAnalysis
Time efficiency:Time efficiency:• CC (n) = (n) = i=0 to n-2i=0 to n-2j=i+1 to n-1j=i+1 to n-111
= = i=0 to n-2i=0 to n-2[(n-1)-(i+1)+1][(n-1)-(i+1)+1]
= = i=0 to n-2i=0 to n-2(n-1-i)(n-1-i)
= (n-1) + (n-2) + (n-3) + … + 1= (n-1) + (n-2) + (n-3) + … + 1
by (S2)by (S2)
= ((n-1)n)/2 = (n= ((n-1)n)/2 = (n22 – n) / 2 – n) / 2
approx. = napprox. = n22/2 /2 O(n O(n22))
Space efficiency:Space efficiency:
Stability:Stability:
Note: # swaps is only Θ(n), or exactly n-1 -Is this good or bad?
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Brute-Force Sorting Algorithm #2Brute-Force Sorting Algorithm #2
Bubble SortBubble Sort Scan the array looking at each consecutive Scan the array looking at each consecutive pair of elements. Swap the two elements if they are out of pair of elements. Swap the two elements if they are out of order. Repeatedly scan the array in an all-pairs manner.order. Repeatedly scan the array in an all-pairs manner.
See pg 100 for the pseudocodeSee pg 100 for the pseudocode
Example:Example:• 7 3 2 57 3 2 5
• 89 45 68 90 29 34 1789 45 68 90 29 34 17
Note: we can do better by stopping if no swaps are neededNote: we can do better by stopping if no swaps are needed• i.e. – the list is sortedi.e. – the list is sorted
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
AnalysisAnalysis
Time efficiency:Time efficiency:• CC (n) = (n) = i=0 to n-2i=0 to n-2j=0 to n-2-ij=0 to n-2-i11
= = i=0 to n-2i=0 to n-2[(n-2-i)-0+1][(n-2-i)-0+1]
= = i=0 to n-2i=0 to n-2(n-1-i)(n-1-i)
= (n-1) + (n-2) + (n-3) + … + 1= (n-1) + (n-2) + (n-3) + … + 1
by (S2)by (S2)
= ((n-1)n)/2 = (n= ((n-1)n)/2 = (n22 – n) / 2 – n) / 2
approx. = napprox. = n22/2 /2 O(n O(n22))
Space efficiency:Space efficiency:
Stability:Stability:
Note: How many swaps must take place in the worst case?
That is, look at the code for Selection Sort and Bubble Sort and note where the swap commands are.
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Improving Brute ForceImproving Brute Force
With brute force, its often possible to do small things that With brute force, its often possible to do small things that decrease the run-timedecrease the run-time• These don’t generally change the asymptotic boundsThese don’t generally change the asymptotic bounds
Example: Sequential SearchExample: Sequential Search• Make sure that you have the value you are looking for appended to Make sure that you have the value you are looking for appended to
the end of the listthe end of the list
– That way, the item will have to be found and we That way, the item will have to be found and we won’t have to constantly check for the end of the listwon’t have to constantly check for the end of the list
• Obviously, we could first sort our listObviously, we could first sort our list
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Brute-Force String MatchingBrute-Force String Matching
patternpattern: a string of : a string of mm characters to search for characters to search for texttext: a (longer) string of : a (longer) string of nn characters to search in characters to search in problem: find a substring in the text that matches the patternproblem: find a substring in the text that matches the pattern
Brute-force algorithmBrute-force algorithm
Step 1 Align pattern at beginning of textStep 1 Align pattern at beginning of text
Step 2 Moving from left to right, compare each character ofStep 2 Moving from left to right, compare each character of pattern to the corresponding character in text until pattern to the corresponding character in text until
– all characters are found to match (successful search); orall characters are found to match (successful search); or
– a mismatch is detecteda mismatch is detected
Step 3 While pattern is not found and the text is not yetStep 3 While pattern is not found and the text is not yet exhausted, realign pattern one position to the right and exhausted, realign pattern one position to the right and repeat Step 2 repeat Step 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Examples of Brute-Force String MatchingExamples of Brute-Force String Matching
• Pattern: Pattern: 001011 001011 Text: Text: 10010101101001100101111010 10010101101001100101111010
• Pattern: Pattern: happyhappy
Text: Text: It is never too late to have a It is never too late to have a happy childhood.happy childhood.
b Pattern: Pattern: happyhappy Text: happhapphapphappy Text: happhapphapphappy
b Pattern: aaabPattern: aaab
Text: aaaaaaaaaaaaaaaaaaaaaaaaaaaabText: aaaaaaaaaaaaaaaaaaaaaaaaaaaab
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Pseudocode and Efficiency Pseudocode and Efficiency
Efficiency: Worst Case?Efficiency: Worst Case?
Expected Average?Expected Average?
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Brute-Force Polynomial EvaluationBrute-Force Polynomial Evaluation
Problem: Find the value of polynomialProblem: Find the value of polynomial
pp((xx) = ) = aannxxnn + + aann-1-1xxnn-1 -1 +… ++… + a a11xx1 1 + + aa0 0
at a point at a point xx = = xx00
Brute-force algorithmBrute-force algorithm
Efficiency:Efficiency:
pp 0.00.0forfor ii nn downtodownto 0 0 dodo powerpower 1 1
forfor jj 1 1 toto ii dodo //compute //compute xxii powerpower powerpower xx pp pp + + aa[[ii] ] powerpower
returnreturn pp
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Polynomial Evaluation: ImprovementPolynomial Evaluation: Improvement
We can do better by evaluating from right to left:We can do better by evaluating from right to left:
pp((xx) = ) = aannxxnn + + aann-1-1xxnn-1 -1 +… ++… + a a11xx1 1 + + aa0 0
at a point at a point xx = = xx00
Better brute-force algorithmBetter brute-force algorithm
Efficiency:Efficiency:
pp aa[0][0]powerpower 1 1forfor ii 1 1 toto nn dodo
powerpower powerpower x x pp p p + + aa[[ii] ] powerpowerreturnreturn pp