Design and Analysis of Algorithms – Chapter 51 Divide and Conquer (I) Dr. Ying Lu [email protected]...
-
Upload
elfrieda-hancock -
Category
Documents
-
view
223 -
download
0
Transcript of Design and Analysis of Algorithms – Chapter 51 Divide and Conquer (I) Dr. Ying Lu [email protected]...
Design and Analysis of Algorithms – Chapter 5 1
Divide and Conquer (I)
Dr. Ying [email protected]
RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms
Design and Analysis of Algorithms – Chapter 5 2
Giving credit where credit is due:Giving credit where credit is due:• Most of the lecture notes are based on the slides from Most of the lecture notes are based on the slides from
the Textbook’s companion websitethe Textbook’s companion website– http://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin
• Some examples and slides are based on lecture notes Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University created by Dr. Ben Choi, Louisiana Technical University and Dr. Chuck Cusack, Hope College and Dr. Chuck Cusack, Hope College
• I have modified many of their slides and added new I have modified many of their slides and added new slides.slides.
RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms
Design and Analysis of Algorithms – Chapter 5 3
Divide and ConquerDivide and Conquer
The most well known algorithm design strategy:The most well known algorithm design strategy:1.1. Divide instance of problem into two or more smaller Divide instance of problem into two or more smaller
instancesinstances
2.2. Solve smaller instances recursivelySolve smaller instances recursively
3.3. Obtain solution to original (larger) instance by combining Obtain solution to original (larger) instance by combining these solutionsthese solutions
Design and Analysis of Algorithms – Chapter 5 4
Divide-and-conquer TechniqueDivide-and-conquer Technique
subproblem 2 of size n/2
subproblem 1 of size n/2
a solution to subproblem 1
a solution tothe original problem
a solution to subproblem 2
a problem of size n
Design and Analysis of Algorithms – Chapter 5 5
Divide and Conquer ExamplesDivide and Conquer Examples
Sorting: mergesort and quicksortSorting: mergesort and quicksort
Tree traversalsTree traversals
Matrix multiplication-Strassen’s algorithmMatrix multiplication-Strassen’s algorithm
Closest pair problemClosest pair problem
Convex hull-QuickHull algorithmConvex hull-QuickHull algorithm
Review MergesortReview Mergesort
Design and Analysis of Algorithms – Chapter 5 6
Design and Analysis of Algorithms – Chapter 5 7
MergesortMergesort
Algorithm:Algorithm: Split array A[1..Split array A[1..nn] in two and make copies of each half in ] in two and make copies of each half in
arrays B[1.. arrays B[1.. nn/2 ] and C[1.. /2 ] and C[1.. nn/2 ]/2 ] Sort arrays B and CSort arrays B and C Merge sorted arrays B and C into array AMerge sorted arrays B and C into array A
Design and Analysis of Algorithms – Chapter 5 8
Using Divide and Conquer: Using Divide and Conquer: MergesortMergesort
Mergesort StrategyMergesort Strategy
Sorted
Merge
Sorted Sorted
Sort recursively by Mergesort
Sort recursively by Mergesort
first last
(first last)2
Design and Analysis of Algorithms – Chapter 5 9
MergesortMergesort
Algorithm:Algorithm: Split array A[1..Split array A[1..nn] in two and make copies of each half in ] in two and make copies of each half in
arrays B[1.. arrays B[1.. nn/2 ] and C[1.. /2 ] and C[1.. nn/2 ]/2 ] Sort arrays B and CSort arrays B and C Merge sorted arrays B and C into array AMerge sorted arrays B and C into array A
Design and Analysis of Algorithms – Chapter 5 10
MergesortMergesort
Algorithm:Algorithm: Split array A[1..Split array A[1..nn] in two and make copies of each half in ] in two and make copies of each half in
arrays B[1.. arrays B[1.. nn/2 ] and C[1.. /2 ] and C[1.. nn/2 ]/2 ]
Sort arrays B and CSort arrays B and C Merge sorted arrays B and C into array A as follows:Merge sorted arrays B and C into array A as follows:
• Repeat the following until no elements remain in one of the arrays:Repeat the following until no elements remain in one of the arrays:
– compare the first elements in the remaining unprocessed compare the first elements in the remaining unprocessed portions of the arraysportions of the arrays
– copy the smaller of the two into A, while incrementing the index copy the smaller of the two into A, while incrementing the index indicating the unprocessed portion of that array indicating the unprocessed portion of that array
• Once all elements in one of the arrays are processed, copy the Once all elements in one of the arrays are processed, copy the remaining unprocessed elements from the other array into A.remaining unprocessed elements from the other array into A.
Design and Analysis of Algorithms – Chapter 5 11
Algorithm: MergesortAlgorithm: Mergesort
Input:Input: Array E and indices first and last, such that the Array E and indices first and last, such that the elements E[i] are defined for first elements E[i] are defined for first i i last. last.
Output:Output: E[first], …, E[last] is a sorted rearrangement of E[first], …, E[last] is a sorted rearrangement of the same elementsthe same elements
void mergeSort(Element[] E, int first, int last)void mergeSort(Element[] E, int first, int last)if (first < last)if (first < last)
int mid = int mid = (first+last)/2(first+last)/2;;mergeSort(E, first, mid);mergeSort(E, first, mid);mergeSort(E, mid+1, last);mergeSort(E, mid+1, last);merge(E, first, mid, last);merge(E, first, mid, last);
return;return;
Design and Analysis of Algorithms – Chapter 5 12
In-Class ExerciseIn-Class Exercise
P175 Problem 5.1.6P175 Problem 5.1.6
Apply Mergesort to sort the list E, X, A, M, P, L, E in Apply Mergesort to sort the list E, X, A, M, P, L, E in alphabetical order. alphabetical order.
Design and Analysis of Algorithms – Chapter 5 13
Algorithm: MergesortAlgorithm: Mergesort
void mergeSort(Element[] E, int first, int last)void mergeSort(Element[] E, int first, int last)if (first < last)if (first < last)
int mid = int mid = (first+last)/2(first+last)/2;;mergeSort(E, first, mid);mergeSort(E, first, mid);
mergeSort(E, mid+1, last);mergeSort(E, mid+1, last);
merge(E, first, mid, last);merge(E, first, mid, last);
return;return;
Time EfficiencyTime Efficiency
Design and Analysis of Algorithms – Chapter 5 14
Mergesort ComplexityMergesort Complexity
Mergesort always partitions the array equally.Mergesort always partitions the array equally. Thus, the recursive depth is always Thus, the recursive depth is always (lg (lg nn) ) The amount of work done at each level is The amount of work done at each level is ((nn) ) Intuitively, the complexity should be Intuitively, the complexity should be ((nn lg lg nn) )
We have, We have, • TT((nn) =2) =2TT((nn/2) /2) ((n) for n>1, T(1)=0 n) for n>1, T(1)=0 ((nn lg lg nn))
The amount of extra memory used is The amount of extra memory used is ((nn))
Design and Analysis of Algorithms – Chapter 5 15
Efficiency of MergesortEfficiency of Mergesort
Number of comparisons is close to theoretical minimum for Number of comparisons is close to theoretical minimum for comparison-based sorting: comparison-based sorting: • lg lg n n ! ≈ ! ≈ nn lg lg n n - 1.44- 1.44 n n
Space requirement: Space requirement: ΘΘ( ( n n ) () (NOTNOT in-place) in-place)
Can be implemented without recursion (bottom-up)Can be implemented without recursion (bottom-up)
All cases have same efficiency: All cases have same efficiency: ΘΘ( ( n n log log nn) )
Design and Analysis of Algorithms – Chapter 5 16
AnimationAnimation
http://math.hws.edu/TMCM/java/xSortLab/index.html
Design and Analysis of Algorithms – Chapter 5 17
The master theoremThe master theorem
Given: a Given: a divide and conquerdivide and conquer algorithm algorithm
Then, the Master Theorem gives us a cookbook for the Then, the Master Theorem gives us a cookbook for the algorithm’s running time:algorithm’s running time:
Design and Analysis of Algorithms – Chapter 5 18
A general divide-and-conquer A general divide-and-conquer recurrencerecurrence
TT((nn) = ) = aTaT((n/bn/b) + ) + f f ((nn)) where where f f ((nn)) ∈∈ ΘΘ((nndd))
T(1) = cT(1) = c
a < ba < bdd T T((nn) ) ∈∈ ΘΘ((nndd)) a = ba = bdd T T((nn) ) ∈∈ ΘΘ((nnd d lg lg n n )) a > ba > bdd T T((nn) ) ∈∈ ΘΘ((nnlog log b b aa))
Please refer to Appendix B for the proofPlease refer to Appendix B for the proof
Design and Analysis of Algorithms – Chapter 5 19
In-class exerciseIn-class exercise
Page 175: Problem 5Page 175: Problem 5 5. Find the order of growth for solutions of the following 5. Find the order of growth for solutions of the following
recurrences.recurrences.• a. T(n) = 4T(n/2) + n, T(1) = 1a. T(n) = 4T(n/2) + n, T(1) = 1
• b. T(n) = 4T(n/2) + nb. T(n) = 4T(n/2) + n2222, T(1) = 1, T(1) = 1
• c. T(n) = 4T(n/2) + nc. T(n) = 4T(n/2) + n3333, T(1) = 1, T(1) = 1
In-Class ExerciseIn-Class Exercise
5.1.9 Let A[0, n-1] be an array of n distinct real 5.1.9 Let A[0, n-1] be an array of n distinct real numbers. A pair (A[i], A[j]) is said to be an numbers. A pair (A[i], A[j]) is said to be an inversion if these numbers are out of order, i.e., i<j inversion if these numbers are out of order, i.e., i<j but A[i]>A[j]. Design an O(nlogn) algorithm for but A[i]>A[j]. Design an O(nlogn) algorithm for counting the number of inversions. counting the number of inversions.
Design and Analysis of Algorithms – Chapter 5 20
Review QuicksortReview Quicksort
Design and Analysis of Algorithms – Chapter 5 21
Design and Analysis of Algorithms – Chapter 5 22
Quicksort by Hoare (1962)Quicksort by Hoare (1962)
Select a Select a pivotpivot (partitioning element) (partitioning element) Rearrange the list so that all the elements in the positions Rearrange the list so that all the elements in the positions
before the pivot are smaller than or equal to the pivot and before the pivot are smaller than or equal to the pivot and those after the pivot are larger than or equal to the pivot those after the pivot are larger than or equal to the pivot
Exchange the pivot with the last element in the first (i.e., ≤) Exchange the pivot with the last element in the first (i.e., ≤) sublist – the pivot is now in its final positionsublist – the pivot is now in its final position
Sort the two sublists recursivelySort the two sublists recursively
p
A[i]≤p A[i]p
Design and Analysis of Algorithms – Chapter 5 23
Quicksort by Hoare (1962)Quicksort by Hoare (1962)
Select a Select a pivotpivot (partitioning element) (partitioning element) Rearrange the list so that all the elements in the positions Rearrange the list so that all the elements in the positions
before the pivot are smaller than or equal to the pivot and before the pivot are smaller than or equal to the pivot and those after the pivot are larger than or equal to the pivot those after the pivot are larger than or equal to the pivot
Exchange the pivot with the last element in the first (i.e., ≤) Exchange the pivot with the last element in the first (i.e., ≤) sublist – the pivot is now in its final positionsublist – the pivot is now in its final position
Sort the two sublists recursivelySort the two sublists recursively
Apply quicksort to sort the list 7 2 9 10 5 4Apply quicksort to sort the list 7 2 9 10 5 4
p
A[i]≤p A[i]p
Design and Analysis of Algorithms – Chapter 5 24
Quicksort ExampleQuicksort Example
Recursive implementation with the left most array Recursive implementation with the left most array entry selected as the pivot element.entry selected as the pivot element.
Design and Analysis of Algorithms – Chapter 5 25
QuicksortQuicksort
Animated Example: Animated Example: http://math.hws.edu/TMCM/java/xSortLab/index.html
Design and Analysis of Algorithms – Chapter 5 26
Quicksort AlgorithmQuicksort Algorithm
Input: Input: Array E and indices first, and last, s.t. elements E[i] are defined for Array E and indices first, and last, s.t. elements E[i] are defined for
first first i i last last Ouput: Ouput:
E[first], …, E[last] is a sorted rearrangement of the arrayE[first], …, E[last] is a sorted rearrangement of the array
Void quickSort(Element[] E, int first, int last)Void quickSort(Element[] E, int first, int last)if (first < last)if (first < last)
Element pivotElement = E[first];Element pivotElement = E[first];int splitPoint = partition(E, pivotElement, first, last);int splitPoint = partition(E, pivotElement, first, last);quickSort (E, first, splitPoint –1 );quickSort (E, first, splitPoint –1 );quickSort (E, splitPoint +1, last );quickSort (E, splitPoint +1, last );
return;return;
Design and Analysis of Algorithms – Chapter 5 27
Quicksort AnalysisQuicksort Analysis
Partition can be done in O(Partition can be done in O(nn) time, where ) time, where nn is the is the size of the arraysize of the array
Let Let TT((nn) be the number of comparisons required ) be the number of comparisons required by Quicksortby Quicksort
If the pivot ends up at position If the pivot ends up at position kk, then we have, then we have• TT((nn) ) TT((nnkk) ) TT((kk 1) 1) nn
To determine best-, worst-, and average-case To determine best-, worst-, and average-case complexity we need to determine the values of complexity we need to determine the values of kk that correspond to these cases.that correspond to these cases.
Design and Analysis of Algorithms – Chapter 5 28
Quicksort AnalysisQuicksort Analysis
Partition can be done in O(Partition can be done in O(nn) time, where ) time, where nn is the is the size of the arraysize of the array
Let Let TT((nn) be the number of comparisons required ) be the number of comparisons required by Quicksortby Quicksort
If the pivot ends up at position If the pivot ends up at position kk, then we have, then we have• TT((nn) ) TT((nnkk) ) TT((kk 1) 1) nn
To determine best-, worst-, and average-case To determine best-, worst-, and average-case complexity we need to determine the values of complexity we need to determine the values of kk that correspond to these cases.that correspond to these cases.
Design and Analysis of Algorithms – Chapter 5 29
Best-Case ComplexityBest-Case Complexity
The best case is clearly when the pivot always The best case is clearly when the pivot always partitions the array equally.partitions the array equally.
Intuitively, this would lead to a recursive depth of Intuitively, this would lead to a recursive depth of at most lg at most lg nn calls calls
We can actually prove this. In this case We can actually prove this. In this case • TT((nn) ) TT((nn/2) /2) TT((nn/2) /2) n n ((nn lg lg nn))
Design and Analysis of Algorithms – Chapter 5 30
Worst-Case and Average-Case Worst-Case and Average-Case ComplexityComplexity
The worst-case is when the pivot always ends up in The worst-case is when the pivot always ends up in the first or last element. That is, partitions the the first or last element. That is, partitions the array as unequally as possible.array as unequally as possible.
In this case In this case • TT((nn) ) ??
Design and Analysis of Algorithms – Chapter 5 31
Worst-Case and Average-Case Worst-Case and Average-Case ComplexityComplexity
The worst-case is when the pivot always ends up in The worst-case is when the pivot always ends up in the first or last element. That is, partitions the the first or last element. That is, partitions the array as unequally as possible.array as unequally as possible.
In this case In this case • TT((nn) ) TT((nn11) ) TT(1(11) 1) n n TT((nn11) ) nn
nn ( (nn11) ) … + 1 … + 1
nn((nn 1)/2 1)/2 ((nn22))
Average case is rather complex, but is where the Average case is rather complex, but is where the algorithm earns its name. The bottom line is:algorithm earns its name. The bottom line is:
)lg(lg386.1)( nnnnnA
Design and Analysis of Algorithms – Chapter 5 32
Summary of QuicksortSummary of Quicksort
Best caseBest case: split in the middle — : split in the middle — ΘΘ( ( n n log log nn) ) Worst caseWorst case: sorted array! — : sorted array! — ΘΘ( ( nn22) ) Average caseAverage case: random arrays — : random arrays — ΘΘ( ( n n log log nn))
Design and Analysis of Algorithms – Chapter 5 33
Summary of QuicksortSummary of Quicksort
Best caseBest case: split in the middle — : split in the middle — ΘΘ( ( n n log log nn) ) Worst caseWorst case: sorted array! — : sorted array! — ΘΘ( ( nn22) ) Average caseAverage case: random arrays — : random arrays — ΘΘ( ( n n log log nn))
Memory requirement?Memory requirement?
Design and Analysis of Algorithms – Chapter 5 34
Summary of QuicksortSummary of Quicksort
Best caseBest case: split in the middle — : split in the middle — ΘΘ( ( n n log log nn) ) Worst caseWorst case: sorted array! — : sorted array! — ΘΘ( ( nn22) ) Average caseAverage case: random arrays — : random arrays — ΘΘ( ( n n log log nn))
Memory requirement?Memory requirement? In-place sorting algorithmIn-place sorting algorithm Considered as the method of choice for internal sorting for Considered as the method of choice for internal sorting for
large files (large files (nn ≥ 10000) ≥ 10000)
Design and Analysis of Algorithms – Chapter 5 35
Summary of QuicksortSummary of Quicksort
Best caseBest case: split in the middle — : split in the middle — ΘΘ( ( n n log log nn) ) Worst caseWorst case: sorted array! — : sorted array! — ΘΘ( ( nn22) ) Average caseAverage case: random arrays — : random arrays — ΘΘ( ( n n log log nn))
Considered as the method of choice for internal sorting for Considered as the method of choice for internal sorting for large files (large files (nn ≥ 10000) ≥ 10000)
Improvements:Improvements:• better pivot selection: median of three partitioning avoids worst better pivot selection: median of three partitioning avoids worst
case in sorted filescase in sorted files• switch to insertion sort on small subfilesswitch to insertion sort on small subfiles• elimination of recursionelimination of recursionthese combine to 20-25% improvementthese combine to 20-25% improvement
In-class exercisesIn-class exercises
Design and Analysis of Algorithms – Chapter 5 36