Divide-and-Conquer

Recursive in structure – Divide the problem into several smaller

sub-problems that are similar to the original but smaller in size

– Conquer the sub-problems by solving them recursively. If they are small enough, just solve them in a straightforward manner.

– Combine the solutions to create a solution to the original problem

An Example: Merge Sort

Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each

Conquer: Sort the two subsequences recursively using merge sort.

Combine: Merge the two sorted subsequences to produce the sorted answer.

Merge-Sort (A, p, r)

INPUT: a sequence of n numbers stored in array A

OUTPUT: an ordered sequence of n numbers

1. if p < r

2. then q [(p+r)/2]

3. Merge-Sort (A, p, q)

4. Merge-Sort (A, q+1, r)

5. Merge (A, p, q, r)

Analysis of Merge Sort

Divide: computing the middle takes (1) Conquer: solving 2 sub-problem takes 2T(n/2) Combine: merging n-element takes (n)

Total:T(n) = (1) if n = 1

T(n) = 2T(n/2) + (n) if n > 1

T(n) = (n lg n) (CLRS/Chapter 4)

Recurrence Relations

Recurrences (Chapter 4)– Substitution Method– Iteration Method– Master Method

Arising from Divide and Conquer

(e.g. MERGE-SORT)T(n) = (1) if n c

T(n) = a T(n/b) + D(n) + C(n) otherwise

Substitution Method

Guessing the form of the solutions, then using mathematical induction to find the constants and show the solution works.

It works well when it is easy to guess. But, there is no general way to guess the correct solution.

An Example

Solve: T(n) = 3T(n/3) + n

T(n) 3c n/3 lg n/3 + n

c n lg (n/3) + n

= c n lg n - c n lg3 + n

= c n lg n - n (c lg 3 - 1)

c n lg n

* The last step is true for c 1 / lg3.

Making a Good Guess

Guessing a similar solution to the one that you have seen before– T(n) = 3T(n/3 + 5) + n similar to T(n) = 3T(n/3) + n

when n is large, the difference between n/3 and (n/3 + 5) is insignificant

Another way is to prove loose upper and lower bounds on recurrence and then reduce the range of uncertainty.– Start with T(n) = (n) & T(n) = O(n2) T(n) = (n log n)

Subtleties

When the math doesn’t quite work out in the induction, try to adjust your guess with a lower-order term. For example:

– We guess T(n) O(n) for T(n) = 3T(n/3)+ 4, but we have T(n) 3c n/3 + 4 = c n + 4

– New guess is T(n) c n - b, where b 0

T(n) 3(c n/3 - b)+4 = c n - 3b + 4 = c n - b - (2b-4)

Therefore, T(n) c n - b, if 2b - 4 0 or if b 2

Changing Variables

Use algebraic manipulation to turn an unknown recurrence similar to what you have seen before. – Consider T(n) = 2T(n1/2) + lg n

– Rename m = lg n and we have

T(2m) = 2T(2m/2) + m

– Set S(m) = T(2m) and we have

S(m) = 2S(m/2) + m S(m) = O(m lg m)

– Changing back from S(m) to T(n), we have

T(n) = T(2m) = S(m) = O(m lg m) = O(lg n lg lg n)

Avoiding Pitfalls

Be careful not to misuse asymptotic notation. For example:

– We can falsely prove T(n) = O(n) by guessing T(n) c n for T(n) = 2T(n/2) + n

T(n) 2c n/2 + n

c n + n

= O(n) Wrong!– The err is that we haven’t proved T(n) c n

Exercises

Solution of T(n) = T(n/2) + 1 is O(lg n)

Solution of T(n) = 2T(n/2 + 17) + n is O(n lg n)

Solve T(n) = 2T(n1/2) + 1 by making a change of variables. Don’t worry whether values are integral.

Iteration Method

Expand (iterate) the recurrence and express it as a summation of terms dependent only on n and the initial conditions

The key is to focus on 2 parameters– the number of times the recurrence needs to be

iterated to reach the boundary condition– the sum of terms arising from each level of the

iteration process Techniques for evaluating summations can

then be used to provide bounds on solution.

An Example

Solve: T(n) = 3T(n/4) + n T(n) = n + 3T(n/4) = n + 3[ n/4 + 3T(n/16) ] = n + 3[n/4] + 9T(n/16) = n + 3[n/4] + 9 [n/16] + 27T(n/64)

T(n) n + 3n/4 + 9n/16 + 27n/64 + … + 3log4 n(1) n (3/4)i + (nlog43)

= 4n+ o(n) = O(n)

Recursion Trees

Keep track of the time spent on the subproblems of a divide and conquer algorithm

A convenient way to visualize what happens when a recursion is iterated

Help organize the algebraic bookkeeping necessary to solve the recurrence

Merge Sort

n

n/2

n/4

n/4

n/2

n/4

n/4

Running times to merge two sublists

Running time to sort the left sublist

Running Time

n

n/2

n/4

n/4

n/2

n/4

n/4

n=n

2¢(n/2) = n

4¢(n/4) = nlg n

Total: n lg n

Recursion Trees and Recurrences

Useful even when a specific algorithm is not specified– For T(n) = 2T(n/2) + n2, we have

Recursion Trees

T(n) = (n2)

Recursion Trees

For T(n) = T(n/3) + T(2n/3) + n

T(n) = O(n lg n)

Master Method

Provides a “cookbook” method for solving recurrences of the form

T(n) = a T(n/b) + f(n)Assumptions:

– a 1 and b 1 are constants

– f(n) is an asymptotically positive function

– T(n) is defined for nonnegative integers– We interpret n/b to mean either n/b or n/b

The Master Theorem

With the recurrence T(n) = a T(n/b) + f(n) as in the previous slide, T(n) can be bounded asymptotically as follows:

1. If f(n)=O(nlogba-) for some constant > 0, then T(n)= (nlogba).

2. If f(n) = (nlogba), then T(n) = (nlogba lg n).

3. If f(n) = ( nlogba+ ) for some constant > 0, and if a f(n/b) c f(n) for some constant c < 1 and all sufficiently large n, then T(n)= (f(n)).

Simplified Master Theorem

Let a 1 and b > 1 be constants and let T(n) be the recurrence

T(n) = a T(n/b) + c nk

defined for n 0.

1. If a > bk, then T(n) = ( nlogba ).

2. If a = bk, then T(n) = ( nk lg n ).

3. If a < bk, then T(n) = ( nk ).

Examples

T(n) = 16T(n/4) + n– a = 16, b = 4, thus nlogba = nlog416 = (n2)– f(n) = n = O(nlog416 - ) where = 1 case 1.– Therefore, T(n) = (nlogba ) = (n2)

T(n) = T(3n/7) + 1– a = 1, b=7/3, and nlogba = nlog 7/3 1 = n0 = 1– f(n) = 1 = (nlogba) case 2.– Therefore, T(n) = (nlogba lg n) = (lg n)

Examples (Cont.)

T(n) = 3T(n/4) + n lg n– a = 3, b=4, thus nlogba = nlog43 = O(n0.793)– f(n) = n lg n = (nlog43 + ) where 0.2 case 3.– Therefore, T(n) = (f(n)) = (n lg n)

T(n) = 2T(n/2) + n lg n– a = 2, b=2, f(n) = n lg n, and nlogba = nlog22 = n– f(n) is asymptotically larger than nlogba, but not

polynomially larger. The ratio lg n is asymptotically less than n for any positive . Thus, the Master Theorem doesn’t apply here.

Exercises

Use the Master Method to solve the following: T(n) = 4T(n/2) + n

T(n) = 4T(n/2) + n2

T(n) = 4T(n/2) + n3