04 Recurrences
Transcript of 04 Recurrences
-
8/2/2019 04 Recurrences
1/25
Huo Hongwei 1
Design and Analysis of Algorithms
Recurrences
Reference:CLRS Chapter 4
Topics:
Substitution method Recursion-tree method
Master method
-
8/2/2019 04 Recurrences
2/25
Huo Hongwei 2
Solving recurrences
The analysis of Mergesort from Lecture 2 required us tosolve a recurrence.
Recurrences are a major tool for analysis of algorithms
Today: Learn a few methods.
Substitution method
Recursion- tree method
Master method
Divide and Conquer algorithms which are analyzable byrecurrences.
-
8/2/2019 04 Recurrences
3/25
Huo Hongwei 3
Recall: Mergesort
otherwise2T(n/2) + (n) ifn = 1(1)T(n) =
MERGE-SORT(A,p,r)
1 if p < r2 then q (p+r)/23 MERGE-SORT (A, p, q)
4 MERGE-SORT (A, q+1, r)
5 MERGE (A, p, q, r)
MERGESORT
-
8/2/2019 04 Recurrences
4/25
Huo Hongwei 4
Substitution method
The most general method:
Guess the form of the solution.
Verify by induction.
Solve for constants.
Ex. T(n) = 4T(n/2) + 100n
Assume that T(1) = (1). Guess O(n3). (Prove O and separately.) Assume that T(k) ck3 fork
-
8/2/2019 04 Recurrences
5/25
Huo Hongwei 5
Example of substitution
T(n) = 4T(n/2) + 100n
4c(n/2)3 + 100n= (c/2)n3 + 100n
=cn3 ((c/2)n3 100n)
cn3 whenever (c/2)n3 100n 0, for example, ifc 200 andn 1.
desiredresidual
desired
residual
-
8/2/2019 04 Recurrences
6/25
Huo Hongwei 6
Example (continued)
We must also handle the initial conditions/the boundaryconditions, that is, ground the induction with base cases.
Base: T(n) = (1) for alln
-
8/2/2019 04 Recurrences
7/25
Huo Hongwei 7
A tighter upper bound?
We shall prove that T(n) = O(n2).
Assume that T(k) ck2 fork
-
8/2/2019 04 Recurrences
8/25
Huo Hongwei 8
A tighter upper bound?
IDEA: Strengthen the induction hypothesis.
Subtract a low-order term.
Assume that T(k) c1k2 c2k fork 100.
Pickc1 big enough to handle the initial conditions.
subtleties
-
8/2/2019 04 Recurrences
9/25
Huo Hongwei 9
Avoiding Pitfalls
Be careful not to misuse asymptotic notation. Forexample:
We can falsely prove T(n) = O(n) by guessing T(n) cnfor T(n) = 2T(n/2) + n
T(n) 2c n/2 + n cn + n= O(n) Wrong!
The error is that we havent proved the exact form ofthe inductive hypothesis T(n) cn.
-
8/2/2019 04 Recurrences
10/25
Huo Hongwei 10
Changing Variables
Use algebraic manipulation to make an unknownrecurrence similar to what you have seen before.
Consider T(n) = 2T(n1/2) + lgn , Rename m = lgn and we have
T(2m) = 2T(2m/2) +m .
Set S(m) = T(2m) and we have
S(m) = 2S(m/2) + m S(m) = O(m lgm) . Changing back from S(m) to T(n), we have
T(n) = T(2m) = S(m) = O(m lgm) = O(lgn lg lgn) .
-
8/2/2019 04 Recurrences
11/25
Huo Hongwei 11
Recursion- tree method
A recursion tree models the costs (time) of a recursiveexecution of an algorithm.
The recursion tree method is good for generatingguesses for the substitution method.
The recursion-tree method can be unreliable, just like anymethod that uses ellipses ().
The recursion-tree method promotes intuition, however.
-
8/2/2019 04 Recurrences
12/25
Huo Hongwei 12
The Construction of a Recursion Tree
Solve T(n) = 3T(n/4) + (n2), we haveT(n)
-
8/2/2019 04 Recurrences
13/25
Huo Hongwei 13
The Construction of a Recursion Tree
Solve T(n) = 3T(n/4) + (n2), we havecn2
n4
T( ) n4
T( )n4
T( )
-
8/2/2019 04 Recurrences
14/25
Huo Hongwei 14
The Construction of a Recursion Tree
Solve T(n) = 3T(n/4) + (n2), we havecn2
n4
c( )2n4
c( )2n4
c( )2
n16T( ) n16T( )n16T( ) n16T( ) n16T( )n16T( )n16T( ) n16T( )n16T( )
-
8/2/2019 04 Recurrences
15/25
Huo Hongwei 15
Construction of recursion tree
The fully expanded tree has lg4n +1 levels, i.e., it has height lg4n.
n /4k
cn2
c(n/4)2
T(1) T(1) T(1) T(1) T(1) T(1) T(1)
cn2
3/16cn2
(3/16)2cn2
Total: O(n2)
c(n/4)2 c(n/4)2
c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2
T(1) T(1) T(1) (n )log43nlog43
lg4n
geometric series
-
8/2/2019 04 Recurrences
16/25
Huo Hongwei 16
Master Method
It provides a cookbook method for solving recurrencesof the form:
T(n) = a T(n/b) + f(n)
wherea 1 andb > 1 are constants andf(n) is anasymptotically positive function
-
8/2/2019 04 Recurrences
17/25
Huo Hongwei 17
Idea of master theorem
Recursion tree
f(n/b)f(n/b) f(n/b)
(1)
f(n)a
f(n/b
2
)f(n/b2
) f(n/b
2
)
ah = logbn
f(n)
a f(n/b)
a2f(n/b2)
#leaves =ah
=alogbn
=nlogba
nlogba (1)
-
8/2/2019 04 Recurrences
18/25
Huo Hongwei 18
Three common cases
Comparef(n) withnlogba:
1.f(n) = O(nlogba-) for some constant > 0.f(n) grows polynomially slower thannlogba (by ann
factor).
Solution: T(n)= (nlogba).
-
8/2/2019 04 Recurrences
19/25
Huo Hongwei 19
Idea of master theorem
Recursion tree
f(n/b)f(n/b) f(n/b)
(1)
f(n)a
f(n/b2)f(n/b2) f(n/b2)
ah = logbn
f(n)
a f(n/b)
a2f(n/b2)
CASE 1: The weight increases geometrically from
the root to the leaves. The leaves hold a
constant fraction of the total weight.
nlogba (1)(nlogba)
-
8/2/2019 04 Recurrences
20/25
Huo Hongwei 20
Three common cases
Comparef(n) withnlogba:
2.f(n)= (nlogba lgkn) for some constantk 0. f(n) andnlogba grow at similar rates.
Solution: T(n)= (nlogba lgk+1n).
-
8/2/2019 04 Recurrences
21/25
Huo Hongwei 21
Idea of master theorem
Recursion tree
f(n/b)f(n/b) f(n/b)
(1)
f(n)a
f(n/b2)f(n/b2) f(n/b2)
ah = logbn
f(n)
a f(n/b)
a2f(n/b2)
CASE 2: (k = 0)The weight is approximately the
same on each of the logbn levels.
nlogba (1)(nlogba lgn)
-
8/2/2019 04 Recurrences
22/25
Huo Hongwei 22
Three common cases
Comparef(n) withnlogba:
3.f(n)= (nlogba+) for some constant > 0. f(n) grows polynomially faster thannlogba (by ann
factor),
andf(n) satisfies the regularity condition that
a f(n/b) c f(n) for some constantc < 1. Solution: T(n)= (f(n)).
-
8/2/2019 04 Recurrences
23/25
Huo Hongwei 23
Idea of master theorem
Recursion tree
f(n/b)f(n/b) f(n/b)
(1)
f(n)a
f(n/b2)f(n/b2) f(n/b2)
ah = logbn
f(n)
a f(n/b)
a2f(n/b2)
CASE 3: The weight decreases geometrically from
the root to the leaves. The root holds a
constant fraction of the total weight.
nlogba (1)(f(n))
-
8/2/2019 04 Recurrences
24/25
Huo Hongwei 24
Examples
T(n) = 4T(n/2) + n
a = 4, b = 2, nlogba = n2 ;f(n) = n. CASE 1:f(n) = O(n2- ) for =1. T(n) = (n2)
T(n) = 4T(n/2) + n2
a = 4, b = 2, nlogba = n2 ;f(n) = n2. CASE 2:f(n) = (n2 lg0n), that is,k = 0. T(n) = (n2 lgn)
-
8/2/2019 04 Recurrences
25/25
Huo Hongwei 25
Examples
T(n) = 4T(n/2) + n3
a = 4, b = 2, nlogba = n2 ;f(n) = n3. CASE 3:f(n) = (n2+ ), for = 1 and 4(cn/2)3 cn3 (regular
cond.) forc = 1/2.
T(n) = (n3) T(n) = 4T(n/2) + n2/lgn
a = 4, b = 2, nlogba = n2 ;f(n) = n3/lgn. Master method does not apply. In particular, for every
constant > 0, we haven = (lgn).