RecurrencesMethods for solving recurrences The substitution method. The iteration method. The master...
Transcript of RecurrencesMethods for solving recurrences The substitution method. The iteration method. The master...
Recurrences
Recurrences
Recurrence
A recurrence is an equality or inequality on a function thatuses values of the function on smaller inputs.
Examples:
T (n) =
1 if n = 1
T (n − 1) + 1 if n > 1
T (n) =
1 if n = 1
2T (bn/2c) + n if n > 1
T (n) =
0 if n ≤ 2
T (b√nc) + 1 if n > 2
T (n) =
1 if n = 1
T (b2n/3c) + T (bn/3c) + 1 if n > 1
Recurrences
Methods for solving recurrences
The substitution method.
The iteration method.
The master method.
Recurrences
The substitution method
In the substitution method, we guess the answer to therecurrence and then prove the correctness using induction.
Recurrences
Example 1: Guessing the exact solution
Recurrence T (n) =
1 if n = 1
2T (n/2) + n if n > 1
Guess T (n) = n log n + n.
Base For n = 1, n log n + n = 1 = T (1).
Induction step Assume T (m) = m logm + m for all m < n.
T (n) = 2T (n/2) + n
= 2(n
2log
n
2+
n
2
)+ n by the induction hypothesis
= n logn
2+ n + n
= n log n − n log 2 + n + n
= n log n − n + n + n
= n log n + n
Recurrences
Example 2: Guessing upper and lower bounds
Recurrence T (n) =
1 if n = 1
2T (n/2) + n if n > 1
Guess T (n) ≤ cn log n for n ≥ 2.
Base T (2) = 2T (1)+2 = 4 ≤ c ·2 log 2 = 2c ⇒ c ≥ 2.
Induction step Assume T (m) ≤ cm logm for all 2 ≤ m < n.
T (n) = 2T (n/2) + n
≤ 2(cn
2log
n
2
)+ n
= cn logn
2+ n
= cn log n − cn + n
≤ cn log n if −cn + n ≤ 0⇒ c ≥ 1
⇒ T (n) ∈ O(n log n).Recurrences
Lower bound
Guess T (n) ≥ dn log n for all n.
Base T (1) = 1 ≥ d · 1 log 1 = 0.
Induction step Assume T (m) ≥ dm logm for all 2 ≤ m < n.
T (n) = 2T (n/2) + n
≥ 2(dn
2log
n
2
)+ n
= dn logn
2+ n
= dn log n − dn + n
≥ dn log n if −dn + n ≤ 0⇒ d ≤ 1
⇒ T (n) ∈ Ω(n log n).SinceT (n) ∈ O(n log n), we conclude that T (n) ∈ Θ(n log n).
Recurrences
Example 3: Revising the guess
Recurrence T (n) = 8T (n/2) + n2.
Guess T (n) ≤ cn3.
Induction step
T (n) = 8T (n/2) + n2
≤ 8c(n
2
)3+ n2
= 8c · n3
8+ n2
= cn3 + n2
6≤ cn3
Recurrences
Example 3: Revising the guess
Sometimes the guess needs to be revised by subtracting alower-order term from the previous guess.
Guess T (n) ≤ cn3 − dn2.
Induction step
T (n) = 8T (n/2) + n2
≤ 8
(c(n
2
)3− d
(n2
)2)+ n2
= 8c · n3
8− 8d · n
2
4+ n2
= cn3 − 2dn2 + n2
= cn3 − dn2 − dn2 + n2
≤ cn3 − dn2 if −dn2 + n2 ≤ 0⇒ d ≥ 1
Recurrences
The iteration method
Expand the recurrence into a summation.
Evaluate the summation.
Recurrences
Example 1
Recurrence T (n) =
0 if n = 0
T (n − 1) + 1 if n > 0
Iteration
T (n) = T (n − 1) + 1
=(T (n − 2) + 1
)+ 1
= T (n − 3) + 3
...
= T (0) + n
= n
Recurrences
Example 2
Recurrence T (n) =
1 if n = 1
T (n − 1) + n if n > 1
Iteration
T (n) = T (n − 1) + n
= T (n − 2) + (n − 1) + n
= T (n − 3) + (n − 2) + (n − 1) + n
...
= 1 + 2 + · · ·+ n
=n(n + 1)
2
Recurrences
Changing free terms
If we are only interested in the asymptotic behavior, the freeterms in the recurrence can be simplified.
T (n) =
1 if n = 1
T (n − 1) + n if n > 1
T2(n) =
3 if n = 1
T2(n − 1) + 3n if n > 1
T3(n) =
1 if n = 1
T3(n − 1) + n + 2√n if n > 1
T2(n) = 3T (n) for all n (T2(n) = 3 + 6 + · · ·+ 3n).
T (n) ≤ T3(n) ≤ 3T (n) for all n.
Recurrences
Recurrence tree
T (n) = T (n/3) + T (2n/3) + n.
1log 3· n log n = n · log3 n ≤ T (n) ≤ n · log3/2 n = 1
log(3/2)· n log n
⇒ T (n) ∈ Θ(n log n).
Recurrences
Recurrence tree
T (n) = T (n/3) + T (2n/3) + n.
1log 3· n log n = n · log3 n ≤ T (n) ≤ n · log3/2 n = 1
log(3/2)· n log n
⇒ T (n) ∈ Θ(n log n).
Recurrences
Recurrence tree
T (n) = T (n/3) + T (2n/3) + n.
1log 3· n log n = n · log3 n ≤ T (n) ≤ n · log3/2 n = 1
log(3/2)· n log n
⇒ T (n) ∈ Θ(n log n).
Recurrences
Recurrence tree
T (n) = T (n/3) + T (2n/3) + n.
1log 3· n log n = n · log3 n ≤ T (n) ≤ n · log3/2 n = 1
log(3/2)· n log n
⇒ T (n) ∈ Θ(n log n).
Recurrences
The master method
A method for solving recurrences of the form
T (n) = aT (n/b) + f (n)
where a ≥ 1, b > 1, and f (n) is an asymptotic positivefunction.
Recurrences
The master theorem
Let T (n) = aT (n/b) + f (n) where a ≥ 1, b > 1 and f (n) isasymptotically positive.
1 If f (n) ∈ O(nlogb a−ε) for some constant ε > 0 thenT (n) ∈ Θ(nlogb a).
2 If f (n) ∈ Θ(nlogb a) then T (n) ∈ Θ(f (n) · log n).
3 f (n) ∈ Ω(nlogb a+ε) for some constant ε > 0 and ifaf (n/b) ≤ cf (n) for some constant c < 1 and allsufficiently large n, then T (n) ∈ Θ(f (n)).
T (n/b) means either bn/bc or dn/be.Case 2 can be generalized: If f (n) ∈ Θ(nlogb a logk n) forsome k ≥ 0 then T (n) ∈ Θ(f (n) · logk+1 n).
Recurrences
Example 1 (case 1)
T (n) = 9T (n/3) + n.
We have a = 9, b = 3, f (n) = n.
logb a = log3 9 = 2.
All the conditions of the first case are satisfied:
a = 9 ≥ 1, b = 3 > 1, f (n) = n ≥ 0.For ε = 1/2, f (n) ∈ O(nlogb a−ε) = O(n3/2).
Hence, T (n) ∈ Θ(nlogb a) = Θ(n2).
Recurrences
Example 2 (case 1)
T (n) = 5T (n/2) + n2.
We have a = 5, b = 2, f (n) = n2.
logb a = log2 5 ≈ 2.322.
All the conditions of the first case are satisfied:
a = 5 ≥ 1, b = 2 > 1, f (n) = n2 ≥ 0.For ε = 0.3, f (n) ∈ O(nlogb a−ε) = O(n2.022).
Hence, T (n) ∈ Θ(nlogb a) = Θ(nlog2 5).
Recurrences
Example 3 (case 2)
T (n) = T (2n/3) + 1.
We have a = 1, b = 3/2, f (n) = 1.
logb a = log3/2 1 = 0.
All the conditions of the second case are satisfied:
a = 1 ≥ 1, b = 3/2 > 1, f (n) = 1 ≥ 0.f (n) ∈ Θ(nlogb a) = Θ(1).
Hence, T (n) ∈ Θ(f (n) · log n) = Θ(log n).
Recurrences
Example 4 (case 3)
T (n) = 3T (n/4) + n log n.
We have a = 3, b = 4, f (n) = n log n.
logb a = log4 3 ≈ 0.792.
All the conditions of the third case are satisfied:
a = 3 ≥ 1, b = 4 > 1, f (n) = n log n ≥ 0.For ε = 0.1, f (n) ∈ Ω(nlogb a+ε) = Ω(n0.892).For all n, af (n/b) = 3n
4 log n4 ≤
34n log n = 3
4 f (n).
Hence, T (n) ∈ Θ(f (n)) = Θ(n log n).
Recurrences
Example 5
T (n) = 2T (n/2) + n/ log n.
We have a = 2, b = 2, f (n) = n/ log n.
logb a = log2 2 = 1.
n/ log n < n, but n/ log n /∈ O(nlogb a−ε) for every ε > 0.
Recurrences