Chapter 4 Divide-and-Conquer
-
Upload
edan-trujillo -
Category
Documents
-
view
43 -
download
0
description
Transcript of Chapter 4 Divide-and-Conquer
1
Chapter 4 Divide-and-Conquer
2
About this lecture
bull Recall the divide-and-conquer paradigm which we used for merge sortndash Divide the problem into a number of sub-problems
that are smaller instances of the same problemndash Conquer the sub-problems by solving them
recursivelybull Base case If the sub-problems are small enough just solve
them by brute forcendash Combine the sub-problem solutions to give a solution
to the original problembull We look at two more algorithms based on divide-
and-conquer
3
About this lecture
bull Analyzing divide-and-conquer algorithmsbull Introduce some ways of solving recurrencesndash Substitution Method (If we know the answer)ndash Recursion Tree Method (Very useful )ndash Master Theorem (Save our effort)
4
Maximum-subarray problem
bull Input an array A[1n] of n numbersndash Assume that some of the numbers are negative
because this problem is trivial when all numbers are nonnegative
bull Output a nonempty subarray A[ij] having the largest sum S[i j] = ai + ai+1 + + aj
13 -3 -25 20 -3 -16 -23 18 20 -7 12 -5 -22 15 -4 7
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
A
maximum subarray
5
A brute-force solutionbull Examine all possible S[i j]bull Two implementationsndash compute each S[i j] in O(n) time O(n3) timendash compute each S[i j+1] from S[i j] in O(1) timendash (S[i i] = A[i] and S[i j+1] = S[i j] + A[j+1])ndash O(n2) time Ex i 1 2 3 4 5 6 A[i] 13 -3 -25 20 -3 -16
S[2 2] = -3 S[2 3] = -28 S[2 4] = -8 S[2 5] = -11
2
n
6
A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of
A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)
low mid high
mid +1
entirely in A[lowmid] entirely in A[mid+1high]
crossing the midpoint
Possible locations of subarrays of A[lowhigh]
7
FIND-MAX-CROSSING-SUBARRAY (A low mid high)
left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)
8
low mid high
mid +1
A[imid]
j
A[mid+1j]
i
A[ij] comprises two subarrays A[imid] and A[mid+1j]
9
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7
Example
A
S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2
mid =5
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7A
S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8
mid =5
maximum subarray crossing mid is S[49] = 16
10
if high == lowReturn (low high A[low]) base case only one element
else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)
FIND-MAXIMUM-SUBARRAY (A low high)
Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)
2highlow
11
Analyzing time complexity
bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY
T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
2
About this lecture
bull Recall the divide-and-conquer paradigm which we used for merge sortndash Divide the problem into a number of sub-problems
that are smaller instances of the same problemndash Conquer the sub-problems by solving them
recursivelybull Base case If the sub-problems are small enough just solve
them by brute forcendash Combine the sub-problem solutions to give a solution
to the original problembull We look at two more algorithms based on divide-
and-conquer
3
About this lecture
bull Analyzing divide-and-conquer algorithmsbull Introduce some ways of solving recurrencesndash Substitution Method (If we know the answer)ndash Recursion Tree Method (Very useful )ndash Master Theorem (Save our effort)
4
Maximum-subarray problem
bull Input an array A[1n] of n numbersndash Assume that some of the numbers are negative
because this problem is trivial when all numbers are nonnegative
bull Output a nonempty subarray A[ij] having the largest sum S[i j] = ai + ai+1 + + aj
13 -3 -25 20 -3 -16 -23 18 20 -7 12 -5 -22 15 -4 7
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
A
maximum subarray
5
A brute-force solutionbull Examine all possible S[i j]bull Two implementationsndash compute each S[i j] in O(n) time O(n3) timendash compute each S[i j+1] from S[i j] in O(1) timendash (S[i i] = A[i] and S[i j+1] = S[i j] + A[j+1])ndash O(n2) time Ex i 1 2 3 4 5 6 A[i] 13 -3 -25 20 -3 -16
S[2 2] = -3 S[2 3] = -28 S[2 4] = -8 S[2 5] = -11
2
n
6
A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of
A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)
low mid high
mid +1
entirely in A[lowmid] entirely in A[mid+1high]
crossing the midpoint
Possible locations of subarrays of A[lowhigh]
7
FIND-MAX-CROSSING-SUBARRAY (A low mid high)
left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)
8
low mid high
mid +1
A[imid]
j
A[mid+1j]
i
A[ij] comprises two subarrays A[imid] and A[mid+1j]
9
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7
Example
A
S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2
mid =5
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7A
S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8
mid =5
maximum subarray crossing mid is S[49] = 16
10
if high == lowReturn (low high A[low]) base case only one element
else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)
FIND-MAXIMUM-SUBARRAY (A low high)
Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)
2highlow
11
Analyzing time complexity
bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY
T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
3
About this lecture
bull Analyzing divide-and-conquer algorithmsbull Introduce some ways of solving recurrencesndash Substitution Method (If we know the answer)ndash Recursion Tree Method (Very useful )ndash Master Theorem (Save our effort)
4
Maximum-subarray problem
bull Input an array A[1n] of n numbersndash Assume that some of the numbers are negative
because this problem is trivial when all numbers are nonnegative
bull Output a nonempty subarray A[ij] having the largest sum S[i j] = ai + ai+1 + + aj
13 -3 -25 20 -3 -16 -23 18 20 -7 12 -5 -22 15 -4 7
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
A
maximum subarray
5
A brute-force solutionbull Examine all possible S[i j]bull Two implementationsndash compute each S[i j] in O(n) time O(n3) timendash compute each S[i j+1] from S[i j] in O(1) timendash (S[i i] = A[i] and S[i j+1] = S[i j] + A[j+1])ndash O(n2) time Ex i 1 2 3 4 5 6 A[i] 13 -3 -25 20 -3 -16
S[2 2] = -3 S[2 3] = -28 S[2 4] = -8 S[2 5] = -11
2
n
6
A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of
A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)
low mid high
mid +1
entirely in A[lowmid] entirely in A[mid+1high]
crossing the midpoint
Possible locations of subarrays of A[lowhigh]
7
FIND-MAX-CROSSING-SUBARRAY (A low mid high)
left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)
8
low mid high
mid +1
A[imid]
j
A[mid+1j]
i
A[ij] comprises two subarrays A[imid] and A[mid+1j]
9
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7
Example
A
S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2
mid =5
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7A
S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8
mid =5
maximum subarray crossing mid is S[49] = 16
10
if high == lowReturn (low high A[low]) base case only one element
else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)
FIND-MAXIMUM-SUBARRAY (A low high)
Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)
2highlow
11
Analyzing time complexity
bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY
T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
4
Maximum-subarray problem
bull Input an array A[1n] of n numbersndash Assume that some of the numbers are negative
because this problem is trivial when all numbers are nonnegative
bull Output a nonempty subarray A[ij] having the largest sum S[i j] = ai + ai+1 + + aj
13 -3 -25 20 -3 -16 -23 18 20 -7 12 -5 -22 15 -4 7
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
A
maximum subarray
5
A brute-force solutionbull Examine all possible S[i j]bull Two implementationsndash compute each S[i j] in O(n) time O(n3) timendash compute each S[i j+1] from S[i j] in O(1) timendash (S[i i] = A[i] and S[i j+1] = S[i j] + A[j+1])ndash O(n2) time Ex i 1 2 3 4 5 6 A[i] 13 -3 -25 20 -3 -16
S[2 2] = -3 S[2 3] = -28 S[2 4] = -8 S[2 5] = -11
2
n
6
A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of
A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)
low mid high
mid +1
entirely in A[lowmid] entirely in A[mid+1high]
crossing the midpoint
Possible locations of subarrays of A[lowhigh]
7
FIND-MAX-CROSSING-SUBARRAY (A low mid high)
left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)
8
low mid high
mid +1
A[imid]
j
A[mid+1j]
i
A[ij] comprises two subarrays A[imid] and A[mid+1j]
9
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7
Example
A
S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2
mid =5
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7A
S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8
mid =5
maximum subarray crossing mid is S[49] = 16
10
if high == lowReturn (low high A[low]) base case only one element
else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)
FIND-MAXIMUM-SUBARRAY (A low high)
Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)
2highlow
11
Analyzing time complexity
bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY
T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
5
A brute-force solutionbull Examine all possible S[i j]bull Two implementationsndash compute each S[i j] in O(n) time O(n3) timendash compute each S[i j+1] from S[i j] in O(1) timendash (S[i i] = A[i] and S[i j+1] = S[i j] + A[j+1])ndash O(n2) time Ex i 1 2 3 4 5 6 A[i] 13 -3 -25 20 -3 -16
S[2 2] = -3 S[2 3] = -28 S[2 4] = -8 S[2 5] = -11
2
n
6
A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of
A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)
low mid high
mid +1
entirely in A[lowmid] entirely in A[mid+1high]
crossing the midpoint
Possible locations of subarrays of A[lowhigh]
7
FIND-MAX-CROSSING-SUBARRAY (A low mid high)
left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)
8
low mid high
mid +1
A[imid]
j
A[mid+1j]
i
A[ij] comprises two subarrays A[imid] and A[mid+1j]
9
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7
Example
A
S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2
mid =5
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7A
S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8
mid =5
maximum subarray crossing mid is S[49] = 16
10
if high == lowReturn (low high A[low]) base case only one element
else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)
FIND-MAXIMUM-SUBARRAY (A low high)
Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)
2highlow
11
Analyzing time complexity
bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY
T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
6
A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of
A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)
low mid high
mid +1
entirely in A[lowmid] entirely in A[mid+1high]
crossing the midpoint
Possible locations of subarrays of A[lowhigh]
7
FIND-MAX-CROSSING-SUBARRAY (A low mid high)
left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)
8
low mid high
mid +1
A[imid]
j
A[mid+1j]
i
A[ij] comprises two subarrays A[imid] and A[mid+1j]
9
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7
Example
A
S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2
mid =5
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7A
S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8
mid =5
maximum subarray crossing mid is S[49] = 16
10
if high == lowReturn (low high A[low]) base case only one element
else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)
FIND-MAXIMUM-SUBARRAY (A low high)
Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)
2highlow
11
Analyzing time complexity
bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY
T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
7
FIND-MAX-CROSSING-SUBARRAY (A low mid high)
left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)
8
low mid high
mid +1
A[imid]
j
A[mid+1j]
i
A[ij] comprises two subarrays A[imid] and A[mid+1j]
9
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7
Example
A
S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2
mid =5
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7A
S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8
mid =5
maximum subarray crossing mid is S[49] = 16
10
if high == lowReturn (low high A[low]) base case only one element
else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)
FIND-MAXIMUM-SUBARRAY (A low high)
Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)
2highlow
11
Analyzing time complexity
bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY
T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
8
low mid high
mid +1
A[imid]
j
A[mid+1j]
i
A[ij] comprises two subarrays A[imid] and A[mid+1j]
9
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7
Example
A
S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2
mid =5
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7A
S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8
mid =5
maximum subarray crossing mid is S[49] = 16
10
if high == lowReturn (low high A[low]) base case only one element
else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)
FIND-MAXIMUM-SUBARRAY (A low high)
Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)
2highlow
11
Analyzing time complexity
bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY
T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
9
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7
Example
A
S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2
mid =5
1 2 3 4 5 6 7 8 9 10
13 -3 -25 20 -3 -16 -23 18 20 -7A
S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8
mid =5
maximum subarray crossing mid is S[49] = 16
10
if high == lowReturn (low high A[low]) base case only one element
else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)
FIND-MAXIMUM-SUBARRAY (A low high)
Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)
2highlow
11
Analyzing time complexity
bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY
T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
10
if high == lowReturn (low high A[low]) base case only one element
else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)
FIND-MAXIMUM-SUBARRAY (A low high)
Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)
2highlow
11
Analyzing time complexity
bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY
T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
11
Analyzing time complexity
bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY
T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
12
Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB
1
kj
n
kikij bac
An O(n3) time naive algorithm
SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj
return C
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
13
Divide-and-Conquer Algorithm
bull Assume that n is an exact power of 2
2221
1211
2221
1211
2221
1211 CC
CCC
BB
BBB
AA
AAA
2221
1211
2221
1211
2221
1211
BB
BB
AA
AA
CC
CC
(41)
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
14
Divide-and-Conquer Algorithm
2112111111 BABAC
2122112121 BABAC
2212121112 BABAC
2222122122 BABAC
A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)
= (n3)Computing A+B O(n2)
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
15
Strassenrsquos method
222131211222121 AASAASBBS
221162211511214 BBSAASBBS
211192221822127 AASBBSAAS
121110 BBS (42)
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
16
Strassenrsquos method
1111 SAP
2222 BSP
1133 BSP
4224 SAP
655 SSP
876 SSP
1097 SSP
624511 PPPPC
2112 PPC
4321 PPC
731522 PPPPC
(43)
(44)
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
17
Strassenrsquos divide-and-conquer algorithm
bull Step 1 Divide each of A B and C into four sub-matrices as in (41)
bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)
bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)
bull Step 4 Compute according to (44)
22211211 CCCC
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
18
Time complexity
T(n) = 7T(n2) + (n2)
= (nlg 7 ) (why)
= (n281)
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
19
Discussion
bull Strassenrsquos method is largely of theoretical interest for n 45
bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)
bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
20
Discussion
bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)
bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)
bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
21
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull eg to show upper bound we find constants c and
n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip
2n
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
22
Substitution Method(if we know the answer)
How to solve this T(n) = 2T( ) + n with T(1) = 1
1 Make a guess eg T(n) = O(n log n)
2 Show it by induction bull Firstly T(2) = 4 T(3) = 5
We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay
bull Other Cases
2n
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
23
Substitution Method(if we know the answer)
bull Induction CaseAssume the guess is true for all n = 23hellipkFor n = k+1 we have
T(n) = 2T( ) + n
= cn lg n ndash cn + n cn log n
Induction case is true
2n
nnnc 2lg22
nncn 2lg
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
24
Substitution Method(if we know the answer)
Q How did we know the value of c and n0
A If induction works the induction case must be
correct c ge 1 Then we find that by setting c = 2 our guess is
correct as soon as n0 = 2
Alternatively we can also use c = 15 Then we just need a larger n0 = 4
(What will be the new base cases Why)
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
25
Substitution Method(New Challenge)
How to solve this
1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case
1)1( 1)2()2()( TnTnTnT
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
26
Substitution Method(New Challenge)
Induction Case (assume guess is true for some base cases)
This term is not what we
want hellip
1
122
1)2()2()(
cn
ncnc
nTnTnT
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
27
Substitution Method(New Challenge)
bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo
Recall Induction may become easier if we prove a ldquostrongerrdquo statement
2nd Attempt Refine our statementTry to show T(n) le cn - b instead
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
28
Substitution Method(New Challenge)
bcn
bncbnc
nTnTnT
122
1)2()2()(
Induction Case
It remains to find c and n0 and prove the base case(s) which is relatively easy
We get the desired term (when b 1)
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
29
Substitution Method(New Challenge 2)
How to solve this
T(n) = 2T( ) + lg n n
Hint Change variable Set m = lg n
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
30
Substitution Method(New Challenge 2)
Set m = lg n we get
T(2m) = 2T(2m2) + m
Next set S(m) = T(2m) = T(n)
S(m) = 2S(m2) + m
We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
31
Recursion Tree Method( Nothing Specialhellip Very Useful )
How to solve this T(n) = 2T(n2) + n2 with T(1) = 1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
32
Recursion Tree Method( Nothing Specialhellip Very Useful )
Expanding the terms we get
T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =
= (n2) + (n) = (n2)
)1(2)21( lg21lg
0Tn nn
k
k
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
33
Recursion Tree Method( Recursion Tree View )
We can express the previous recurrence by
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
34
Further expressing gives us
This term is from T(n2)
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
35
Recursion Tree Method( New Challenge )
How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1
What will be the recursion tree view
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
36
The corresponding recursion tree view is
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
37
Master Method( Save our effort )
When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately
The Master Theorem has 3 cases hellip
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
38
Master Theorem
Theorem (Case 1)
If f(n) = O(nlogb a - ) for some constant 0
then T(n) = (nlogb a)
Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants
wherewe interpret nb to mean either nb or nb
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
39
Theorem (Case 2)
If f(n) = (nlogb a)
then T(n) = (nlogb a lg n)
Theorem (Case 3)
If f(n) = (nlogb a + ) for some constant 0
and if af(nb) c f(n) for some constant c 1 and all sufficiently large n
then T(n) = (f(n))
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
40
Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)
2 Solve T(n) = T(2n3) + 1 (case 2)
3 Solve T(n) = 3T(n4) + nlgn (case 3)
4 How about this T(n) = 2T(n2) + n lg n
5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n
6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1
41
Homework
bull Exercise 41-3 (due Oct 12)bull Practice at home 41-5 42-142-5bull Exercise 45-4 Problem 41(bdf)(due Oct 19)bull Practice at home 43-744-843-644-745-1