Book: Introduction to Algorithms , by: Thomas H. Cormen Charles E. Leiserson
© 2001 by Charles E. Leiserson Introduction to AlgorithmsDay 17 L9.1 Introduction to Algorithms...
-
Upload
jenna-mcintyre -
Category
Documents
-
view
224 -
download
0
Transcript of © 2001 by Charles E. Leiserson Introduction to AlgorithmsDay 17 L9.1 Introduction to Algorithms...
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.1
Introduction to Algorithms6.046J/18.401J/SMA5503
Lecture 9Prof. Charles E. Leiserson
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.2
Binary-search-tree sort
Create an empty BSTfor i = 1 to n do TREE-INSERT (T, A[i])Perform an inorder tree walk of T.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.3
Analysis of BST sortBST sort performs the same comparisons asquicksort, but in a different order!
The expected time to build the tree is asymptoticallythe same as the running time of quicksort.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.4
Node depthThe depth of a node = the number of comparisonsmade during TREE-INSERT. Assuming all inputpermutations are equally likely, we have
Average node depth
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.5
Expected tree heightBut, average node depth of a randomly builtBST = O(lg n) does not necessarily mean that itsexpected height is also O(lg n) (although it is).
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.6
Height of a randomly builtbinary search tree
Outline of the analysis:• Prove Jensen’s inequality, which says thatf(E[X]) ≤ E[f(X)] for any convex function f andrandom variable X.• Analyze the exponential height of a randomlybuilt BST on n nodes, which is the randomvariable Yn = 2Xn, where Xn is the randomvariable denoting the height of the BST.• Prove that 2E[Xn] ≤ E[2Xn ] = E[Yn] = O(n3),and hence that E[Xn] = O(lg n).
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.7
Convex functionsA function f : R → R is convex if for allα,β ≥ 0 such that α + β = 1, we have
f(αx + βy) ≤ αf(x) + βf(y)
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.8
Convexity lemmaLemma. Let f : R → R be a convex function,and let {α1, α2 , …, αn} be a set of nonnegativeconstants such that Σk αk = 1. Then, for any set{x1, x2, …, xn} of real numbers, we have
Proof. By induction on n. For n = 1, we haveα1 = 1, and hence f(α1x1) ≤ α1f(x1) trivially.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.9
Proof (continued)Inductive step:
Algebra.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.10
Proof (continued)Inductive step:
Convexity.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.11
Proof (continued)Inductive step:
Convexity.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.12
Proof (continued)
Inductive step:
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.13
Lemma. Let f be a convex function, and let Xbe a random variable. Then, f (E[X]) ≤ E[ f (X)].
Definition of expectation.
Jensen’s inequality
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.14
Lemma. Let f be a convex function, and let Xbe a random variable. Then, f (E[X]) ≤ E[ f (X)].
Jensen’s inequality
Convexity lemma (generalized).
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.15
Lemma. Let f be a convex function, and let Xbe a random variable. Then, f (E[X]) ≤ E[ f (X)].
Jensen’s inequality
Tricky step, but true—think about it.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.16
Analysis of BST heightLet Xn be the random variable denotingthe height of a randomly built binarysearch tree on n nodes, and let Yn = 2Xn
be its exponential height.
If the root of the tree has rank k, then
Xn= 1 + max{Xk–1, Xn–k} ,
since each of the left and right subtreesof the root are randomly built. Hence,we have
Yn= 2· max{Yk–1, Yn–k} .
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.17
Analysis (continued)Define the indicator random variable Znk as
1 if the root has rank k,
0 otherwise.
Thus, Pr{Znk = 1} = E[Znk] = 1/n, and
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.18
Exponential height recurrence
Take expectation of both sides.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.19
Exponential height recurrence
Linearity of expectation.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.20
Exponential height recurrence
Independence of the rank of the rootfrom the ranks of subtree roots.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.21
Exponential height recurrence
The max of two nonnegative numbersis at most their sum, and E[Znk] = 1/n.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.22
Exponential height recurrence
Each term appearstwice, and reindex.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.23
Solving the recurrence
Use substitution toshow that E[Yn] ≤ cn3
for some positiveconstant c, which wecan pick sufficientlylarge to handle theinitial conditions.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.24
Solving the recurrence
Use substitution toshow that E[Yn] ≤ cn3
for some positiveconstant c, which wecan pick sufficientlylarge to handle theinitial conditions. Substitution.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.25
Solving the recurrence
Use substitution toshow that E[Yn] ≤ cn3
for some positiveconstant c, which wecan pick sufficientlylarge to handle theinitial conditions.
Integral method.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.26
Solving the recurrence
Use substitution toshow that E[Yn] ≤ cn3
for some positiveconstant c, which wecan pick sufficientlylarge to handle theinitial conditions.
Solve the integral.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.27
Algebra.
Solving the recurrence
Use substitution toshow that E[Yn] ≤ cn3
for some positiveconstant c, which wecan pick sufficientlylarge to handle theinitial conditions.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.28
Putting it all together, we have
2E[Xn] ≤ E[2Xn ]
Jensen’s inequality, sincef(x) = 2x is convex.
The grand finale
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.29
The grand finale
Putting it all together, we have
Definition.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.30
The grand finale
Putting it all together, we have
What we just showed.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.31
The grand finale
Putting it all together, we have
Taking the lg of both sides yields
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.32
Post mortem
Q. Does the analysis have to be this hard?
Q. Why bother with analyzing exponential height?
Q. Why not just develop the recurrence on
directly?
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.33
Post mortem (continued)A. The inequality
max{a, b} ≤ a + b .provides a poor upper bound, since the RHSapproaches the LHS slowly as |a – b| increases.The bound
max{2a, 2b} ≤ 2a + 2b
allows the RHS to approach the LHS far morequickly as |a – b| increases. By using theconvexity of f(x) = 2x via Jensen’s inequality,we can manipulate the sum of exponentials,resulting in a tight analysis.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 17 L9.34
Thought exercises
• See what happens when you try to do the analysis on Xn directly.• Try to understand better why the proof uses an exponential. Will a quadratic do?• See if you can find a simpler argument. (This argument is a little simpler than the one in the book—I hope it’s correct!)