SortingLower Bounds

n Beating Them

Recap • Divide and Conquer

– Know how to break a problem into smaller problems, such that

– Given a solution to the smaller problems we can easily find a solution to the bigger problem

Recap• Divide and Conquer

– Divide and solve the smaller pieces recursively

– Solve the big problem using the solutions to the smaller problems.

Recap• The Quicksort algorithm:

– chooses an element, called pivot.– Splits the input into three groups:

• smaller than the pivot• equal to pivot• larger than pivot.

– Recursively sort the smaller and the larger groups independently

How fast can we sort?• Can we do better than O(n lg n)?

• Well… it depends?

• It depends on the model of computation!

Comparison sorting• Only comparisons are used to

determine the order of elements• Example

– Selection Sort– Insertion sort– Merge Sort– Quick Sort– Heapsort

Lower Bounds• How do we prove that any

algorithm will need cnlogn time in the worst case?

• First Assumption: All these algorithms are allowed to only compare elements in the input. (Say only < ?)

I choose a permutation

of {1,2,3,…N}

Comparison Sort Game

I ask you yes/no questions.

Time is the number of questions asked in the worst case.

I answer.I determine the permuation.

Decision TreesAnother way of looking at the game

• Example: 3 element sort (a1, a2, a3)

• One yes/no question? (Is x < y?)

a1? a2

“Less than” leads to left branch“Greater than” leads to right branch

Decision Trees• Example: 3 element sort (a1, a2, a3) a1? a2

a2? a3

Decision Trees• Example: 3 element sort (a1, a2, a3)

a2? a3

a1? a2

a1, a2, a3

Decision Trees• Example: 3 element sort (a1, a2, a3)

a2? a3

a1? a2

a1? a3a2? a3

a1? a3

a1, a2, a3a3, a1, a2

a1, a3, a2

a2, a1, a3

a2, a3, a1 a3, a2, a1

Decision Trees• Each leaf contains a

permutation indicating that permutation of order has been established

• A decision tree can model a comparison sort

Decision Trees• It is the tree of all possible

instruction traces• Running time = length of path• Worst case running time =

length of the longest path (height of the tree)

Bounding log N!

N! = 1 × 2 × 3 × … × N/2 × … × N

N factors each at most N.

N/2 factors each at least N/2.

NN

N/2 N/2

N/2 log(N/2) log(N!) N log(N)

= (N log(N)).

Lower bound for comparison sorting

• Theorem: Any decision tree that sorts n elements has height (n lg n)

• Proof: There are n! leaves

• A height h binary tree has no more than 2h nodes and it should be able to resolve all n! permutations. )lg(

lg2

lg2

)!lg(!2

nnh

ncnnnh

nhnh

Optimal Comparison Sorting Algorithms

• Merge sort• Quick sort• Heapsort