More sorting algorithms: Heap

sort & Radix sort

Heap Data Structure and Heap Sort

(Chapter 7.6)

Basic DefinitionBasic Definition• Depth of a tree

– The depth of a node in a tree is the number of edges in the unique path from the root to that node

– The depth of a tree is the maximum depth of all nodes in the tree

– A leaf in a tree is any node with no children

– Internal node is any node that has at least one child

Depth of tree nodesDepth of tree nodes• Depth of a node:

– If node is the root, then depth = 0– Otherwise, depth of its parent + 1

• Depth of a tree is the maximum depth of its leaves 0

1 1

2 2

A tree of depth 2

TerminologiesTerminologies• Complete binary tree

– Every internal node has two children– All leaves have depth d

• Essentially complete binary tree– It is a complete binary tree down to a depth of

d-1– The nodes with depth d are as far to the left as

possible

A complete binary treeA complete binary tree• A complete binary tree is a binary tree such

that:– All internal nodes have 2 children– All leaves have the same depth d

• Number of nodes at level k = 2k - 1 • Total number of nodes in a complete binary

tree with depth d is n = 2d+1 – 1– Exercise: Proof by induction

7 = 22+1 - 1

A full binary tree of depth = height = 2

A complete binary tree A complete binary tree (cont.) (cont.)

• Number the nodes of a full (complete) binary tree of depth d: – root at depth 0 is numbered 1– The nodes at depth 1, …, d are

numbered consecutively from left to right, in increasing depth

– You can store the nodes in a 1D array in increasing order of node number

1

2 3

5 64 7

Essential complete binary Essential complete binary

treetree • An essential complete binary tree of

depth d and n nodes is a binary tree such that its nodes would have the numbers 1, …, n in a binary tree of depth d.

• The number of nodes 2d n 2d+1 -1• d = lg n (See the next slide for proof)1

2 3

5 64

1

2 3

5 64 7

Depth of an essential Depth of an essential complete binary treecomplete binary tree

• Number of nodes n satisfy: 2d n 2d+1 – 1 (1)• By taking the log base 2, we get: d lg n d + 1 (2) •Since d is integer but lg n may not be an integer, d = lg n • For complete binary tree, d = lg n because (1) & (2) are satisfied for a

complete binary tree too

Heap PropertyHeap Property• Heap A heap is an essentially complete binary

tree such that– The values stored at the nodes come from an

ordered set– The value stored at each node is less than or

equal to the values stored at its children min-heap

• Usage of heap– Heap sorting– Priority queue

Priority Queue• A priority queue is a collection of zero or

more items,– Each item is associated with a priority

• Operations:– Insert a new item– Find the item with the highest priority– Delete the highest priority item

Heapsort Algorithm• Build a heap• For i = 1 to n – 1 {

– Remove the root from the heap and insert it into answer[i]

– Move the rightmost leaf node to the root and remove the rightmost leaf

– Heapify• Rearrange the new tree to support the heap property

} return answer[1..n]

Heap data structureHeap data structure

- - Exercise: Do heapsort using this heapExercise: Do heapsort using this heap

3 2

8 7

18 14 9

29 6

1

23

4 5 6 7

8 9 10

1

1 3 2 8 7 29 6

1 2 3 4 5 6 7 8 9

18 14 9

10

last

array

root = 1

Parent(i) = i/2

Left(i)=2i

Right(i)=2i+1

How to build a heap in the first place?

for i = n/2 downto 1 do heapify

Exercise: Build a min-heap- Take a bottom-up approach- starting from node 5

3 29

8 9

18 14 1

2 6

1

23

4 i = 5 6 7

8 9 10

7

Worst case time complexityfor heaps

• Build heap with n items (n) (Proof: page 292 - 294)

• insert() into a heap with n items (lg n)

• deleteMin() from a heap with n items (lg n)

• findMin() (1)

• Total O(nlgn)

Linear sorts

Radix sort (Chapter 7.9)

Radix sort

• Main idea– Break key into “digit” representation

key = id, id-1, …, i2, i1– "digit" can be a number in any base, a

character, etc

• Radix sort: for i= 1 to d sort “digit” i using a stable sort

• Analysis : (d (stable sort time)) where d is the number of “digit”s

Radix sort- with decimal digits

178139326572294321910368

12345678

910321572294326178368139

910321326139368572178294

139178294321326368572910

Input list Sorted list

Radix sort

• Which stable sort?– Since the range of values of a digit is

small the best stable sort to use is Counting Sort.

– When counting sort is used the time complexity is (d n)) • Good performance when d << n. For

example, consider a case in which you need to sort 10,000,000 social security numbers

– n = 10,000,000 but d = 9 (n) time complexity

Radix sort with unstable digit sort

1713

12

1317

1713

Input listList not sorted

Since unstableand both keys equal to 1

Is Quicksort stable? NO

• Note that data is sorted by key• Since it is unstable, quicksort cannot be used for radix sort• Is mergesort stable? What about insertion sort?

515548

123

485551

Key Data

After partitionof 0 to 2

After partitionof 1 to 2

485551