CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.
-
date post
19-Dec-2015 -
Category
Documents
-
view
230 -
download
8
Transcript of CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.
![Page 1: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/1.jpg)
CS 253: Algorithms
Chapter 6
Heapsort
Appendix B.5
Credit: Dr. George Bebis
![Page 2: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/2.jpg)
2
Special Types of Trees
Def: Full binary tree a binary tree in which each node is either a leaf or has degree exactly 2.
Def: Complete binary treea binary tree in which all leaves are on the same level and all internal nodes have degree 2.
Full binary tree
2
14 8
1
16
7
4
3
9 10
12
Complete binary tree
2
1
16
4
3
9 10
![Page 3: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/3.jpg)
Definitions Height of a node = the number of edges on the
longest simple path from the node down to a leaf
Level of a node = the length of a path from the root to the node
Height of tree = height of root node
2
14 8
1
16
4
3
9 10
Height of root = 3
Height of (2)= 1 Level of (10)= 2
![Page 4: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/4.jpg)
Useful Properties
2
14 8
1
16
4
3
9 10
Height of root = 3
Height of (2)= 1 Level of (10)= 2
• There are at most 2l nodes at level (or depth) l of a binary tree
• A binary tree of height h has at most (2h+1 – 1) nodes
• **here A binary tree with n nodes has height at least lgn. (see Ex
6.1-1&2)
1212
122 1
1
0
h
hh
i
in
![Page 5: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/5.jpg)
5
The Heap Data Structure
Def: A heap is a nearly complete binary tree with the following two properties:◦ Structural property: all levels are full, except
possibly the last one, which is filled from left to right◦ Order (heap) property: for any node x,
Parent(x) ≥ x
Heap
5
7
8
4
2
From the heap property, it follows that:
The root is the maximum element of the heap!
![Page 6: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/6.jpg)
Array Representation of Heaps
A heap can be stored as an array A.
◦ Root of tree is A[1]
◦ Left child of A[i] = A[2i]
◦ Right child of A[i] = A[2i + 1]
◦ Parent of A[i] = A[ i/2 ]
◦ Heapsize[A] ≤ length[A]
The elements in the subarray A[(n/2+1) .. n] are leaves
![Page 7: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/7.jpg)
7
Heap Types
Max-heaps (largest element at root), have the
max-heap property: ◦ for all nodes i, excluding the root: A[PARENT(i)] ≥ A[i]
Min-heaps (smallest element at root), have the
min-heap property:◦ for all nodes i, excluding the root: A[PARENT(i)] ≤ A[i]
![Page 8: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/8.jpg)
8
Operations on Heaps
Maintain/Restore the max-heap property◦ MAX-HEAPIFY
Create a max-heap from an unordered array◦ BUILD-MAX-HEAP
Sort an array in place◦ HEAPSORT
Priority queues
![Page 9: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/9.jpg)
9
Maintaining the Heap PropertyMAX-HEAPIFY
Suppose a node i is smaller than a child and Left and Right subtrees of i are max-
heaps
How do you fix it?
To eliminate the violation: Exchange node i with the larger child Move down the tree Continue until node is not smaller than
children
◦ MAX-HEAPIFY
![Page 10: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/10.jpg)
ExampleMAX-HEAPIFY(A, 2, 10)
A[2] violates the heap property
A[2] A[4]
A[4] violates the heap property
A[4] A[9]
Heap property restored
![Page 11: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/11.jpg)
Maintaining the Heap Property
Assumptions:◦Left and Right
subtrees of i are max-heaps
◦A[i] may be smaller than its children
Alg: MAX-HEAPIFY(A, i, n)1. l ← LEFT(i)2. r ← RIGHT(i)3. if l ≤ n and A[l] > A[i]4. then largest ←l5. else largest ←i6. if r ≤ n and A[r] > A[largest]7. then largest ←r8. if largest i9. then exchange A[i] ↔
A[largest]10. MAX-HEAPIFY(A,
largest, n)
![Page 12: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/12.jpg)
MAX-HEAPIFY Running Time
It traces a path from the root to a leaf (longest path length h)
At each level, it makes two comparisons
Total number of comparisons = 2h
Running Time of MAX-HEAPIFY = O(2h) = O(lgn)
![Page 13: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/13.jpg)
13
Building a Heap
Alg: BUILD-MAX-HEAP(A)
1. n = length[A]
2. for i ← n/2 downto 1
3. do MAX-HEAPIFY(A, i, n)
Convert an array A[1 … n] into a max-heap (n = length[A])
The elements in the subarray A[(n/2+1) .. n] are leaves
Apply MAX-HEAPIFY on elements between 1 and n/2 (bottom up)
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
4 1 3 2 16 9 10 14 8 7A:
![Page 14: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/14.jpg)
Example: A4 1 3 2 16 9 10 14 8 7
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
14
2 8
1
16
7
4
10
9 3
1
2 3
4 5 6 7
8 9 10
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 1014
2 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
14
2 8
16
7
1
4
10
9 3
1
2 3
4 5 6 7
8 9 108
2 4
14
7
1
16
10
9 3
1
2 3
4 5 6 7
8 9 10
i = 5 i = 4 i = 3
i = 2 i = 1
![Page 15: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/15.jpg)
15
Running Time of BUILD MAX HEAP
Running time: O(nlgn)
This is not an asymptotically tight upper bound! Why?
Alg: BUILD-MAX-HEAP(A)
1. n = length[A]
2. for i ← n/2 downto 1
3. do MAX-HEAPIFY(A, i,
n)
O(lgn) O(n)
![Page 16: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/16.jpg)
Running Time of BUILD MAX HEAP
MAX-HEAPIFY takes O(h) the cost of MAX-HEAPIFY on a node i proportional to the height of the node i in the tree
Height Level
h0 = 3 (lgn)
h1 = 2
h2 = 1
h3 = 0
i = 0
i = 1
i = 2
i = 3 (lgn)
No. of nodes
20
21
22
23
hi = h – i height of the heap rooted at level ini = 2i number of nodes at level i
)(2)(00
nOihhnnTh
i
ii
h
ii
see the next slide
![Page 17: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/17.jpg)
i
h
iihnnT
0
)( Cost of HEAPIFY at level i (# of nodes at that level)
ihh
i
i 0
2 Replace the values of ni and hi computed before
hh
iih
ih2
20
Multiply by 2h both at the nominator and denominator
and write 2i as (1/2-i)
h
kk
h k
0 22 Change variables: k = h - i
0 2k
k
kn The sum above is smaller than
the sum of all elements to and h = lgn
)(nORunning time of BUILD-MAX-
HEAPT(n) = O(n)
Running Time of BUILD MAX HEAP
2)2/11(
2/1
n x=1/2 in the summation
20 )1( x
xkx
k
k
![Page 18: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/18.jpg)
18
Goal:
◦ Sort an array using heap representations
Idea:
◦ Build a max-heap from the array
◦ Swap the root (the maximum element) with
the last element in the array
◦ “Discard” this last node by decreasing the
heap size
◦ Call MAX-HEAPIFY on the new root
◦ Repeat this process until only one node
remains
Heapsort
![Page 19: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/19.jpg)
19
Example: A=[7, 4, 3, 1, 2]
MAX-HEAPIFY(A, 1, 4) MAX-HEAPIFY(A, 1, 3) MAX-HEAPIFY(A, 1, 2)
MAX-HEAPIFY(A, 1, 1)
![Page 20: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/20.jpg)
20
Alg: HEAPSORT(A)
1. BUILD-MAX-HEAP(A)
2. for i ← length[A] downto 2
3. do exchange A[1] ↔ A[i]
4. MAX-HEAPIFY(A, 1, i -
1)
Running time: O(nlgn)
Can be shown to be Θ(nlgn)
O(n)
O(lgn)
n-1 times
![Page 21: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/21.jpg)
Priority Queues
Each element is associated with a value (priority)
The key with the highest (lowest) priority is extracted first
Support the following operations:
◦ INSERT(S, x) : inserts element x into set S
◦ EXTRACT-MAX(S) : removes and returns element of S with largest
key
◦ MAXIMUM(S) : returns element of S with largest key
◦ INCREASE-KEY(S, x, k) : increases value of element x’s key to k
(Assume k ≥ x’s current key value)
14 5
![Page 22: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/22.jpg)
22
HEAP-MAXIMUM
Goal:◦ Return the largest element of
the heap
Alg: HEAP-MAXIMUM(A)1. return A[1]
Running time: O(1)
Heap A:
Heap-Maximum(A) returns 7
![Page 23: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/23.jpg)
23
HEAP-EXTRACT-MAXGoal:
◦ Extract the largest element of the heap (i.e., return the max value and also remove that element from the heap
Idea: ◦ Exchange the root element with the last
◦ Decrease the size of the heap by 1 element
◦ Call MAX-HEAPIFY on the new root, on a heap of size n-1Heap A: Root is the largest element
![Page 24: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/24.jpg)
Example: HEAP-EXTRACT-MAX
8
2 4
14
7
1
16
10
9 3
max = 168
2 4
14
7
1
10
9 3
Heap size decreased with 1
4
2 1
8
7
14
10
9 3
Call MAX-HEAPIFY(A, 1, n-1)
![Page 25: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/25.jpg)
25
HEAP-EXTRACT-MAX
Alg: HEAP-EXTRACT-MAX(A, n)
1. if n < 1
2. then error “heap underflow”
3. max ← A[1]
4. A[1] ← A[n]
5. MAX-HEAPIFY(A, 1, n-1) % remakes heap
6. return max
Running time: O(lgn)
![Page 26: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/26.jpg)
HEAP-INCREASE-KEYGoal:
◦ Increases the key of an element i in the heap
Idea:◦ Increment the key of A[i] to its new value◦ If the max-heap property does not hold
anymore: traverse a path toward the root to find the proper place for the newly increased key
8
2 4
14
7
1
16
10
9 3i
Key [i] ← 15
![Page 27: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/27.jpg)
Example: HEAP-INCREASE-KEY
14
2 8
15
7
1
16
10
9 3
i
8
2 4
14
7
1
16
10
9 3i
Key [i ] ← 15
8
2 15
14
7
1
16
10
9 3i
15
2 8
14
7
1
16
10
9 3i
![Page 28: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/28.jpg)
Alg: HEAP-INCREASE-KEY(A, i, key)
1. if key < A[i]
2. then error “new key is smaller than current key”
3. A[i] ← key
4. while i > 1 and A[PARENT(i)] < A[i]
5. do exchange A[i] ↔ A[PARENT(i)]
6. i ← PARENT(i)
Running time: O(lgn)8
2 4
14
7
1
16
10
9 3i
Key [i] ← 15
![Page 29: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/29.jpg)
29
-
MAX-HEAP-INSERT
Goal: Inserts a new element into a
max-heap
Idea: Expand the max-heap with a
new element whose key is -
Calls HEAP-INCREASE-KEY to set the key of the new node to its correct value and maintain the max-heap property
8
2 4
14
7
1
16
10
9 3
15
8
2 4
14
7
1
16
10
9 3
![Page 30: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/30.jpg)
30
Example: MAX-HEAP-INSERT
-
8
2 4
14
7
1
16
10
9 3
Insert value 15:Start by inserting -
15
8
2 4
14
7
1
16
10
9 3
Increase the key to 15Call HEAP-INCREASE-KEY on A[11] = 15
7
8
2 4
14
15
1
16
10
9 3
The restored heap containingthe newly added element
7
8
2 4
15
14
1
16
10
9 3
![Page 31: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/31.jpg)
Alg: MAX-HEAP-INSERT(A, key,
n)
1. heap-size[A] ← n + 1
2. A[n + 1] ← -
3. HEAP-INCREASE-KEY(A, n + 1, key)
Running time: O(lgn)
-
8
2 4
14
7
1
16
10
9 3
![Page 32: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/32.jpg)
32
Summary
We can perform the following operations on
heaps:
◦ MAX-HEAPIFY O(lgn)
◦ BUILD-MAX-HEAP O(n)
◦ HEAP-SORT O(nlgn)
◦ MAX-HEAP-INSERT O(lgn)
◦ HEAP-EXTRACT-MAX O(lgn)
◦ HEAP-INCREASE-KEY O(lgn)
◦ HEAP-MAXIMUM O(1)
Average:
O(lgn)
![Page 33: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/33.jpg)
33
Priority Queue Using Linked List
Average: O(n)
Increase key: O(n)
Extract max key: O(1)
12 4
![Page 34: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/34.jpg)
Problemsa. What is the maximum number of
nodes in a max heap of height h?
b.What is the maximum number of leaves?
c. What is the maximum number of internal nodes?
d.Assuming the data in a max-heap are distinct, what are the possible locations of the second-largest element?
![Page 35: CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d395503460f94a1343e/html5/thumbnails/35.jpg)
35
Demonstrate, step by step, the operation of Build-Heap on the array A = [5, 3, 17, 10, 84, 19, 6, 22, 9]
Let A be a heap of size n. Give the most efficient algorithm for the following tasks:
(a) Find the sum of all elements
(b) Find the sum of the largest lgn elements
Problems