Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1...
Transcript of Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1...
![Page 1: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/1.jpg)
CS 240Fall 2015
Mike Lam, Professor
Heap Sort
![Page 2: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/2.jpg)
Priority Queues
● Sorting using PQs
– Add all the items to the PQ
– Remove all the items● In sorted order
● Unsorted list implementation
– Similar to selection sort; phase 1 is O(n) and phase 2 is O(n2)
● Sorted list implementation
– Similar to insertion sort; phase 1 is O(n2) and phase 2 is O(n)
● Heap implementation
– New sorting algorithm: "heap sort"
– Not divide-and-conquer, but still O(n log n)
![Page 3: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/3.jpg)
Heaps for Sorting
● Linked-based heap implementation
– Requires O(n) extra memory
– Can use min or max heaps
– Add operations: O(n log n)
– Remove operations: O(n log n)
● In-place array heap implementation
– No extra memory required
– Need to use max heaps
– Add operations: O(n log n) or O(n)
– Remove operations: O(n log n)
![Page 4: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/4.jpg)
Heap Implementation
● Because heaps are complete trees, there is a veryconvenient array-based representation
● Breadth-first traversal (level numbering)
– Assign each node in the tree an index
– The root is index 0
– The left subchild of node k is index 2k+1
– The right subchild of node k is index 2k+2
– The parent of node k is at index floor((k-1) / 2)
![Page 5: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/5.jpg)
Heap Sort
● Basic idea: build heap in-place then repeatedlyremove max item
● Phase 1 ("heapification")
– Start with single-item max heap w/ first item in list
– Add each subsequent item to the heap
● Phase 2 (sorting)
– Repeatedly remove the maximum item and storing itat the end of the list in a down-ward growing sortedregion
– Down-heap bubbling to restore heap-order property
![Page 6: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/6.jpg)
Heap Sort
bubble_up (items, i):
while i > 0 && i and PARENT(i) are out of order: swap items[i] and PARENT(i) i = PARENT(i)
bubble_down (items, n, i):
while (i < n): max = max of i, LEFT(i), and RIGHT(i) if need to swap items at i and max: swap items[i] and items[max] i = max else: break
![Page 7: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/7.jpg)
Heap Sort
heap_sort (items, n):
// top-down construction for i in [1, n-1]: bubble_up(items, i)
// remove each item and place it in the sorted region for j in [n-1, 1] by -1: swap items[0] and items[j] bubble_down(items, j, 0)
![Page 8: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/8.jpg)
Example
● List: [17, 25, 100, 2, 3, 36, 1, 7, 19]
![Page 9: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/9.jpg)
Example
17
25 17 100
17 25
start with 17
add 25
bubble up
25 17 100 add 100
100 17 25 bubble up
100 17 25add 2(no bubbling needed)
2
100 17 25 2 3 add 3(no bubbling needed)
36 1 7 19
3 36 1 7 19
2 3 36 1 7 19
2 3 36 1 7 19
2 3 36 1 7 19
100 2 3 36 1 7 19
25 100 2 3 36 1 7 19
![Page 10: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/10.jpg)
Example
100 17 25 2 3 36 add 36
100 17 36 2 3 25 bubble up
100 17 36 2 3 25 1add 1(no bubbling needed)
100 17 36 2 3 25 1 add 77 19 ...
100 17 36 7 3 25 1 bubble up2 19 ...
100 17 36 7 3 25 1 add 192 19 ...
100 19 36 17 3 25 1 bubble up2 7 ...
7 19
1 7 19
1 7 19
![Page 11: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/11.jpg)
Example
● List: [17, 25, 100, 2, 3, 36, 1, 7, 19]
● Heap: [100, 19, 36, 17, 3, 25, 1, 2, 7]
![Page 12: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/12.jpg)
Example
100 19 36 17 3 25 1 original heap2 7
7 19 36 17 3 25 1 remove max2 100
36 19 25 17 3 7 1 2 100 bubble down
2 19 25 17 3 7 1 36 100 remove max
25 19 7 17 3 2 1 36 100 bubble down
1 19 7 17 3 2 27 36 100 remove max
19 17 7 1 3 2 27 36 100 bubble down
![Page 13: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/13.jpg)
Example
19 17 7 1 3 2 27 36 100
2 17 7 1 3 19 27 36 100 remove max
17 3 7 1 2 19 27 36 100 bubble down
2 3 7 1 17 19 27 36 100 remove max
7 3 2 1 17 19 27 36 100 bubble down
1 3 2 7 17 19 27 36 100 remove max
3 1 2 7 17 19 27 36 100 bubble down
![Page 14: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/14.jpg)
Example
3 1 2 7 17 19 27 36 100 bubble down
2 1 3 7 17 19 27 36 100remove max(no bubbling needed)
1 2 3 7 17 19 27 36 100 done
![Page 15: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/15.jpg)
Example
● List: [17, 25, 100, 2, 3, 36, 1, 7, 19]
● Heap: [100, 19, 36, 17, 3, 25, 1, 2, 7]
● Final: [1, 2, 3, 7, 17, 19, 25, 36, 100]
![Page 16: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/16.jpg)
Heap Sort Analysis
● Phase 1 (heap grows):
– If each add() operation requires O(log n) time, thisphase will require O(n log n) time
● Phase 2 (heap shrinks):
– Each remove_max() operation requires O(log n) time, so this phase will require O(n log n) time
![Page 17: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/17.jpg)
Heap Sort Analysis
● Phase 1 (heap grows):
– If each add() operation requires O(log n) time, thisphase will require O(n log n) time
● Phase 2 (heap shrinks):
– Each remove_max() operation requires O(log n) time, so this phase will require O(n log n) time
Can we do any better?
![Page 18: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/18.jpg)
Heapification
● Current approach: up-heap bubbling
– Bubble up each newly added item● Top-down heap construction
– Worst-case running time: O(n log n)
● Another option: down-heap bubbling
– Possible when we have all elements in advance● Bottom-up heap construction
– Bubble down from each non-leaf node● Fewer nodes belong in the interior, so this is better in the long run
(see argument in 11.1.3)
– Worst-case running time: O(n)
![Page 19: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/19.jpg)
Heapification
● Benefit of bottom-up construction
Image taken from: https://en.wikipedia.org/wiki/Heapsort
![Page 20: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/20.jpg)
Heap Sort
● Worst case: O(n log n)
– Even if we use bottom-up heapification, because removals are O(n log n)
● In-place
● Not stable
– Up-heap and down-heap bubbling does not preserve ordering of equalelements
● No improvement for nearly-ordered lists
– Still builds heap, re-ordering elements twice
● However, no pathological cases
● Good alternative to quick sort in certain cases
– Example: intro sort
![Page 21: Mike Lam, ProfessorHeap Sort Basic idea: build heap in-place then repeatedly remove max item Phase 1 ("heapification") – Start with single-item max heap w/ first item in list –](https://reader034.fdocuments.net/reader034/viewer/2022050205/5f5898070af3fd40d27000f0/html5/thumbnails/21.jpg)
Heap Sort
● Good example of CS 240 cross-cutting
● Abstract data type (priority queue) to solve problem
– Sorting data
● Concrete data structure (heap) to implement ADT w/ certainproperties
– No additional memory
– O(1) access to parents and children
– O(log n) additions and removals
● Big picture: clever data structure enabling an efficientalgorithm