DS Sorting

8/6/2019 DS Sorting

1/73

8/6/2019 DS Sorting

2/73

Bubble sortBubble sort

Compare each element (except the last one) with its neighborCompare each element (except the last one) with its neighborto the rightto the right

If they are out of order, swap themIf they are out of order, swap them

This puts the largest element at the very endThis puts the largest element at the very end

The last element is now in the correct and final placeThe last element is now in the correct and final place

Compare each element (except the lastCompare each element (except the last twotwo) with its neighbor to) with its neighbor tothe rightthe right

If they are out of order, swap themIf they are out of order, swap them

This puts the second largest element next to lastThis puts the second largest element next to last

The last two elements are now in their correct and finalThe last two elements are now in their correct and finalplacesplaces

Compare each element (except the lastCompare each element (except the last threethree) with its neighbor) with its neighborto the rightto the right

Continue as above until you have no unsorted elements onContinue as above until you have no unsorted elements onthe leftthe left

8/6/2019 DS Sorting

3/73

8/6/2019 DS Sorting

4/73

Code for bubble sortCode for bubble sort

void bubbleSort(int a[], int size)void bubbleSort(int a[], int size)

{{int outer, inner;int outer, inner;for (outer =sizefor (outer =size -- 1; outer > 0; outer1; outer > 0; outer----) {) { // counting down// counting down

for (inner = 0; inner < outer; inner++) {for (inner = 0; inner < outer; inner++) { // bubbling up// bubbling upif (a[inner] > a[inner + 1]) {if (a[inner] > a[inner + 1]) { // if out of order...// if out of order...

int temp = a[inner];int temp = a[inner]; // ...then swap// ...then swapa[inner] = a[inner + 1];a[inner] = a[inner + 1];

a[inner + 1] = temp;a[inner + 1] = temp;}}}}

}}

}}

8/6/2019 DS Sorting

5/73

Analysis of bubble sortAnalysis of bubble sort for (outer = sizefor (outer = size -- 1; outer > 0; outer1; outer > 0; outer----) {) {

for (inner = 0; inner < outer; inner++) {for (inner = 0; inner < outer; inner++) {if (a[inner] > a[inner + 1]) {if (a[inner] > a[inner + 1]) {

// code for swap omitted// code for swap omitted}}

}}}} LetLet n = sizen = size--1 =1 = size of the arraysize of the array

The outer loop is executedThe outer loop is executed nn--11 times (call ittimes (call it nn, thats close enough), thats close enough)

Each time the outer loop is executed, the inner loop is executedEach time the outer loop is executed, the inner loop is executed

Inner loop executesInner loop executes nn--11 times at first, linearly dropping to just oncetimes at first, linearly dropping to just once

On average, inner loop executes aboutOn average, inner loop executes about n/2n/2 times for each execution oftimes for each execution ofthe outer loopthe outer loop

In the inner loop, the comparison is always done (constantIn the inner loop, the comparison is always done (constant

time), the swap might be done (also constant time)time), the swap might be done (also constant time)

Result isResult is n * n/2 * kn * n/2 * k, that is,, that is, O(nO(n22/2 + k) = O(n/2 + k) = O(n22))

8/6/2019 DS Sorting

6/73

Insertion SortInsertion Sort Implemented by inserting a particular element atImplemented by inserting a particular element at

the appropriate position.the appropriate position. First iteration starts with the comparison of 1First iteration starts with the comparison of 1stst

element with the 0element with the 0thth elementelement

In second iteration 2In second iteration 2ndnd element is compared withelement is compared withthe 0the 0thth element is compared with the 0element is compared with the 0thth and 1and 1stst

element.element.

In general, in every iteration an element isIn general, in every iteration an element iscompared with all the elements before it. And ifcompared with all the elements before it. And ifelement in question can be inserted at suitableelement in question can be inserted at suitable

position, then space created for it by shiftingposition, then space created for it by shifting

elements one position to the rightelements one position to the right

8/6/2019 DS Sorting

7/73

Insertion sortInsertion sort

5 7 0 3 4 2 6 15 7 0 3 4 2 6 1 (0)(0)

5 75 7 00 3 4 2 6 13 4 2 6 1 (0)(0)

005

75

7 33 4 2 6 14 2 6 1 (2)(2)00 33 5 75 7 44 2 6 12 6 1 (2)(2)

0 30 3 44 5 75 7 226 16 1 (2)(2)

00 22 3 4 5 73 4 5 7 6611 (4)(4)

0 2 3 4 50 2 3 4 566 77 11 (1)(1)

00 11 2 3 4 5 6 72 3 4 5 6 7 (6)(6)

8/6/2019 DS Sorting

8/73

Insertion sortInsertion sort

voidvoid insertionsortinsertionsort((intint a [ ],a [ ], intint size)size){{

intint i, temp, j;i, temp, j;for(i=1;i=0;j----)){{

if(a[j]>temp)if(a[j]>temp)

a[j+1]=a[j];a[j+1]=a[j];else break;else break;}}a[j+1]=temp;a[j+1]=temp;

}}

}}

8/6/2019 DS Sorting

9/73

Analysis of insertion sortAnalysis of insertion sort

We run once through the outer loop, inserting each ofWe run once through the outer loop, inserting each ofn elements; this is a factor ofn elements; this is a factor ofnn

On average, there areOn average, there are n/2n/2 elements already sortedelements already sorted

The inner loop looks at (and moves) half of theseThe inner loop looks at (and moves) half of these

This gives a second factor ofThis gives a second factor ofn/4n/4

Hence, the time required for an insertion sort of anHence, the time required for an insertion sort of an

array ofarray ofnn elements is proportional toelements is proportional to nn22/4/4 Discarding constants, we find that insertion sort isDiscarding constants, we find that insertion sort is

O(nO(n22))

8/6/2019 DS Sorting

10/73

SummarySummary

Bubble sort and insertion sort are allBubble sort and insertion sort are all O(nO(n22))

As we will see later, we can do much better than this withAs we will see later, we can do much better than this withsomewhat more complicated sorting algorithmssomewhat more complicated sorting algorithms

WithinWithin O(nO(n22)),,

Bubble sort is very slow, and should probably never be usedBubble sort is very slow, and should probably never be usedfor anythingfor anything

Insertion sort is usually the fastest of the twoInsertion sort is usually the fastest of the two----in fact, forin fact, for

small arrays (say, 10 or 15 elements), insertion sort is fastersmall arrays (say, 10 or 15 elements), insertion sort is faster

than more complicated sorting algorithmsthan more complicated sorting algorithms Aother algorithm selection sort and insertion sort are goodAother algorithm selection sort and insertion sort are good

enough for small arrays.enough for small arrays.

8/6/2019 DS Sorting

11/73

Divide and Conquer AlgorithmsDivide and Conquer Algorithms

Base case: the problem is small enough, solvedirectly

Divide the problem into two or more similar and

smaller subproblems

Recursively solve the subproblems

Combine solutions to the subproblems

We will study two divide and Conquer algorithmsWe will study two divide and Conquer algorithms Merge SortMerge Sort

Quick SortQuick Sort

8/6/2019 DS Sorting

12/73

Divide and Conquer AlgorithmsDivide and Conquer Algorithms

Base case

single element (n=1 ), return

Divide Ainto two subarrays: FirstPart, SecondPart Two Subproblems:

sort the FirstPart

sort the SecondPart

Recursively

sort FirstPart sort SecondPart

Combine sorted FirstPart and sorted second part

8/6/2019 DS Sorting

13/73

voidvoid mergesort(mergesort(intint lo,lo, intint hi)hi)

{{

ifif(lo

8/6/2019 DS Sorting

14/73

MergeSort

8/6/2019 DS Sorting

15/73

MergeSort (Example) - 2

8/6/2019 DS Sorting

16/73


8/6/2019 DS Sorting

17/73


8/6/2019 DS Sorting

18/73


8/6/2019 DS Sorting

19/73


8/6/2019 DS Sorting

20/73


8/6/2019 DS Sorting

21/73


8/6/2019 DS Sorting

22/73


8/6/2019 DS Sorting

23/73


8/6/2019 DS Sorting

24/73


8/6/2019 DS Sorting

25/73


8/6/2019 DS Sorting

26/73


8/6/2019 DS Sorting

27/73


8/6/2019 DS Sorting

28/73


8/6/2019 DS Sorting

29/73


8/6/2019 DS Sorting

30/73


8/6/2019 DS Sorting

31/73


8/6/2019 DS Sorting

32/73


8/6/2019 DS Sorting

33/73

8/6/2019 DS Sorting

34/73


8/6/2019 DS Sorting

35/73


8/6/2019 DS Sorting

36/73

14 23 45 98 6 33 42 67

8/6/2019 DS Sorting

37/73

Merge

23 45 98 33 42 6714 6

8/6/2019 DS Sorting

38/73

Merge

23 45 98 6 42 67

6

14 33

8/6/2019 DS Sorting

39/73

Merge

14 45 98 6 42 67

6 14

23 33

8/6/2019 DS Sorting

40/73

Merge

14 23 98 6 42 67

6 14 23

45 33

8/6/2019 DS Sorting

41/73

Merge

14 23 98 6 33 67

6 14 23 33

45 42

8/6/2019 DS Sorting

42/73

Merge

14 23 98 6 33 42

6 14 23 33 42

45 67

8/6/2019 DS Sorting

43/73

Merge

14 23 45 6 33 42

6 14 23 33 42 45

98 67

8/6/2019 DS Sorting

44/73

Merge

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67

8/6/2019 DS Sorting

45/73

Merge

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67 98

8/6/2019 DS Sorting

46/73

MergeMerge--Sort AnalysisSort Analysis

TimeTime

Most of the work is in the mergingMost of the work is in the merging

Total time:Total time: OO(n log n)(n log n)

Space:Space: OO(n), more space than other sorts.(n), more space than other sorts.

8/6/2019 DS Sorting

47/73

Quick sort AlgorithmQuick sort Algorithm

Given an array ofGiven an array ofnnelements (e.g., integers):elements (e.g., integers):

If array only contains one element, returnIf array only contains one element, return

ElseElse

pick one element to use aspick one element to use as pivot.pivot.

Partition elements into two subPartition elements into two sub--arrays:arrays:

Elements less than or equal to pivotElements less than or equal to pivot

Elements greater than pivotElements greater than pivot

Quicksort two subQuicksort two sub--arraysarrays

Return resultsReturn results

8/6/2019 DS Sorting

48/73

QuicksortQuicksort

To sortTo sort a[left...right]a[left...right]1.1. if left < rightif left < right

1.1.1.1. Partition a[left...right] such thatPartition a[left...right] such thatall a[left...pall a[left...p--1] are less than a[p], and1] are less than a[p], and

all a[p+1...right] are >= a[p]all a[p+1...right] are >= a[p]

1.2.1.2. Quicksort a[left...pQuicksort a[left...p--1]1]1.3.1.3. Quicksort a[p+1...right]Quicksort a[p+1...right]

2.2.TerminateTerminate

8/6/2019 DS Sorting

49/73

PartitioningPartitioning

A key step in the Quicksort algorithm isA key step in the Quicksort algorithm ispartitioningpartitioningthe arraythe array We choose some (any) numberWe choose some (any) number pp in the array to usein the array to use

as aas a pivotpivot WeWe partitionpartition the array into three parts:the array into three parts:

p

numbers =

p

p

8/6/2019 DS Sorting

50/73


Choose an array value (say, the first) to use asChoose an array value (say, the first) to use asthe pivotthe pivot

Starting from the left end, find the firstStarting from the left end, find the first

element that is greater than or equal to theelement that is greater than or equal to thepivotpivot

Searching backward from the right end, findSearching backward from the right end, find

the first element that is less than the pivotthe first element that is less than the pivot Interchange (swap) these two elementsInterchange (swap) these two elements

Repeat, searching from where we left off,Repeat, searching from where we left off,

until doneuntil done

8/6/2019 DS Sorting

51/73


To partition a[left...right]To partition a[left...right]

1.1. Set p = a[left], l = left + 1, r = right;Set p = a[left], l = left + 1, r = right;

2.2.while l < r, dowhile l < r, do2.1.2.1.while l < right & a[l] < p, set l = l + 1while l < right & a[l] < p, set l = l + 1

2.2.2.2.while r > left & a[r] >= p, set r = rwhile r > left & a[r] >= p, set r = r -- 11

2.3.2.3. if l < r, swap a[l] and a[r]if l < r, swap a[l] and a[r]

3.3. Set a[left] = a[r], a[r] = pSet a[left] = a[r], a[r] = p

4.4.TerminateTerminate

8/6/2019 DS Sorting

52/73

Example of partitioningExample of partitioning

choose pivot:choose pivot: 44 3 6 9 2 4 3 1 2 1 8 9 3 5 63 6 9 2 4 3 1 2 1 8 9 3 5 6

search:search: 44 33 66 9 2 4 3 1 2 1 8 99 2 4 3 1 2 1 8 9 33 5 65 6

swap:swap: 44 33 33 9 2 4 3 1 2 1 8 99 2 4 3 1 2 1 8 9 66 5 65 6

search:search: 44 33 33 99 2 4 3 1 22 4 3 1 2 11 8 9 68 9 6 5 65 6

swap:swap: 44 33 33 11 2 4 3 1 22 4 3 1 2 99 8 9 68 9 6 5 65 6

search:search: 44 33 33 1 21 2 44 3 13 1 22 99 8 9 68 9 6 5 65 6

swap:swap: 44 33 33 1 21 2 22 3 13 1 44 99 8 9 68 9 6 5 65 6 search:search: 44 33 33 1 21 2 2 32 3 11 44 99 8 9 68 9 6 5 65 6 (left > right)(left > right)

swap with pivot:swap with pivot: 11 33 33 1 21 2 2 32 3 44 44 99 8 9 68 9 6 5 65 6

8/6/2019 DS Sorting

53/73

Analysis ofQuick sortAnalysis ofQuick sortbest casebest case

Suppose each partition operation divides theSuppose each partition operation divides the

array almost exactly in halfarray almost exactly in half

Then the depth of the recursion isThen the depth of the recursion isloglog

22nn

Because thats how many times we can halveBecause thats how many times we can halve nn

However, there are many recursions!However, there are many recursions!

How can we figure this out?How can we figure this out?

We note thatWe note that

Each partition is linear over its subarrayEach partition is linear over its subarray

All the partitions at one level cover the arrayAll the partitions at one level cover the array

8/6/2019 DS Sorting

54/73

Partitioning at various levelsPartitioning at various levels

8/6/2019 DS Sorting

55/73

Best caseBest case

We cut the array size in half each timeWe cut the array size in half each time

So the depth of the recursion inSo the depth of the recursion in loglog22nn

At each level of the recursion, all the partitionsAt each level of the recursion, all the partitionsat that level do work that is linear inat that level do work that is linear in nn

O(logO(log22n) * O(n) = O(nlogn) * O(n) = O(nlog22n)n)

Hence in the average case, quicksort has timeHence in the average case, quicksort has timecomplexitycomplexityO(nlogO(nlog22n)n)

What about the worst case?What about the worst case?

8/6/2019 DS Sorting

56/73

Worst caseWorst case

In the worst case, partitioning always divides theIn the worst case, partitioning always divides the

sizesize nnarray into these three parts:array into these three parts:

A length one part, containing the pivot itselfA length one part, containing the pivot itself

A length zero part, andA length zero part, and

A lengthA length nn--11part, containing everything elsepart, containing everything else

We dont recur on the zeroWe dont recur on the zero--length partlength part

Recurring on the lengthRecurring on the length nn--11part requires (inpart requires (inthe worst case) recurring to depththe worst case) recurring to depth nn--11

8/6/2019 DS Sorting

57/73

Worst case partitioningWorst case partitioning

8/6/2019 DS Sorting

58/73

8/6/2019 DS Sorting

59/73

Typical case for quicksortTypical case for quicksort

If the array is sorted to begin with, QuicksortIf the array is sorted to begin with, Quicksort

is terrible:is terrible: O(nO(n22))

It is possible to construct other bad casesIt is possible to construct other bad cases However, Quicksort isHowever, Quicksort is usuallyusuallyO(nlogO(nlog22n)n)

The constants are so good that Quicksort isThe constants are so good that Quicksort is

generally the fastest algorithm knowngenerally the fastest algorithm known Most realMost real--world sorting is done by Quicksortworld sorting is done by Quicksort

8/6/2019 DS Sorting

60/73

Picking a better pivotPicking a better pivot

Before, we picked theBefore, we picked thefirstfirstelement of the subarray toelement of the subarray touse as a pivotuse as a pivot

If the array is already sorted, this results inIf the array is already sorted, this results in O(nO(n22))behaviorbehavior

Its no better if we pick theIts no better if we pick the lastlastelementelement

We could do anWe could do an optimaloptimalquicksort (guaranteedquicksort (guaranteedO(n log n)O(n log n)) if we always picked a pivot value that) if we always picked a pivot value thatexactly cuts the array in halfexactly cuts the array in half

Such a value is called aSuch a value is called a medianmedian: half of the values: half of the valuesin the array are larger, half are smallerin the array are larger, half are smaller

The easiest way to find the median is toThe easiest way to find the median is to sortsort thethearray and pick the value in the middle (!)array and pick the value in the middle (!)

8/6/2019 DS Sorting

61/73

Median of threeMedian of three

Obviously, it doesnt make sense to sort the array inObviously, it doesnt make sense to sort the array inorder to find the median to use as a pivotorder to find the median to use as a pivot

Instead, compare justInstead, compare just threethreeelements of our (sub)arrayelements of our (sub)arraythe first, the last, and the middlethe first, the last, and the middle

Take theTake the medianmedian(middle value) of these three as pivot(middle value) of these three as pivot Its possible (but not easy) to construct cases whichIts possible (but not easy) to construct cases which

will make this techniquewill make this technique O(nO(n22))

Suppose we rearrange (sort) these three numbers so thatSuppose we rearrange (sort) these three numbers so that

the smallest is in the first position, the largest in the lastthe smallest is in the first position, the largest in the lastposition, and the other in the middleposition, and the other in the middle

This lets us simplify and speed up the partition loopThis lets us simplify and speed up the partition loop

8/6/2019 DS Sorting

62/73

Heap SortHeap Sort

Uses a heap as its data structure

In-place sorting algorithm memory efficient

Time complexity O(n log(n))

8/6/2019 DS Sorting

63/73

What is a HeapWhat is a Heap

A heap is also known as a priority queue and canA heap is also known as a priority queue and can

be represented by a binary tree with thebe represented by a binary tree with thefollowing properties:following properties:

Structure propertyStructure property::A heap is a completely filled binary treeA heap is a completely filled binary treewith the exception of the bottom row, which is filled from left towith the exception of the bottom row, which is filled from left torightright

Heap Order property

Heap Order property:: For every nodeFor every node xx in the heap, thein the heap, theparent ofparent ofxx greater than or equal to the value ofgreater than or equal to the value ofxx..

(known as a maxHeap).(known as a maxHeap).

8/6/2019 DS Sorting

64/73

Example:Example:

53

44 25

15 21 13 18

3 12 5 7

a heap

8/6/2019 DS Sorting

65/73

AlgorithmAlgorithm

Step 1. Build HeapStep 1. Build Heap O(n)O(n)

Build binary tree taking N items as input, ensuringBuild binary tree taking N items as input, ensuringthe heap structure property is held, in other words,the heap structure property is held, in other words,

build a complete binary tree.build a complete binary tree. Heapify the binary tree making sure the binary treeHeapify the binary tree making sure the binary tree

satisfies the Heap Order property.satisfies the Heap Order property.

Step 2. Perform n deleteMax operationsStep 2. Perform n deleteMax operations O(log(n))O(log(n))

Delete the maximum element in the heapDelete the maximum element in the heap which iswhich isthe root node, and place this element at the end ofthe root node, and place this element at the end ofthe sorted array.the sorted array.

8/6/2019 DS Sorting

66/73

Simplifying thingsSimplifying things For speed and efficiency we can represent the heap with an array. Place the rootFor speed and efficiency we can represent the heap with an array. Place the root

at array index 1, its left child at index 2, its right child at index 3, so on and soat array index 1, its left child at index 2, its right child at index 3, so on and soforthforth

53

44 25

15 21 13 18

3 12 5 7

0 1 2 3 4 5 6 7 8 9 10 11

5353 4444 2525 1515 2121 1313 1818 33 1212 55 77

8/6/2019 DS Sorting

67/73

53

44 25

15 21 13 18

3 12 5 7

0 1 2 3 4 5 6 7 8 9 10 11

5353 4444 2525 1515 2121 1313 1818 33 1212 55 77

For any node i, the following formulas apply:

The index of its parent = i / 2

Index of left child = 2 * i

Index of right child = 2 * i + 1

8/6/2019 DS Sorting

68/73

Sample RunSample Run

Start with unordered array of dataStart with unordered array of dataArray representation:Array representation:

Binary tree representation:Binary tree representation:

2121 1515 2525 33 55 1212 77 1919 4545 22 99

21

15 25

3 5 12 7

19 45 2 9

8/6/2019 DS Sorting

69/73

8/6/2019 DS Sorting

70/73

8/6/2019 DS Sorting

71/73

21

19 12

15 9 5 7

2 3

2121 1919 1212 1515 99 55 77 22 33 2525 4545

19

15 12

3 9 5 7

2

1919 1515 1212 33 99 55 77 22 2121 2525 4545

19

15 12

3 9 5 7

2

1919 1515 1212 33 99 55 77 22 2121 2525 4545

15

9 12

3 2 5 7

1515 99 1212 33 22 55 77 1919 2121 2525 4545

8/6/2019 DS Sorting

72/73

15

9 12

3 2 5 7

1515 99 1212 33 22 55 77 1919 2121 2525 4545

12

9 7

3 2 5

1212 99 77 33 22 55 1515 1919 2121 2525 4545

12

9 7

3 2 5

1212 99 77 33 22 55 1515 1919 2121 2525 4545

9

5 7

3 2

99 55 77 33 22 1212 1515 1919 2121 2525 4545

and finally

2

22 33 55 77 99 1212 1515 1919 2121 2525 4545

8/6/2019 DS Sorting

73/73

ConclusionConclusion

11stst StepStep-- Build heap, O(n) time complexityBuild heap, O(n) time complexity

22ndnd StepStep perform n deleteMax operations, each withperform n deleteMax operations, each withO(log(n)) time complexityO(log(n)) time complexity

total time complexity = O(n log(n))total time complexity = O(n log(n))

Pros: fast sorting algorithm, memory efficient,Pros: fast sorting algorithm, memory efficient,especially for very large values of n.especially for very large values of n.

Cons: slower of the O(n log(n)) sorting algorithmsCons: slower of the O(n log(n)) sorting algorithms

DS Sorting

Documents

Transcript of DS Sorting