Unit vii Sorting

1

Unit – VII Sorting

Prepared By:Dabbal Singh Mahara

2

• Introduction• Bubble sort• Insertion sort• Selection sort• Quick sort• Merge sort• Comparison and efficiency of sorting

Contents

3

Sorting is among the most basic problems in algorithm design. Sorting is important because it is often the first step in more complex

algorithms. Sorting is to take an unordered set of comparable items and arrange them

in some order. That is, sorting is a process of arranging the items in a list in some order

that is either ascending or descending order. Let a[n] be an array of n elements a0,a1,a2,a3........,an-1 in memory. The

sorting of the array a[n] means arranging the content of a[n] in either increasing or decreasing order.

i.e. a0<=a1<=a2<=a3<.=.......<=an-1

Introduction

• Efficient sorting is important for optimizing the use of other algorithms (such as search and merge algorithms) that require sorted lists to work correctly.

4

Terminology● Internal Sort:

Internal sorting algorithms assume that data is stored in an array in main memory of computer. These methods are applied to small collection of data. That is, the entire collection of data to be sorted is small enough that the sorting can take place within main memory. Examples are: Bubble, Insertion, Selection, Quick, merge etc.

• External Sort: When collection of records is too large to fit in the main memory, records must reside in peripheral or external memory. The only practical way to sort it is to read some records from the disk do some rearranging then write back to disk. This process is repeated until the file is sorted. The sorting techniques to deal with these problems are called external sorting. Sorting large collection of records is central to many applications, such as processing of payrolls and other business databases. . Example: external merge sort

5

•In-place SortThe algorithm uses no additional array storage, and hence (other than perhaps the system’s recursion stack) it is possible to sort very large lists without the need to allocate additional working storage. Examples are: Bubble sort, Insertion Sort, Selection sort

• Stable Sort: sort is said to be stable if elements with equal keys in the input list are

kept in the same order in the output list. If all keys are different then this distinction is not necessary. But if there are equal keys, then a sorting algorithm is stable if whenever there are two records (let's say R and S) with the same key, and R appears before S in the original list, then R will always appear before S in the sorted list. However, assume that the following pairs of numbers are to be sorted by their

first component:• (4, 2) (3, 7) (3, 1) (5, 6)• (3, 7) (3, 1) (4, 2) (5, 6) (order maintained)• (3, 1) (3, 7) (4, 2) (5, 6) (order changed)

• Adaptation to Input:if the sorting algorithm takes advantage of the sorted or nearly sorted input, then the algorithm is called adaptive otherwise not. Example: insertion sort is adaptive

6

512354277 101

1 2 3 4 5 6

5 12 35 42 77 101

1 2 3 4 5 6

• Input: A sequence of n numbers a1, a2, . . . , an

• Output: A permutation (reordering) a1’, a2’, . . . , an’ of the input

sequence such that a1’ ≤ a2’ ≤ · · · ≤ an’

The Sorting Problem

7

Elementary Sorting methodsBubble Sort:

• The basic idea of this sort is to pass through the array sequentially several times.

• Each pass consists of comparing each element in the array with its successor (for example a[i] with a[i + 1]) and interchanging the two elements if they are not in the proper order.

• After each pass an element is placed in its proper place and is not considered in succeeding passes.

8

"Bubbling Up" the Largest Element

• Traverse a collection of elements– Move from the front to the end– “Bubble” the largest value to the end using pair-

wise comparisons and swapping

512354277 101

1 2 3 4 5 6

9




512354277 101

1 2 3 4 5 6

Swap42 77

10




512357742 101

1 2 3 4 5 6

Swap35 77

11




512773542 101

1 2 3 4 5 6

Swap12 77

12




577123542 101

1 2 3 4 5 6

No need to swap

13




577123542 101

1 2 3 4 5 6

Swap5 101

14




77123542 5

1 2 3 4 5 6

101

Largest value correctly placed

15

Items of Interest

• Notice that only the largest value is correctly placed

• All other values are still out of order• So we need to repeat this process

77123542 5

1 2 3 4 5 6

101

Largest value correctly placed

16

Repeat “Bubble Up” How Many Times?• If we have N elements…

• And if each time we bubble an element, we place it in its correct location…

• Then we repeat the “bubble up” process N – 1 times.

• This guarantees we’ll correctly place all N elements.

“Bubbling” All the Elements

77123542 5

1 2 3 4 5 6

101

5421235 77

1 2 3 4 5 6

101

42 5 3512 77

1 2 3 4 5 6

101

42 35 512 77

1 2 3 4 5 6

101

42 35 12 5 77

1 2 3 4 5 6

101

N -

1

Reducing the Number of Comparisons

12 35 42 77 101

1 2 3 4 5 6

5

77123542 5

1 2 3 4 5 6

101

5421235 77

1 2 3 4 5 6

101

42 5 3512 77

1 2 3 4 5 6

101

42 35 512 77

1 2 3 4 5 6

101

19

AlgorithmBubbleSort(A, n){

for(i = 0; i <n-1; i++){

for(j = 0; j < n-i-1; j++){

if(A[j] > A[j+1]){

temp = A[j];A[j] = A[j+1];A[j+1] = temp;

} }}

}

20

Properties:· Stable · O(1) extra space· O(n2) comparisons and swaps· Adaptive: O(n) when nearly sorted input· Higher overhead than insertion sort

Time Complexity:Inner loop executes for (n-1) times when i=0, (n-2) times when i=1 and so on:Time complexity = (n-1) + (n-2) + (n-3) + …………………………. +2 +1 = O(n2)

Space Complexity:Since no extra space besides 3 variables is needed for sortingSpace complexity = O(n)

21

Here, we notice that after each pass, an element is placed in its proper order and is not considered succeeding passes. Furthermore, we need n-1 passes to sort n elements.

Example:

22

Selection Sort

Idea: Find the least (or greatest) value in the array, swap it into the leftmost(or rightmost)component (where it belongs), and then forget the leftmost component. Do this repeatedly.

Let a[n] be a linear array of n elements. The selection sort works as follows:

pass 1: Find the location loc of the smallest element in the list of n elements a[0], a[1], a[2], a[3], …......,a[n-1] and then interchange a[loc] and a[0].

Pass 2: Find the location loc of the smallest element in the sub-list of n-1 elements a[1], a[2],a[3], …......,a[n-1] and then interchange

a[loc] and a[1] such that a[0], a[1] ............. and so on.

Finally, we will get the sorted list a[0]<=a[1]<= a[2]<=a[3]<= .....<= a[n-1].

Selection Sort Example

23

Algorithm:SelectionSort(A){

for( i = 0;i < n-1 ;i++) {

least=A[i];loc=i;for ( j = i + 1;j < n ;j++){

if (A[j] < least)least= A[j]; loc=j;

} if(i!=loc)

swap(A[i],A[loc]);}

}

24

Example: Consider the array: 15, 10, 20, 25, 5

After Pass 1: 5, 10, 20, 25, 15

After Pass 2: 5, 10, 20, 25, 15

After Pass 3: 5, 10, 15, 25, 20

After Pass 4: 5, 10, 15, 20, 25

Time Complexity:Inner loop executes for (n-1) times when i=0, (n-2) times when i=1 and so on: Time complexity = (n-1) + (n-2) + (n-3) + …………………………. +2 +1= O(n2)The complexity of this algorithm is same as that of bubble sort, but number of swap operations is reduced greatly.

25

Properties:· Not- stable · In-place sorting (O(1) extra space)· Most time depends upon comparisons O(n2) comparisons· Not adaptive· Minimum number of swaps, so in the applications where

cost of swapping items is high selection sort is the algorithm of choice.

Space Complexity:Since no extra space besides 5 variables is needed for sorting,Space complexity = O(n)

26

Insertion SortIdea:

like sorting a hand of playing cards start with an empty left hand and the cards facing down on the table. Remove one card at a time from the table,Compare it with each of the cards already in the hand, from right to left and insert it into the correct position in the left hand. The cards held in the left hand are sorted

Suppose an array a[n] with n elements. The insertion sort works as follows:pass 1:

a[0] by itself is trivially sorted.Pass 2:

a[1] is inserted either before or after a[0] so that a[0], a[1] is sorted.Pass 3:

a[2] is inserted into its proper place in a[0],a[1] that is before a[0], between a[0] and a[1], or after a[1] so that a[0],a[1],a[2] is sorted.

.......................................pass N:

a[n-1] is inserted into its proper place in a[0],a[1],a[2],........,a[n-2] so that a[0],a[1],a[2],............,a[n-1] is sorted with n elements.

27

Example:

28

Algorithm:InsertionSort(A){

for( i = 1;i < n ;i++) {

temp = A[i] for ( j = i -1;j >=0;j--){

if(A[j]>temp )A[j+1] = A[j];

}A[j+1] = temp;

}}

30

Best case: If array elements are already sorted, inner loop executes only 1 time for i=1,2,3,… , n-1 for each. So,total time complexity = 1+1+1+ …………..+1 (n-1) times = n-1 = O(n)

Space Complexity:Since no extra space besides 3variables is needed for sorting,Space complexity = O(n)

31

Properties:· Stable Sorting

· In-place sorting (O(1) extra space)· Most time depends upon comparisons O(n2)

comparisons· Run time depends upon input (O(n) when nearly sorted

input)· O(n2) comparisons and swaps

32

• This algorithm is based on the divide and conquer paradigm. • The main idea behind this sorting is: partitioning of the elements into

two groups and sort these parts recursively. The two partitions contain values that are either greater or smaller than the key .

• It possesses very good average case complexity among all the sorting algorithms.

Quick-Sort

Steps for Quick Sort:Divide: partition the array into two non-empty sub arrays.

Conquer: two sub arrays are sorted recursively.

Combine: two sub arrays are already sorted in place so no need to combine

33

Quicksort

• To sort a[left...right]:1. if left < right:

1.1. Partition a[left...right] such that: all a[left...p-1] are less than a[p],

and all a[p+1...right] are >= a[p]1.2. Quicksort a[left...p-1]1.3. Quicksort a[p+1...right]

2. Terminate

34

Partitioning in Quicksort • A key step in the Quicksort algorithm is partitioning the

array– We choose some (any) number p in the array to use as a pivot– We partition the array into three parts:

p

numbers less than p

numbers greater than or equal to p

p

35

Partitioning in Quicksort

• Choose an array value (say, the first) to use as the pivot

• Starting from the left end, find the first element that is greater than the pivot

• Searching backward from the right end, find the first element that is less than or equal to the pivot

• Interchange (swap) these two elements• Repeat, searching from where we left off, until

done

36

Partitioning

• To partition a[left...right]:1. Set pivot = a[left], l = left + 1, r = right;2. while l < r, do

2.1. while l < right & a[l] <= pivot , set l = l + 1

2.2. while r > left & a[r] > pivot , set r = r - 1

2.3. if l < r, swap a[l] and a[r]3. Set a[left] = a[r], a[r] = pivot 4. Terminate

37

Algorithm:QuickSort(A,l,r){

if(l<r){

p = Partition(A,l,r);QuickSort(A,l,p-1);QuickSort(A,p+1,r);

}}

Properties:• Not -stable Sorting

· In-place sorting (O(log n) extra space)· Not Adaptive · O(n2) time, typically O(nlogn)

38

Partition(A,l,r){

x =l;y =r ;p = A[l];while(x<y){

while(A[x] <= p)x++;

while(A[y] >p)y--;

if(x<y)swap(A[x],A[y]);

}A[l] = A[y];A[y] = p;return y; //return position of pivot

}

39

Example: a[]={5, 3, 2, 6, 4, 1, 3, 7}

(1 3 2 3 4) 5 (6 7)

and continue this process for each sub-arrays and finally we get a sorted array.

Trace of QuickSort Algorithm

6 5 9 12 3 40 1 2 3 4 5

quickSort(arr,0,5)

quickSort(arr,0,5)

6 5 9 12 3 40 1 2 3 4 5

partition(arr,0,5)

quickSort(arr,0,5)

6 5 9 12 3 40 1 2 3 4 5

partition(arr,0,5)

pivot= ?

Partition Initialization...

quickSort(arr,0,5)

6 5 9 12 3 40 1 2 3 4 5

partition(arr,0,5)

pivot=6


quickSort(arr,0,5)

6 5 9 12 3 40 1 2 3 4 5

partition(arr,0,5)

left rightpivot=6


quickSort(arr,0,5)

6 5 9 12 3 40 1 2 3 4 5

partition(arr,0,5)

left rightpivot=6

right moves to the left untilvalue that should be to leftof pivot...

quickSort(arr,0,5)

6 5 9 12 3 40 1 2 3 4 5

partition(arr,0,5)

left rightpivot=6

quickSort(arr,0,5)

6 5 9 12 3 40 1 2 3 4 5

partition(arr,0,5)

left rightpivot=6

left moves to the right untilvalue that should be to rightof pivot...

quickSort(arr,0,5)

6 5 9 12 3 40 1 2 3 4 5

partition(arr,0,5)

left rightpivot=6

quickSort(arr,0,5)

6 5 4 12 3 90 1 2 3 4 5

partition(arr,0,5)

left rightpivot=6


quickSort(arr,0,5)

6 5 4 12 3 90 1 2 3 4 5

partition(arr,0,5)

left rightpivot=6

swap arr[left] and arr[right]

quickSort(arr,0,5)

6 5 4 3 12 90 1 2 3 4 5

partition(arr,0,5)

left

right

pivot=6

right & left CROSS!!!

quickSort(arr,0,5)

6 5 4 3 12 90 1 2 3 4 5

partition(arr,0,5)

left

right

pivot=6

right & left CROSS!!!1 - Swap pivot and arr[right]

quickSort(arr,0,5)

3 5 4 6 12 90 1 2 3 4 5

partition(arr,0,5)

left

right

pivot=6


quickSort(arr,0,5)

3 5 4 6 12 90 1 2 3 4 5

partition(arr,0,5)

left

right

pivot=6

right & left CROSS!!!1 - Swap pivot and arr[right]2 - Return new location of pivot to caller

return 3

quickSort(arr,0,5) 3 5 4 6 12 90 1 2 3 4 5

Recursive calls to quickSort()using partitioned array...

pivotposition

quickSort(arr,0,5)

3 5 4 6 12 90 1 2 3 4 5

quickSort(arr,0,3)

3 5 4 60 1 2 3

quickSort(arr,4,5)

12 94 5

quickSort(arr,0,5)

quickSort(arr,0,3)

3 5 4 60 1 2 3

quickSort(arr,4,5)

12 94 5

partition(arr,0,3)

quickSort(arr,0,3)

3 5 4 60 1 2 3

quickSort(arr,4,5)

12 94 5

partition(arr,0,3)


quickSort(arr,0,5)

quickSort(arr,0,3)

3 5 4 60 1 2 3

quickSort(arr,4,5)

12 94 5

partition(arr,0,3)


left

quickSort(arr,0,5)

quickSort(arr,0,3)

3 5 4 60 1 2 3

quickSort(arr,4,5)

12 94 5

partition(arr,0,3)


left right

quickSort(arr,0,5)


quickSort(arr,0,3)

3 5 4 60 1 2 3

quickSort(arr,4,5)

12 94 5

partition(arr,0,3)

left right

quickSort(arr,0,5)

quickSort(arr,0,3)

3 5 4 60 1 2 3

quickSort(arr,4,5)

12 94 5

partition(arr,0,3)

left

right

quickSort(arr,0,5)

quickSort(arr,0,3)

3 5 4 60 1 2 3

quickSort(arr,4,5)

12 94 5

partition(arr,0,3)

left

right

right & left CROSS!!!

quickSort(arr,0,5)

quickSort(arr,0,3)

3 5 4 60 1 2 3

quickSort(arr,4,5)

12 94 5

partition(arr,0,3)

left

right


quickSort(arr,0,5)

quickSort(arr,0,3)

3 5 4 60 1 2 3

quickSort(arr,4,5)

12 94 5

partition(arr,0,3)

left

right


right & left CROSS!!!1 - Swap pivot and arr[right]2 - Return new location of pivot to caller

return 0

quickSort(arr,0,5)

quickSort(arr,0,3)

3 5 4 60 1 2 3

quickSort(arr,4,5)

12 94 5

quickSort(arr,0,5)

Recursive calls to quickSort()using partitioned array...

quickSort(arr,0,3) quickSort(arr,4,5)

12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)

5 4 61 2 3


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)

5 4 61 2 3

Base case triggered...halting recursion.


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)

5 4 61 2 3

Base Case: Return


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)

5 4 61 2 3

partition(arr,1,3)



12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)

5 4 61 2 3

partition(arr,1,3)

Partition Initialization...left


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)

5 4 61 2 3

partition(arr,1,3)

left right



12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)

5 4 61 2 3

partition(arr,1,3)

left

right


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)

5 4 61 2 3

partition(arr,1,3)

left

right

right & left CROSS!


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)

5 4 61 2 3

partition(arr,1,3)

left

right

right & left CROSS!1- swap pivot and arr[right]


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)

4 5 61 2 3

partition(arr,1,3)

left

right

right & left CROSS!1- swap pivot and arr[right]


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)

4 5 61 2 3

partition(arr,1,3)

left

right

right & left CROSS!1- swap pivot and arr[right]2 – return new position of pivot

return 2


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)


4 5 61 2 3


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)


4 5 61 2 3

partition(arr,1,2)


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)


4 5 61 2 3


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)


63


4 51 2


12 94 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)


63


4 51 2

partition(arr,4,5)


9 124 5

quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)


63


4 51 2

partition(arr,4,5)


quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)


63


4 51 2

quickSort(arr,4,5)

9 124 5

quickSort(arr,6,5)


quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)


63


4 51 2

quickSort(arr,4,5)

9 124 5

partition(arr,4,5)

quickSort(arr,6,5)


quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)


63


4 51 2

quickSort(arr,4,5)

9 124 5

quickSort(arr,6,5)


quickSort(arr,0,5)

quickSort(arr,0,0)

30

quickSort(arr,1,3)


63


4 51 2

quickSort(arr,4,5)

9

12

4

5

quickSort(arr,4,4)

quickSort(arr,5,5)

quickSort(arr,6,5)

97

•Best Case:Divides the array into two partitions of equal size, thereforeT(n) = 2T(n/2) + O(n) , Solving this recurrence we get,T(n)=O(nlogn)

• Worst case:when one partition contains the n-1 elements and another partition contains

only one element.Therefore its recurrence relation is:

T(n) = T(n-1) + O(n), Solving this recurrence we get, T(n)=O(n2)•Average case:

Good and bad splits are randomly distributed across throughout the treeT1(n)= 2T'(n/2) + O(n) BalancedT'(n)= T(n –1) + O(n) UnbalancedSolving:B(n)= 2(B(n/2 –1) + Θ(n/2)) + Θ(n)

= 2B(n/2 –1) + Θ(n)= O(nlogn)=>T(n)=O(nlogn)

Analysis of Quick Sort

98

Merge Sort

Merge sort is divide and conquer algorithm. It is recursive algorithm having three steps. To sort an array A[l . . r]:• Divide

Divide the n-element sequence to be sorted into two sub-sequences of n/2 elements

• ConquerSort the sub-sequences recursively using merge sort. When the size of the sequences is 1 there is nothing more to do

•CombineMerge the two sorted sub-sequences into single sorted array. Keep track of smallest element in each sorted half and inset smallest of two elements into auxiliary array. Repeat this until done.

99

Algorithm: MergeSort(A, l, r){If(l<r) {

m=(l+r)/2MergeSort(A, l, m)MergeSort(A, m+1, r)Merge(A, l, m+1, r) }}

Properties:· stable Sorting

· Not In-place sorting (O( n) extra space)

· Not Adaptive · Time complexity: O(nlogn)

100

Merge(A, B, l, m, r){

x= l, y=m, k =l;While(x<m && y<r){

if(A[x] < A[y]){ B[k] = A[x]; k++; x++;}

else{ B[k] = A[y]; k++; y++;}

} while(x<m) {

B[k] = A[x];k++;x++;

} while(y<r) {

B[k] = A[y];k++; y++;

}For(i =1;i<=r;i++){

A[i] = B[i];}

}

101

Example: a[]= {4, 7, 2, 6, 1, 4, 7, 3, 5, 2, 6}

103

Time Complexity:Recurrence Relation for Merge sort:

T(n) = 1 if n=1T(n) = 2 T(n/2) + O(n) if n>1

Solving this recurrence we get T(n) = O(nlogn)

Space Complexity:It uses one extra array and some extra variables during sorting, therefore,Space Complexity = 2n + c = O(n)

104

Sorting Comparison:

105

Thank You !

By Dabal Mahara

Unit vii Sorting

Education

Transcript of Unit vii Sorting