Some Sorting Algorithms

8/11/2019 Some Sorting Algorithms

1/117

Sorting

Data Structures & Algo- Dr Ahmar Rashid 1


2/117

Introduction Sorting is the process of ordering a list of objects,

according to some linear order, such as

a1a2a3a4a5a6

Sorting can be divided into two parts i.e. Internal, and external.

Internalsorting When all the data is stored in the main memory

The random access nature of the memory can be utilized Externalsorting

When the size of data too large

The main memory cannot accommodate all the data

Virtual memory (swapping the data in/out of the memory) isthus used



3/117

Introduction

Externalsorting

The bottleneck is usually the data movement

between the secondary storage and the main

memory.

Data movement is efficient if it is moved in the

form of large blocks.

However, moving large blocks of data is efficient

only if it is physically located in contiguous

locations.



4/117

Introduction

The simplest algorithms usually take O(n2)

time to sort nobjects, and suited for sorting

short lists.

One of the most popular algorithms is Quick-

Sort takes O(nlogn)time on average.

Quick-Sort works for most commonapplications, although in worst case it can

take O(n2)time.



5/117

Introduction

There are other sorting techniques, such as

Merge-Sort and Heap-Sort that take time

O(nlogn)in worst case.

Merge-Sort however, is well suited for

external sorting.

There are other algorithms such as bucketand radix sort when the keys are integers

of known range. They take time O(n).



6/117

Introduction The sorting problem is to arrange a sequence of records so that the

values of their key fields form a non-decreasing sequence.

Given records r1, r1, . rnwith key values k1, k1, . kn, respectivelywe must produce the same records in an order ri1, ri2, . rinsuch thatthe keys are in the corresponding non-decreasing order.

key(ri1)key(ri2) key(ri3) key(ri4) key(ri5) key(ri6) ki1 ki2 ki3 ki4 ki5 ki6

The records may NOT have distinct values, and can appear in anyorder.

Different criteria to evaluate the running time, as follows:

Number of algorithm steps. Number of comparisons between the keys (for expensive comparisons).

The number of times a record is moved (for large records).



7/117

Bubble Sort One of the simplest sorting methods.

The basic idea is the weight of the record.

The records are kept in an array held vertically.

light records bubbling up to the top.

We make repeated passes over the array from bottomto top.

If two adjacent elements are out of order i.e.lighter one is below, we reverse the order.



8/117

Bubble Sort

The overall effect, is that after the first pass

the lightest record will bubble all the way

to the top.

On the second top pass, the second lowest

rises to the second position, and so on.

On second pass we need not try bubbling tothe top position, because we know that the

lightest record is already there.



9/117

Bubble Sortintn =N; // N is the size of the array;

for(int i = 0; i < N; i++){

for(intj = 1; j < n; j++) {

if(A[j] < A[j-1]) {

swap(j-1 , j);

}//end if

} //end inner for

}//end inner for

Complexity ?

O(N2)


Algorithm does not exit

until all the data is checked


10/117


for(int i = 0; i < N; i++){

intswapped = 0;

for(intj = 1; j < n; j++) {

if(A[j] < A[j-1]) {

swap(j-1 , j);

swapped = 1;

}//end if

} //end inner for

// n = n-1;

if (swapped == 0)break;

}//end inner for

Complexity ?

O(N

2

)Data Structures & Algo- Dr Ahmar Rashid 10

Algorithm exits if no swap donein previous (outer loop) step


11/117


for(int i = 0; i < N; i++){

intswapped = 0;

for(intj = 1; j < n; j++) {

if(A[j] < A[j-1]) {

swap(j-1 , j);

swapped = 1;

}//end if

} //end inner for

n = n-1;


}//end inner for

Complexity ?

O(N

2

)Data Structures & Algo- Dr Ahmar Rashid 11

Algorithm exits if no swap done

in previous (outer loop) step

No bubbling to the top position, because

the lightest record is already there.


12/117

Bubble Sort Example (First Pass)62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)i= 0

j= 1 58(0)

62

(1)

55

(2)

10

(3)

45

(4)

44

(5)

6

(6)

90

(7)

58 (0) 55 (1) 62(2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)j= 2

58 (0) 55 (1) 10(2) 62(3) 45 (4) 44 (5) 6 (6) 90 (7)j= 3

58 (0) 55 (1) 10 (2) 45(3) 62(4) 44 (5) 6 (6) 90 (7)j= 4

58 (0) 55 (1) 10 (2) 45 (3) 44(4) 62(5) 6 (6) 90 (7)j= 5

58 (0) 55 (1) 10 (2) 45 (3) 44 (4) 6(5) 62(6) 90 (7)j= 6

58 (0) 55 (1) 10 (2) 45 (3) 44 (4) 6 (5) 62(6) 90(7)j= 7

swap(A [0] , A [1])

swap(A [1] , A [2])

swap(A [2] , A [3])

swap(A [5] , A [6])

swap(A [3] , A [4])

swap(A [4] , A [5])

intn =N; // N is the size of the array;

for(int i = 0; i < N; i++){

intswapped = 0;

for(intj = 1; j < n; j++)

if(A[j] < A[j-1]) {

swap(j-1 , j); swapped = 1;

}//end if

n = n-1;

if (swapped == 0)

break; }//End outer forData Structures & Algo- Dr Ahmar Rashid 12


13/117

Bubble Sort Example (Second Pass)i= 0

j= 1j= 2

j= 3

j= 4

j= 5

j= 6

swap(A [0] , A [1])

swap(A [1] , A [2])

swap(A [2] , A [3])

swap(A [3] , A [4])

swap(A [4] , A [5])


for(int i = 0; i < N; i++){

intswapped = 0;


if(A[j] < A[j-1]) {


}//end if

n = n-1;

if (swapped == 0)

break; }//End outer for

58 (0) 55 (1) 10 (2) 45 (3) 44 (4) 6 (5) 62 (6) 90 (7)

55(0) 58(1) 10 (2) 45 (3) 44 (4) 6 (5) 62 (6) 90 (7)

55 (0) 10(1) 58(2) 45 (3) 44 (4) 6 (5) 62 (6) 90 (7)

55 (0) 10 (1) 45(2) 58(3) 44 (4) 6 (5) 62 (6) 90 (7)

55 (0) 10 (1) 45 (2) 44(3) 58(4) 6 (5) 62 (6) 90 (7)

55 (0) 10 (1) 45 (2) 44 (3) 6(4) 58(5) 62 (6) 90 (7)

55 (0) 10 (1) 45 (2) 44 (3) 6 (4) 58(5) 62(6) 90 (7)



14/117

Bubble Sort Example (Third Pass)i= 0

j= 1j= 2

j= 3

j= 4

j= 5


for(int i = 0; i < N; i++){

intswapped = 0;


if(A[j] < A[j-1]) {


}//end if

n = n-1;

if (swapped == 0)break; }//End outer for

55 (0) 10 (1) 45 (2) 44 (3) 6 (4) 58 (5) 62 (6) 90 (7)

10(0) 45(1) 44(2) 6(3) 55(4) 58(5) 62 (6) 90 (7)



15/117

Bubble Sort Example (Fourth Pass)i= 0

j= 1j= 2

j= 3

j= 4


for(int i = 0; i < N; i++){

intswapped = 0;

for(intj = 1; j < n; j++)if(A[j] < A[j-1]) {


}//end if

n = n-1;

if (swapped == 0)


10 (0) 45 (1) 44 (2) 6 (3) 55 (4) 58 (5) 62 (6) 90 (7)

10(0) 44(1) 6(2) 45(3) 55(4) 58 (5) 62 (6) 90 (7)



16/117

Bubble Sort Example (Fifth Pass)i= 0

j= 1j= 2

j= 3


for(int i = 0; i < N; i++){

intswapped = 0;


if(A[j] < A[j-1]) {


}//end if

n = n-1;

if (swapped == 0)


10(0) 6(1) 44(2) 45(3) 55 (4) 58 (5) 62 (6) 90 (7)

10 (0) 44 (1) 6 (2) 45 (3) 55 (4) 58 (5) 62 (6) 90 (7)



17/117

Bubble Sort Example (Sixth Pass)i= 0

j= 2


for(int i = 0; i < N; i++){

intswapped = 0;


if(A[j] < A[j-1]) {


}//end if

n = n-1;

if (swapped == 0)


6(0) 10(1) 44(2) 45 (3) 55 (4) 58 (5) 62 (6) 90 (7)

10 (0) 6 (1) 44 (2) 45 (3) 55 (4) 58 (5) 62 (6) 90 (7)

j= 1



18/117

Bubble Sort Example (Seventh Pass)i= 0

j= 1


for(int i = 0; i < N; i++){

intswapped = 0;


if(A[j] < A[j-1]) {


}//end if

n = n-1;

if (swapped == 0)


6(0) 10(1) 44 (2) 45 (3) 55 (4) 58 (5) 62 (6) 90 (7)

6 (0) 10 (1) 44 (2) 45 (3) 55 (4) 58 (5) 62 (6) 90 (7)



19/117


for(int i = 0; i < N; i++){

for(intj = 1; j < n; j++) {

if(A[j] < A[j-1]) {

swap(j-1 , j);

}//end if

} //end inner for

}//end inner for

Complexity ?

O(N2)


Algorithm does not exit

until all the data is checked


20/117

Bubble Sort output(1) worst case



21/117

Bubble Sort output(2) mixed data



22/117

Bubble Sort output(3) best case



23/117


for(int i = 0; i < N; i++){

intswapped = 0;

for(intj = 1; j < n; j++) {

if(A[j] < A[j-1]) {

swap(j-1 , j);

swapped = 1;

}//end if

} //end inner for

// n = n-1;


}//end inner for

Complexity ?

O(N2)


Algorithm exits if no swap donein previous (outer loop) step


24/117

Bubble Sort output (1) best case



25/117

Insertion Sort

On the ithpass we insert the ithelement

A[i]into its rightful place among

A[1],A[2],A[i-1]which were placed in

sorted order.

After this insertionA[1],A[2],A[i]are in

sorted order.



26/117

Insertion Sort

for(inti =1, i < n, i++){

temp =A[i];

for(intj = i,j>0 && A[j-1]> temp,j--)

A[j] =A[j-1]A[j] = temp;

}// end outer for

Complexity ?

O(N2)



27/117

j> 0 && A [j -1] > temp

62 > 58

A[1] = A[0]

Insertion Sort Example (First Pass)62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)

for(inti =1, i < n, i++){temp =A[i];

for(intj = i,j > 0 && A[j-1]> temp,j--)

A[j] =A[j-1]

A[j] = temp;

}// end outer for

i= 1

j= 1

62(0)

62(1)

55(2)

10(3)

45(4)

44(5)

6(6)

90(7)

temp= 58

A [1] = A [0]j= 0

58 (0) 62(1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)

A [0] = temp

= 58;


j= 0 exit

A[j] = temp (= 58)


28/117

Insertion Sort Example (Second Pass)58(0) 62(1) 55(2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)i= 2

j= 2 58(0)

62(1)

62(2)

10(3)

45(4)

44(5)

6(6)

90(7)

temp= 55

A [2] = A [1]

j= 1 58 (0) 58 (1) 62 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)



A[j] =A[j-1]

A[j] = temp;

}// end outer for

A [1] = A [0]

j= 0 55 (0) 58(1) 62 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)

A [0] = temp

= 55;


62 > 55

58 > 55


29/117

Insertion Sort Example (Third Pass)55(0) 58(1) 62 (2) 10(3) 45 (4) 44 (5) 6 (6) 90 (7)i= 3

j= 3 55(0)

58(1)

62(2)

62(3)

45(4)

44(5)

6(6)

90(7)

temp= 10

A [3] = A [2]

j= 2 55(0) 58 (1) 58(2) 62 (3) 45 (4) 44 (5) 6 (6) 90 (7)



A[j] =A[j-1]

A[j] = temp;

}// end outer for

A [2] = A [1]

j= 1 55 (0) 55 (1) 58 (2) 62 (3) 45 (4) 44 (5) 6 (6) 90 (7)

A [0] = temp= 55;

A [1] = A [0]

j= 0 10 (0) 55 (1) 58 (2) 62 (3) 45 (4) 44 (5) 6 (6) 90 (7)



30/117

Insertion Sort Example (Fourth Pass)10(0) 55(1) 58 (2) 62 (3) 45(4) 44 (5) 6 (6) 90 (7)i= 4

j= 4 10(0)

55(1)

58(2)

62(3)

62(4)

44(5)

6(6)

90(7)

temp= 45

A [4] = A [3]

j= 3 10(0) 55(1) 58(2) 58(3) 62 (4) 44 (5) 6 (6) 90 (7)



A[j] =A[j-1]

A[j] = temp;

}// end outer for

A [3] = A [2]

j= 2 10(0) 55 (1) 55(2) 58 (3) 62 (4) 44 (5) 6 (6) 90 (7)

A [1] = temp= 45;

A [2] = A [1]

j= 1 10(0) 45 (1) 55(2) 58 (3) 62 (4) 44 (5) 6 (6) 90 (7)



31/117

Insertion Sort Example (Fifth Pass)10(0) 45(1) 55 (2) 58 (3) 62 (4) 44(5) 6 (6) 90 (7)i= 5

j= 5 10(0)

45(1)

55(2)

58(3)

62(4)

62(5)

6(6)

90(7)

temp= 44

A [5] = A [4]

j= 4 10(0) 45(1) 55 (2) 58(3) 58(4) 62 (5) 6 (6) 90 (7)



A[j] =A[j-1]

A[j] = temp;

}// end outer for

A [4] = A [3]

j= 3 10(0) 45(1) 55(2) 55(3) 58 (4) 62 (5) 6 (6) 90 (7)

A [1] = temp

= 44;

A [3] = A [2]

j= 2 10(0) 45 (1) 45(2) 55 (3) 58 (4) 62 (5) 6 (6) 90 (7) A [2] = A [1]

j= 1 10(0) 44 (1) 45 (2) 55 (3) 58 (4) 62 (5) 6 (6) 90 (7)



32/117

Insertion Sort Example (Sixth Pass)10(0) 44(1) 45 (2) 55 (3) 58 (4) 62 (5) 6(6) 90 (7)i= 6

j= 6 10(0)

44(1)

45(2)

55(3)

58(4)

62(5)

62(6)

90(7)

temp= 6

A [6] = A [5]

j= 5 10(0) 44(1) 45 (2) 55 (3) 58(4) 58(5) 62(6) 90 (7)



A[j] =A[j-1]

A[j] = temp;

}// end outer for

A [5] = A [4]

j= 4 10(0) 44(1) 45 (2) 55(3) 55(4) 58 (5) 62(6) 90 (7)

A [0] = temp

= 6;

A [4] = A [3]

j= 3 10(0) 44(1) 45(2) 45(3) 55 (4) 58 (5) 62(6) 90 (7) A [3] = A [2]

j= 2 10(0) 44 (1) 45(2) 45 (3) 55 (4) 58 (5) 62(6) 90 (7)

j= 1 10 (0) 10 (1) 45 (2) 45 (3) 55 (4) 58 (5) 62(6) 90 (7)

j= 0 6 (0) 10(1) 45 (2) 45 (3) 55 (4) 58 (5) 62(6) 90 (7)

A [2] = A [1]

A [1] = A [0]



33/117

Insertion Sort Example (Sixth Pass)6(0) 10(1) 45 (2) 45 (3) 55 (4) 58 (5) 62(6) 90 (7)i= 7 temp= 90

A [j-1]> temp ?No

Quit



A[j] =A[j-1]

A[j] = temp;

}// end outer forData Structures & Algo- Dr Ahmar Rashid 33

l h


34/117

Insertion Sort Example (Sixth Pass)6(0) 10(1) 45 (2) 45 (3) 55 (4) 58 (5) 62(6) 90 (7)i= 7 temp= 90

A [j-1]> temp ?No

Quit



A[j] =A[j-1]

A[j] = temp;

}// end outer forData Structures & Algo- Dr Ahmar Rashid 34


35/117

Analysis of Insertion Sort

Because of the nested loops, each of which can take

niterations, insertion sort is O(n

2

). Furthermore, this bound is tight, because input in

reverse order can actually achieve this bound.

A precise calculation shows that the test at line 3can

be executed at most itimes for each value of i.Summing over all igives a total of

If the input is presorted, the running time isO(n)

because the test in the inner forloop always fails

immediately

The average running time also O(n2)

1

1

)(1...21 2n

i

nni



36/117

Selection Sort

Find the minimum value in the list

Swap it with the value in the first position

Repeat the steps above for the remainder

of the list (starting at the second position

and advancing each time)

http://en.wikipedia.org/wiki/Selection_sortData Structures & Algo- Dr Ahmar Rashid 36


37/117

Selection Sortfor(inti =0, i < n, i++){

min = i;

for(intj = i+1,j < n ,j++){

if ( A[j]


38/117

The Selection Sort might swap an

array element with itself--this is

harmless.

7 2 8 5 4

2 7 8 5 4

2 4 8 5 7

2 4 5 8 7

2 4 5 7 8

Selection Sort - Example


min = i;


if ( A[j]


39/117

SelectionSort Example62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)


min = i;

for(intj = i+1,j < n ,j++){if ( A[j]


40/117

SelectionSort Example - continued62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)

for(inti =0, i < n, i++){min = i;


if ( A[j]


41/117

Selection Sort vs Insertion Sort

Selection sort's advantage is that

While insertion sorttypically makes fewercomparisons than selection sort,

Insertion sort requires more writes than the

selection sortbecause the inner loop of the

insertion sort can require shifting large sectionsof the sorted portion of the array. In general, insertion sort will write to the array O(n2) times

Whereas selection sort will write/swap only O(n) times

For this reason selection sort may be preferable

in cases where writing to memory is significantlymore expensive than reading,

such as with EPROM or flash memory


Comparisons of different sorting


42/117

Comparisons of different sortingalgorithms

Bubble Sort Insertion Sort Selection Sort

(n2) comparisons (n2) comparisons (n2) comparisons

(n2) swaps (n2) writes (n) swaps

Adaptive: O(n)

running time when

nearly sorted (Best

case running time)

Adaptive: O(n)

running time when

nearly sorted (Best

case running time)

Not adaptive (n2)

running time when

nearly sorted (Best

case running time)



43/117

Merge Sort The fundamental operation in this algorithm is merging

two sorted lists. Because the lists are sorted, this can be done in one

pass through the input, if the output is put in a third list.

The basic merging algorithm takes

two input arrays: aand b, an output array: c

three counters: aptr, bptr, and cptr,

which are initially set to the beginning of their respective arrays.

The smaller of a[aptr] and b[bptr] is copied to the next

entry in c, and the appropriate counters are advanced.

When either input list is exhausted, the remainder of

the other list is copied to c.



44/117

Merge Sort : Example

1(0) 13(1) 24 (2) 26 (3) 2(0) 15(1) 27 (2) 38 (3)

aptr bptr

cptr



45/117


1(0) 13(1) 24 (2) 26 (3)

1(0)

2(0) 15(1) 27 (2) 38 (3)

aptr bptr

cptr



46/117


1(0) 13(1) 24 (2) 26 (3)

1(0) 2(1)

2(0) 15(1) 27 (2) 38 (3)

aptr bptr

cptr



47/117


1(0) 13(1) 24 (2) 26 (3)

1(0) 2(1) 13(2)

2(0) 15(1) 27 (2) 38 (3)

aptr bptr

cptr



48/117


1(0) 13(1) 24 (2) 26 (3)

1(0) 2(1) 13(2) 15(3)

2(0) 15(1) 27 (2) 38 (3)

aptr bptr

cptr



49/117


1(0) 13(1) 24 (2) 26 (3)

1(0) 2(1) 13(2) 15(3) 24 (2)

2(0) 15(1) 27 (2) 38 (3)

aptr bptr

cptr



50/117


1(0) 13(1) 24 (2) 26 (3)

1(0) 2(1) 13(2) 15(3) 24 (2) 26 (3)

2(0) 15(1) 27 (2) 38 (3)

aptr bptr

cptr



51/117


1(0) 13(1) 24 (2) 26 (3)

1(0) 2(1) 13(2) 15(3) 24 (4) 26 (5) 27 (6) 38 (7)

2(0) 15(1) 27 (2) 38 (3)

aptr bptr

cptr


Merge Sort


52/117

Merge Sort

voidmergesort( input_type a[], unsigned int n )

{input_type *tmp_array;

tmp_array = (input_type *) malloc ( (n) * sizeof (input_type) );

if( tmp_array != NULL )

{m_sort( a, tmp_array, 0, n-1 );

free( tmp_array );

}

else

cout


53/117

Merge Sort

voidm_sort( input_type a[], input_type tmp_array[ ], int left, int right )

{int center;

if( left


54/117

g pvoidmerge( input_type a[ ], input_type tmp_array[ ], int l_pos, int r_pos, int right_end )

{

int i, left_end, num_elements, tmp_pos;

left_end = r_pos - 1;tmp_pos = l_pos;

num_elements = right_end - l_pos + 1;

/* main loop */

while( ( 1_pos


55/117

g p ( )

62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)

Mergesort(a, 8)

m_sort( a, tmp_array, 0, 7 )

m_sort( a, tmp_array, 0, 3) m_sort( a, tmp_array, 4, 7)

m_sort(0, 1) m_sort(2, 3 ) m_sort(4, 5) m_sort(6, 7 )

Merge(a, tmp_array, 0, 4, 7 )

m_sort(0,0) m_sort(1,1) m_sort(2,2) m_sort(3,3) m_sort(4,4) m_sort(5,5) m_sort(6,6) m_sort(7,7)

Merge(a, tmp_array, 0, 2, 3 ) Merge(a, tmp_array, 4, 6, 7 )

Merge(0, 1, 1 ) Merge(2, 3, 3 ) Merge(4, 5, 5 ) Merge(6, 7, 7 )

62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)

62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)

62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44(5) 6 (6) 90 (7)


Merge Sort Example (Merging process)


56/117

g p ( g g p )

62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)

Mergesort(a, 8)

m_sort( a, tmp_array, 0, 7 )

m_sort( a, tmp_array, 0, 3) m_sort( a, tmp_array, 4, 7)

m_sort(0, 1) m_sort(2, 3 ) m_sort(4, 5) m_sort(6, 7 )

Merge(a, tmp_array, 0, 4, 7 )

m_sort(0,0) m_sort(1,1) m_sort(2,2) m_sort(3,3) m_sort(4,4) m_sort(5,5) m_sort(6,6) m_sort(7,7)

Merge(a, tmp_array, 0, 2, 3 ) Merge(a, tmp_array, 4, 6, 7 )

Merge(0, 1, 1 ) Merge(2, 3, 3 ) Merge(4, 5, 5 ) Merge(6, 7, 7 )

62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)

62(0)

58(1)

55(2)

10(3)

45

(4)

44

(5)

6(6)

90(7)

62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)

58 (0) 62 (1) 10 (2) 55 (3)

10 (0) 55 (1) 58 (2) 62 (3) 6 (4) 44 (5) 45 (6) 90 (7)

44 (4) 45 (5) 6 (6) 90 (7)

6 (0) 10 (1) 44 (2) 45 (3) 55 (4) 58 (5) 62 (6) 90 (7)



57/117


aptr bptr

cptr

10 (0) 55 (1) 58 (2) 62 (3) 6 (4) 44 (5) 45 (6) 90 (7)

62 (0) 58 (1) 55 (2) 10 (3) 45 (4) 44 (5) 6 (6) 90 (7)



58/117


6 (0)

aptr bptr

cptr

10 (0) 55 (1) 58 (2) 62 (3) 6 (4) 44 (5) 45 (6) 90 (7)



59/117


6(0) 10 (1)

aptr

cptr

10 (0) 55 (1) 58 (2) 62 (3) 6 (4) 44 (5) 45 (6) 90 (7)

bptr



60/117


6(0) 10 (1) 44 (2)

aptr

cptr

10 (0) 55 (1) 58 (2) 62 (3) 6 (4) 44 (5) 45 (6) 90 (7)

bptr



61/117


6(0) 10 (1) 44 (2) 45 (3)

aptr

cptr

10 (0) 55 (1) 58 (2) 62 (3) 6 (4) 44 (5) 45 (6) 90 (7)

bptr



62/117


6(0) 10 (1) 44 (2) 45 (3) 55 (4)

aptr

cptr

10 (0) 55 (1) 58 (2) 62 (3) 6 (4) 44 (5) 45 (6) 90 (7)

bptr


l


63/117


6(0) 10 (1) 44 (2) 45 (3) 55 (4) 58 (5)

aptr

cptr

10 (0) 55 (1) 58 (2) 62 (3) 6 (4) 44 (5) 45 (6) 90 (7)

bptr



64/117

M S E l


65/117


6(0) 10 (1) 44 (2) 45 (3) 55 (4) 58 (5) 62 (6) 90 (7)

aptr

cptr

10 (0) 55 (1) 58 (2) 62 (3) 6 (4) 44 (5) 45 (6) 90 (7)

bptr


Q i k S


66/117

Quick Sort

As its name implies, quicksortis the fastest

known sorting algorithm in practice. It is very fast, mainly due to a very tight and highly

optimized inner loop

Its average running time is O(n log n)

It has O(n2)worst-case performance, but this can be made exponentially unlikely with a little

effort

The quicksort algorithm is simple to understand

and prove correct Like mergesort, quicksort is a divide-and-conquer

recursive algorithm


Q i k S t


67/117

Quick Sort

The basic algorithm to sort an array Sconsists

of the following four easy steps: If the number of elements in Sis 0or 1, then return

2. Pick any element vin S. This is called the pivot.

3. PartitionS- {v}(the remaining elements in S)

into two disjoint groups:S1= {x S- {v}| x v}andS2= {x S- {v}| xv}

4.Return{quicksort(S1)followed byvfollowed by

quicksort(S2)}


Q i k S t


68/117

Quick Sort

Since the partition step ambiguously

describes what to do with elements equal tothe pivot, this becomes a design decision.

Part of a good implementation is handling this

case as efficiently as possible.

Intuitively, we would hope that

about half the keys that are equal to the pivot go

into S1

while the other half into S2, much as we like binary

search trees to be balanced.


Quick Sort


69/117

Quick Sort Selecting

the Pivot


Quick Sort Selecting the Pivot


70/117

Quick Sort Selecting the Pivot1- The popular, uninformed choice:

Use the first element as the pivot

This is acceptable if the input is random

But, if the input is presorted or in reverse order, then the pivotprovides a poor partition, because virtually all the elements gointo S1or S2

It happens consistently throughout the recursive calls

Quicksort will take quadratic time to do essentiallynothing at all, which is quite embarrassing

2- A Safe Maneuver

A safe course is merely to choose the pivot randomly

This strategy is generally perfectly safe, unless the randomnumber generator has a flaw

However, random number generation is generally anexpensive commodity and does not reduce the averagerunning time of the rest of the algorithm at all


Quick Sort Selecting the Pivot


71/117

Quick Sort Selecting the Pivot The best choice of pivot would be the median of the

file.

The median of a group of nnumbers is the (n/2)-thlargest number

Unfortunately, this is hard to calculate and would slowdown quicksort considerably

A good estimate can be obtained by picking threeelements randomly and using the median of thesethree as pivot.

The randomness turns out not to help much

So, the common course is to use as pivot the

median of the left, right and center elements

A = {8, 1, 4, 9, 6, 3, 5, 2, 7, 0} pivot: v = A[left+ right)/2]

= A [(0+9)/2] = A[4] = 6


QuickSort :Partitioning strategy


72/117

QuickSort :Partitioning strategyExample

8(0) 1 (1) 4 (2) 9 (3) 6 (4) 3 (5) 5 (6) 2 (7) 7 (8) 0 (9)

i The basic algorithm

While iis to the left ofj,

we moveiright, skipping over elements smaller than the pivot We movejleft, skipping over elements larger than the pivot

When iandjhave stopped,

iis pointing at a large element, and

jis pointing at a small element

If iis to the left ofj, those elements are swapped

The effect is to push a large element to the rightand a smallelement to the left.

In the example above, iwould not move andjwould slide over

one place

pivot j




73/117

QuickSort :Partitioning strategyStep 1

8(0) 1 (1) 4 (2) 9 (3) 6 (4) 3 (5) 5 (6) 2 (7) 7 (8) 0 (9)

i j pivot




74/117


8(0) 1 (1) 4 (2) 9 (3) 0(4) 3 (5) 5 (6) 2 (7) 7 (8) 6(9)

i j

Start by swapping thepivotwith the right, startingjat right-1A[i] = 8 > pivot

Stop iright over here

pivot




75/117


8(0) 1 (1) 4 (2) 9 (3) 0 (4) 3 (5) 5 (6) 2 (7) 7 (8) 6 (9)

i j

A[j] = 7 > pivot Move Left

A[j] = 2 < pivot

Stopjright over here

SwapA[i] andA[j]

pivotj




76/117


2 (0) 1 (1) 4 (2) 9 (3) 0 (4) 3 (5) 5 (6) 8(7) 7 (8) 6 (9)

i

A[j] = 7 > pivot Move Right

A[j] = 2 < pivot


SwapA[i] andA[j]

pivotj




77/117


2(0) 1 (1) 4 (2) 9 (3) 0 (4) 3 (5) 5 (6) 8 (7) 7 (8) 6 (9)

i

A[i] = 2 < pivot Move Right

A[i] = 1 < pivot

Move Right

A[i] = 4 < pivot

Move Right

A[i] = 9 > pivot


pivotji i i


A[j] = 5 < pivot


j




78/117


2(0) 1 (1) 4 (2) 5(3) 0 (4) 3 (5) 9(6) 8 (7) 7 (8) 6 (9)

pivoti j


A[i] = 1 < pivot

Move Right

A[i] = 4 < pivot

Move Right

A[i] = 9 > pivot



A[j] = 5 < pivot


SwapA[i] andA[j]



79/117

Qu c So t : a t t o g st ategyStep 3

2(0) 1 (1) 4 (2) 5 (3) 0 (4) 3 (5) 9 (6) 8 (7) 7 (8) 6 (9)


A[i] = 0 < pivot

Move Right

A[i] = 3 < pivot

Move Right

A[i] = 9 > pivot


pivoti ji i i




80/117

Q g gyStep 3

2(0) 1 (1) 4 (2) 5 (3) 0 (4) 3 (5) 9 (6) 8 (7) 7 (8) 6 (9)

pivot


A[j] = 3 < pivot


iandjhave crossed

So no swap forA[i] andA[j]

InsteadSwapA[i] andA[pivot]

jj i



A[i] = 0 < pivot

Move Right

A[i] = 3 < pivot

Move Right

A[i] = 9 > pivot




81/117

Q g gyStep 3

2(0) 1 (1) 4 (2) 5 (3) 0 (4) 3 (5) 6(6) 8 (7) 7 (8) 9(9)

pivotj i



A[i] = 0 < pivot

Move Right

A[i] = 3 < pivot

Move Right

A[i] = 9 > pivot



A[j] = 3 < pivot


iandjhave crossed

So no swap forA[i] andA[j]

InsteadSwapA[i] andA[pivot]


82/117

QuickSort : Recursive calls

2(0) 1 (1) 4 (2) 5 (3) 0 (4) 3 (5) 6(6) 8 (7) 7 (8) 9 (9)

i

2(0) 1 (1) 4 (2) 5 (3) 0 (4) 3 (5) 6(6) 8 (7) 7 (8) 9 (9)

i + 1i -10 N -1



83/117

QuickSort : Left Recursive call

2(0) 1 (1) 4 (2) 5 (3) 0 (4) 3 (5)

i pivot j



84/117



85/117

QStep 1: Movements

2(0) 1 (1) 3 (2) 5 (3) 0 (4) 4 (5)

i pivotj




86/117

Step 1: Swap

2(0) 1 (1) 3 (2) 5 (3) 0 (4) 4 (5)

i pivotj



87/117



88/117

Step 2: Movements

2(0) 1 (1) 3 (2) 0 (3) 5 (4) 4 (5)

i pivotj




89/117

Step 2: Swap with the pivot

2(0) 1 (1) 3 (2) 0 (3) 5 (4) 4 (5)

i pivot




90/117

Step 2: Swap with the pivot

2(0) 1 (1) 3 (2) 0 (3) 4(4) 5(5)

i pivot


Q i kS R i C ll


91/117

QuickSort :Recursive Calls

2(0) 1 (1) 3 (2) 0 (3) 4(4) 5(5)

i

2(0) 1 (1) 3 (2) 0 (3) 4(4) 5(5)

i -10 i + 1 Right


QuickSort


92/117

voidquick_sort( input_type a[ ], unsigned int n )

{q_sort( a, 0, n-1 );

}


QuickSort: Core Function


93/117

voidq_sort( input_type a[], int left, int right )

{

inti, j; intpivot;

if( left + CUTOFF


94/117

/* Return median of left, center, and right. */

/* Order these and hide pivot */

intmedian3( input_type a[], int left, int right )

{

intcenter;

center = (left + right) / 2;

if ( a[left] >a[center] )

swap( &a[left], &a[center] );

if ( a[left] >a[right] )

swap( &a[left], &a[right] );

if ( a[center] >a[right] )

swap( &a[center], &a[right] );/* a[left]


95/117

QuickSort: Medians


96/117

/* Return median of left, center, and right. */

/* Order these and hide pivot */

intmedian3( input_type a[], int left, int right )

{

intcenter;

center = (left + right) / 2;

if ( a[left] >a[center] )

swap( &a[left], &a[center] );

if ( a[left] >a[right] )

swap( &a[left], &a[right] );

if ( a[center] >a[right] )

swap( &a[center], &a[right] );/* a[left] 0

8(9)

8 > 6

6 (4)

/* hide pivot */

6 (4) 7 (8)

pivot

/* return pivot */

QuickSort: Core Function


97/117


{

inti, j; intpivot;

if( left + CUTOFF


98/117


{

inti, j; intpivot;

if( left + CUTOFF


99/117

Counting Sort


100/117

g Suppose that there are Nintegers to be sorted,

A[1N], such that the range of the integers is

from1 toM

Start by defining an array B[1M]and initializing

all its elements to 0. ( O(M) )

Scan throughA[i], for i = 1,,N,and copy each

A[i] into B[ A[i] ].( ON) )

Scan through B[i], for i = 1,,M, and retrieve all

the non-zero elements of B[i].( O(M) )

Complexity: O(M + N)

If M ~ NComplexity: O(N)

If M ~ N2Complexity: O(N2)

Example

Sort 5 7 9 2 4 6 1

Counting Sort


101/117

5(0) 7 (1) 9 (2) 2 (3) 4 (4) 6 (5) 1 (6)A

B(0) (1) (2) (3) (4)

5(5) (6) (7) (8) (9)

(0) (1) (2) (3) (4) 5 (5) (6) 7(7) (8) (9)

(0) (1) (2) (3) (4) 5 (5) (6) 7 (7) (8) 9(9)

(0) (1) 2(2) (3) (4) 5 (5) (6) 7 (7) (8) 9 (9)

(0) (1) 2 (2) (3) 4(4) 5 (5) (6) 7 (7) (8) 9 (9)

(0) (1) 2 (2) (3) 4 (4) 5 (5) 6(6) 7 (7) (8) 9 (9)

(0) 1(1) 2 (2) (3) 4 (4) 5 (5) 6 (6) 7 (7) (8) 9 (9)

B[ A[0] ] = A[0]

B[ 5 ] = 5

B[ A[i] ] = A[i]

B[ 7 ] = 7

B[ 9 ] = 9

B[ 2 ] = 2

B[ 4 ] = 4

B[ 6 ] = 6

B[ 1 ] = 1

Final Output: 1 2 4 5 6 7 9

Counting Sort


102/117

How to cope up with duplicates?B becomes a an array of pointers, (rather than

an array of integers). Each element has a

head pointer (points to the head of a linked list)

tail pointer (points to the tail of the linked list)A[j] is added at the tail of the list B[ A[j]]

Finally, the array B is sequentially traversed

and each nonempty list retrieved.

Complexity: O(M + N)

Counting Sort

Counting Sort Duplicate keys


103/117

5(0) 7 (1) 9 (2) 2 (3) 4 (4) 6 (5) 7 (6) 1 (7) 2 (8) 7 (9)A

B

Final Output: 1 2 2 4 5 6 7 7 7 9

(0)

Null(1)

head(2)

head(3)

Null(4)

head(5)

head(6)

head(7)

head(8)

Null(9)

head

1 2 4 5 6 7 9

2 7

7

Radix Sort


104/117

Counting sort becomes expensive if Mis largecompared to N

Radix sort may be built upon counting sort

Basic Idea: every integer can be represented

by at most k digits

d1d2dkwherediare digits in base r

most significant digit

least significant digit

Radix Sort


105/117

Algorithm

Take the least significant digit (or group of bits) of eachkey.

Group the keys based on that digit, but otherwise keep

the original order of keys

Repeat the grouping process with each more significantdigit.

The sort in step 2 is usually done using bucket

sortor counting sort, which are efficient in this case

since there are usually only a small number of digits.

Radix Sort


106/117

Example

Sort: 170, 45, 275, 190, 802, 24, 2, 66

First pass

(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

170 802 24 45 66

275190 2

First pass

Radix Sort


107/117

First pass(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

Second pass(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

170 802 24 45 66

275190 2

Second passRadix Sort


108/117

Second pass

(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

Third pass

(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

170802

24

45 66

275

1902

Final Output: 2 24 45 66 170 190 275 802

Al ith R di S t(A N d)

Radix Sort 170, 45, 275, 190, 802, 24, 2, 66(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)


109/117

Algorithm RadixSort(A, N, d)

for p = 0 to 9

Q[p] =NullD = 1

for k = 1 to d

D = D * 10

for i = 0 to N

t = (A[i] mod D) div (D/10)

enqueue(A[i], Q[t])

j = 0

for p = 0 to N

do while Q[p] is not empty

A[j] = dequeue(Q[p])

j = j +1

D = 10

i = 0t = (A[0] mod 10) div (1)

= 170mod 10= 0

enqueue(170, Q[0])

(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

170

Al ith R di S t(A N d)

Radix Sort 170, 45, 275, 190, 802, 24, 2, 66(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)


110/117


for p = 0 to 9

Q[p] =NullD = 1

for k = 1 to d

D = D * 10

for i = 0 to Nt = (A[i] mod D) div (D/10)

enqueue(A[i], Q[t])

j = 0

for p = 0 to N



j = j +1

D = 10

i = 1t = (A[1]mod 10) div (1)

= 45mod 10= 5

enqueue(45, Q[5])

(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

170 45


111/117

Algorithm RadixSort(A N d)

Radix Sort 170, 45, 275, 190, 802, 24, 2, 66(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)


112/117


for p = 0 to 9

Q[p] =NullD = 1

for k = 1 to d

D = D * 10


enqueue(A[i], Q[t])

j = 0

for p = 0 to N



j = j +1

D = 10

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

170 802 24 45 66

275190 2

A = { 170, 190, 802, 2, 24, 45, 275, 66}


Radix Sort(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

A = { 170, 190, 802, 2, 24, 45, 275, 66}


113/117


for p = 0 to 9

Q[p] =NullD = 1

for k = 1 to d

D = D * 10


enqueue(A[i], Q[t])

j = 0

for p = 0 to N



j = j +1

D = 100

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

170

i = 0t = (A[0]mod100)div(10)

= (170mod100)div(10)=70/10=7

enqueue(170, Q[7])


Radix Sort(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

A = { 170, 190, 802, 2, 24, 45, 275, 66}


114/117


for p = 0 to 9

Q[p] =NullD = 1

for k = 1 to d

D = D * 10


enqueue(A[i], Q[t])

j = 0

for p = 0 to N



j = j +1

D = 100

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

190

i = 1t = (A[1]mod100)div(10)

= (190mod100)div(10)=90/10=9

enqueue(190, Q[9])

170


Radix Sort(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

A = { 170, 190, 802, 2, 24, 45, 275, 66}


115/117


for p = 0 to 9

Q[p] =NullD = 1

for k = 1 to d

D = D * 10


enqueue(A[i], Q[t])

j = 0

for p = 0 to N



j = j +1

D = 100

i = 2, 3, 4, 5, 6, 7, 8, 9

170

802

24

45 66 275 190

2

A = {802, 2, 24, 45, 66, 275, 170, 190}


Radix Sort(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

A = {802, 2, 24, 45, 66, 275, 170, 190}


116/117


for p = 0 to 9

Q[p] =NullD = 1

for k = 1 to d

D = D * 10


enqueue(A[i], Q[t])

j = 0

for p = 0 to N



j = j +1

D = 1000i = 0

t = (A[0]mod1000)div(1000)

= (802mod1000)div(1000)=802/100=8

enqueue(802, Q[8])

802


Radix Sort

(0) (1) (2) (3) (4) (5) (6) (7) (8) (9)

A = {802, 2, 24, 45, 66, 275, 170, 190}


117/117


for p = 0 to 9

Q[p] =NullD = 1

for k = 1 to d

D = D * 10


enqueue(A[i], Q[t])

j = 0

for p = 0 to Ndo while Q[p] is not empty


D = 1000

802170

24

45

66

275

190

2

i = 1, 2, 3, 4, 5, 6, 7, 8, 9

Final Output

Some Sorting Algorithms

Documents

Transcript of Some Sorting Algorithms