04.09.09 IT 60101: Lecture #17 1

Foundation of Computing Systems

Lecture 17

Sorting Algorithms II

04.09.09 IT 60101: Lecture #17 2

Sorting by Exchange

• Interchange (exchange) pairs of elements that are out

of order until no more such pair exists. • Bubble sort

• Shell sort

• Quick sort

04.09.09 IT 60101: Lecture #17 3

Sorting by Exchange: Bubble Sort

1

2

3

n

i

(i-1) elem ents are sortedand stored here

i-th largest elem entssinks and placed here

i-th outer loop

Sm aller elem entsbubble up in inner loop

1

2

3

n

n-1

04.09.09 IT 60101: Lecture #17 4

Bubble Sort: Illustration

55

33

22

99

77

1

2

3

44

66

88

4

5

6

7

8

55

22

88

33

77

1

2

3

99

44

66

4

5

6

7

8

55

77

66

22

33

1

2

3

99

88

44

4

5

6

7

8

33

66

44

55

22

1

2

3

99

88

77

4

5

6

7

8

22

44

66

55

33

1

2

3

99

88

77

4

5

6

7

8

22

55

66

44

33

1

2

3

99

88

77

4

5

6

7

8

22

55

66

44

33

1

2

3

99

88

77

4

5

6

7

8

22

55

66

44

33

1

2

3

99

88

77

4

5

6

7

8

22

55

66

44

33

1

2

3

99

88

77

4

5

6

7

8

Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 Pass 6 Pass 7Input list Output list

04.09.09 IT 60101: Lecture #17 5

Complexity Analysis of Bubble Sort

• Case 1: The input list is already in sorted order

• Number of comparisons

• Number of movements

2

1)(

nnnC

M (n) = 0

04.09.09 IT 60101: Lecture #17 6

Complexity Analysis of Bubble Sort

• Case 2: The input list is sorted but in reverse order

• Number of comparisons

• Number of movements

2

1)(

nnnC

2

1)(

nnnM

04.09.09 IT 60101: Lecture #17 7

Complexity Analysis of Bubble Sort

• Case 3: Elements in the input list are in random order

Number of comparisons

Number of movements

2

1)(

nnnC

Let pj be the probability that the largest element is in the unsorted part is in j-th (1≤j≤n-i+1) location.

The average number of swaps in the i-th pass is

j

in

j

pjin

1

1

1

04.09.09 IT 60101: Lecture #17 8

Complexity Analysis of Bubble Sort

• Case 3: Elements in the input list are in random order

Number of movements

121 inppp1

1

in

jinin

in

j

11

11

1

The average number of swaps in the i-th pass is

2

in

The average number of movements

1

1 2)(

n

i

innM

4

)1(

nn

04.09.09 IT 60101: Lecture #17 9

Bubble Sort: Summary

2

)1()(

nnnC 0)( nM 0)( nS

2

)1()(

nnnC

2

)1()(

nnnM

0)( nS2

)1()(

nnnC

4

)1()(

nnnM

0)( nS

Case Comparisons Movement Memory Remark

Case 1

Input list is in sorted order

Case 2

Input list is sorted in reversed order

Case 3

Input list is in random order

2

)1()(

nncnT

)()( 2nnT

)1()( ncnnT

)()( 2nnT )1(4

3)( nnnT

)()( 2nnT

Run time, T(n) Complexity Remark

Best case

Worst case

Average case

04.09.09 IT 60101: Lecture #17 10

Bubble Sort: Refinements

• Funnel sort

• Cocktail sort

96

54

63

41

25

1

2

3

16

78

39

4

5

6

7

8

25

63

39

54

41

1

2

3

96

16

78

4

5

6

7

8

25

39

63

54

41

1

2

3

96

78

16

4

5

6

7

8

25

54

16

39

41

1

2

3

96

78

63

4

5

6

7

8

25

16

54

41

39

1

2

3

96

78

63

4

5

6

7

8

25

41

54

16

39

1

2

3

96

78

63

4

5

6

7

8

25

41

54

39

16

1

2

3

96

78

63

4

5

6

7

8

16

41

54

39

25

1

2

3

96

78

63

4

5

6

7

8

22

55

66

44

33

1

2

3

99

88

77

4

5

6

7

8

Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 Pass 6 Pass 7Input list Output list

04.09.09 IT 60101: Lecture #17 11

Cocktail Sort

99

22

66

44

55

1

2

3

11

33

88

4

5

6

7

8

55

66

88

22

44

1

2

3

99

11

33

4

5

6

7

8

11

22

66

44

55

1

2

3

99

33

88

4

5

6

7

8

11

55

66

22

44

1

2

3

99

88

33

4

5

6

7

8

11

33

55

44

22

1

2

3

99

88

66

4

5

6

7

8

11

44

55

33

22

1

2

3

99

88

66

4

5

6

7

8

11

44

55

33

22

1

2

3

99

88

66

4

5

6

7

8

Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 Pass 6Input lis t

04.09.09 IT 60101: Lecture #17 12

Sorting by Exchange: Shell Sort

• Sorting methods based on comparison

– Comparisons and hence movements of data take place between adjacent entries only

– This leads to a number of redundant comparisons and data movements

– A mechanism should be followed with which the comparisons can take in long leaps instead of short

• Donald L. Shell (1959)

– Use increments: 121 ,...,,, hhhh ttt

04.09.09 IT 60101: Lecture #17 13

Shell Sort: Illustration

21 5645247332915938671685944372 60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1

7

6

5

4

3

2

P ass 1 w ith h = 7

21 5667248594916038451673324359 72

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1

5

4

3

2

P ass 2 w ith h = 5

This

is th

e lis

t afte

r pas

s 1

04.09.09 IT 60101: Lecture #17 14

Shell Sort: Illustration

21 5667248594916038451673324359 72

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1

5

4

3

2

P ass 2 w ith h = 5

16 9167438572736038592156322445 94

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1

3

2

This

is th

e lis

t afte

r pas

s 2

04.09.09 IT 60101: Lecture #17 15

Shell Sort: Illustration

16 9167438572736038592156322445 94

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1

3

2

P ass 3 w ith h = 3

P ass 4 w ith h = 1

This

is th

e lis

t afte

r pas

s 3

16 9172738567596056432445322138 94

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1

16 9185737267605956454338322421 94

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

This

is th

e lis

t afte

rpa

ss 4

O utpu t lis t

04.09.09 IT 60101: Lecture #17 16

Issues in Shell Sort

• Algorithm to be used to sort subsequences in shell sort

– Straight insertion sort

– Shell sort is better than the insertion sort

• Lower number of passes than n number of passes in insertion sort

• Deciding the values of increments

• Several choices have been made

04.09.09 IT 60101: Lecture #17 17

Increments in Shell Sort

1: Only two increments h and 1 • h is approximately =

• Time complexity (average) =

2: ht = 2t -1 for 1≤ t ≤ log2n, where n is the size of the input list

• Increments: 2t -1, …, 15, 7, 3, 1

• Time complexity : (worst) (average)

3732.1 n

O(n5/3)

)( 2/3n )(~ 4/5n

04.09.09 IT 60101: Lecture #17 18

Increments in Shell Sort

3: ht = (3t -1)/2 for 1≤ t ≤ l Where l is chosen as the smallest integer such that , n being the size of the input list

• Worst case time complexity is O(n3/2 )

nhl 13

Many more……….

04.09.09 IT 60101: Lecture #17 19

Complexity Analysis of Shell Sort

rncnC .)(

21 r

0)( nM

0)( nSrncnC .)( 2

)1()(

nnnM

0)( nSrncnC .)( 4

)1()(

nnnM

0)( nS

Case Comparisons Movement Memory Remark

Case 1Input list is in sorted order

Case 2Input list is sorted in reverse order

Case 3Input list is in random order

rncnT .)( )()( rnnT

2

)1(.)(

nnncnT r

)()( 2nnT 4

)1(.)(

nnncnT r

21 r

)()( 2nnT

Run time, T(n) Complexity Remark

Best case

Worst case

Average case

04.09.09 IT 60101: Lecture #17 20

Sorting by Exchange: Quick Sort

• Divide-and-Conquer

Problem

Solution

P2P1 P3 Pn

D ivide

C onquer

C om b ine

04.09.09 IT 60101: Lecture #17 21

Divide-and-Conquer in Quick Sort1 2 n

Partition

1 2 n

R ight sub lis t

pp-1 p+1

Left sub lis t

1 2 np-1 p+1

Sort le ft sub lis t recurs ivley Sort right sub lis t recurs ivley

R ecurs ion R ecurs ion

04.09.09 IT 60101: Lecture #17 22

Partition Method in Quick Sort

l r

loc P ivot e lem ent

l r

loc P ivot e lem ent

Scan until A [loc] < A [r] and loc < r r

l

loc

In terchange if A [loc] >= A [r] and current scan is s topped r

Pivot e lem ent

04.09.09 IT 60101: Lecture #17 23

Partition Method in Quick Sort

l

locP ivot e lem ent

Scan until A [loc] > A [r] and loc < l r

l

loc

In terchange if A [loc] =< A [l and current scan is s topped

r

l

Pivot e lem ent

l

locP ivot e lem ent

Scan until A [loc] > A [r] and loc < l r

l

loc

In terchange if A [loc] =< A [l and current scan is s topped

r

l

Pivot e lem ent

04.09.09 IT 60101: Lecture #17 24

Partition Method: Illustration

55 3377661144992288

loc

left right

scan from right to left

(a) A t the begining of the partition

33 5577661144992288

loc

left right

scan from left to right

(b) A fter the scaning from right to left while the pivot isat left. Next, it begins the scanning from left to right

04.09.09 IT 60101: Lecture #17 25

Partition Method: Illustration

33 5577661144992288

loc

left right

scan from left to right

(b) A fter the scaning from right to left while the pivot isat left. Next, it begins the scanning from left to right

33 8877661144992255

loc

left right

scan from right to left

(c) A fter the scaning from left to right while the pivot isat right. Next, it begins the scanning from right to left

04.09.09 IT 60101: Lecture #17 26

Partition Method: Illustration

33 8877661144992255

loc

left right

scan from right to left

(c) A fter the scaning from left to right while the pivot isat right. Next, it begins the scanning from right to left

33 8877665544992211

loc

left right

scan from left to right

(d) A fter the scaning from right to left while the pivot isat left. Next, it begins the scanning from left to right

04.09.09 IT 60101: Lecture #17 27

Partition Method: Illustration

33 8877665544992211

loc

left right

scan from left to right

(d) A fter the scaning from right to left while the pivot isat left. Next, it begins the scanning from left to right

33 8877669944552211

loc

left right

scan from right to left

(e) A fter the scaning from left to right while the pivot isat right. Next, it begins the scanning from right to left

04.09.09 IT 60101: Lecture #17 28

Partition Method: Illustration

33 8877669944552211

loc

left right

scan from right to left

(e) A fter the scaning from left to right while the pivot isat right. Next, it begins the scanning from right to left

33 8877669955442211

loc

leftright

(f) A fter the scaning from right to left while the pivot isat left. left < right condition violates. Partition is done.

Left sub list. A ll elem ents areless than the pivot elem ent, 55

Right sub list. A ll elem ents aregreater than the pivot elem ent, 55

04.09.09 IT 60101: Lecture #17 29

Quick Sort: Illustration55 3377661144992288

55

44221133 77 886699

33 99

4422 11

22

66 7788

88

11 77 66

77

66

04.09.09 IT 60101: Lecture #17 30

Complexity Analysis of Quick Sort

• Memory requirementSize of the stack =

• Number of comparisons Let, T(n) represents total time to sort n elements and P(n) represents the time for perform a partition of a list of n elements

1log)( 2 nnS

T(n) = P(n) +T(nl) +T(nr) with T(1) = T(0) = 0

where, nl = number of elements in the left sub list nr = number of elements in the right sub list and 0 ≤ nl, nr < n

04.09.09 IT 60101: Lecture #17 31

Complexity Analysis of Quick Sort

• Case 1: Elements in the list are in ascending order Number of comparisons

Number of movements

C(n) = n-1 + C(n-1), with C(1) = C(0) = 0

12)3()2()1()( nnnnC

2

)1(

nn

M(n) = 0

04.09.09 IT 60101: Lecture #17 32

Complexity Analysis of Quick Sort

• Case 2: Elements in the list are in reverse order Number of comparisons

Number of movements

C(n) = n-1 + C(n-1), with C(1) = C(0) = 0

12)3()2()1()( nnnnC

2

)1(

nn

even is if ,

2

odd is if ,2

1

)(n

n

nn

nM

04.09.09 IT 60101: Lecture #17 33

Complexity Analysis of Quick Sort

• Case 3: Elements in the list are in random order i-th location

(i-1) (n-i)

)1()1(1

)1()(1

1

nCiCn

nnCn

i

Number of comparisons

with C(1) = C(0) = 0

1

1

)(2

)1()(n

i

iCn

nnC

nnnnC e 4577.0log)1(2)(

04.09.09 IT 60101: Lecture #17 34

Complexity Analysis of Quick Sort

• Case 3: Elements in the list are in random order

Number of Movements

n

i

inMiMin

nM1

)()1(11

)(

1

1

)(2

2

1)(

n

i

iMn

nnM

nnnnM e 4577.0log)1(2)(

04.09.09 IT 60101: Lecture #17 35

Complexity Analysis: Summary

2

)1()(

nnnC 0)( nM

1)( nS2

)1()(

nnnC

2)(

nnM

1)( nS

nnnnC e 4577.0log)1(2)( nnnnM e 4577.0log)1(2)( 1log)( 2 nnS

Case Comparisons Movement Memory Remark

Case 1Input list is in sorted order

Case 2Input list is sorted in reversed order

Case 3Input list is in random order

2

)1()(

nncnT

)()( 2nnT

22

)1()(

nnncnT )()( 2nnT

cnnncnT e 8577.0log)1(4)(

1log2)( 2 nnncnT nnnT 2log)(

Run time, T(n) Complexity Remark

Worst case

Worst case

Best/average case

04.09.09 IT 60101: Lecture #17 36

Variations in Quick Sort

• Randomized quick sort

• Random sampling quick sort

• Singleton’s quick sort

• Multi-partition quick sort

• Leftist quick sort

• Lomuto’s quick sort

04.09.09 IT 60101: Lecture #17 37

Lomuto’s quick sort

right part

l r

w in d ow

p tr

(a ) In itia l s tep

U n exam in ed

right part

l r

w in d ow

p tr

(b ) In term ed ia te s tep

elem en t < p ivo t

p ivo t e lem en t

p ivo t e lem en t

m iddle partleft part

elem en t > = p ivo t u n exam in ed

right part

l r

w in d ow

loc

(c ) F in a l s tep

elem en t < p ivo t

p ivo t e lem en t

left part

elem en t > = p ivo t

04.09.09 IT 60101: Lecture #17 38

Summary: Sorting by Exchange

)()( 2nnT

)()( 2nnT )()( 2nnT

)()( 3/5nnT )(~ 4/5n

)()( 2nnT

)()( 3/5nnT )(~ 4/5n

)log()( 2 nnnT

)()( 2nnT

)log()( 2 nnnT

Algorithm Best case Worst case Average case

Bubble sort

Shell sort

Quick sort

04.09.09 IT 60101: Lecture #17 39

Summary: Sorting by Exchange

Ascending order

Input Size

Tim

e (

ms)

0

2040

60

80100

120

140

160180

200

Bubb le sort

Q u ick sort

She ll sort

asc10 asc100 asc500 asc1K asc5K asc10K

04.09.09 IT 60101: Lecture #17 40

Summary: Sorting by Exchange

0

100

200

300

400

500

600

700

Input Size

Tim

e (m

s)

Bubb le sort

Q u ick sort

She ll sort

Descending order

dsc10 dsc100 dsc500 dsc1K dsc5K dsc10K

04.09.09 IT 60101: Lecture #17 41

Summary: Sorting by Exchange

0

100

200

300

400

500

600

700Random order

Input Size

Tim

e (m

s)

B ubb le sort

Q u ick sort

S he ll so rt

ran10 ran100 ran500 ran1K ran5K ran10K