Unit 1. Sorting and Divide and Conquer

Lecture 1 Introduction to Algorithm and Sorting

What is an Algorithm?

• An algorithm is a computational procedure that takes some value, or a set of values, as input and produces some values, or a set of values, as output.

• So, it is a sequence of computational steps that transform the input into the output.

• Correction: sequence -> combination

Sequential and Parallel

• In sequential computation, an algorithm is a sequence of computational steps.

• However, it is not true in parallel computation.

• In this course, we study only algorithms in sequential computation.

Algorithms for Sorting

.'''such that sequence

input of }',...,','{n permutatioA :Output

}.,...,,{ numbers of sequenceA :Input

21

21

21

n

n

n

aaa

aaa

aaan

Insertion Sort, Merge Sort

6. 5, 4, 3, 2, 1, :Output

3. 1, 6, 4, 2, 5, :Input

e.g.,

Efficiency

• Running time from receiving the input to producing the output.

)log(

)( 2

nnO

nOInsert Sort

Merge Sort

Running time

.for )()(0

such that

0 and 0 constantsexist there

means ))(()(

0

0

nnncgnf

nc

ngOnf

function!

theofgrowth theisabout carereally what weTherefore,

.log

large,ly sufficient is When

bigger? is who,log and

22

1

22

1

nncnc

n

nncnc

keyiA

ii

iAiA

keyiAi

ji

jAkey

Alengthj

]1[

1

][]1[ do

][ and 0 while

1

][ do

][ to2for

Insertion SortAarray input

for.-end

;]1[

while;-end

;1

];[]1[

begin do

][ and 0 while

;1

];[

begin do

][ to2for

keyiA

ii

iAiA

keyiAi

ji

jAkey

Alengthj

3 1, ,6 5, 4, 2,

3 1, 6, 5, ,4 2,

3 1, 6, ,4 5, 2,

3 1, 6, 4, 5, ,2

3 1, 6, 4, ,2 5,

�

�

�

�

�

6 5, 4, ,3 2, 1,

6 5, ,3 4, 2, 1,

6 ,3 5, 4, 2, 1,

3 6, 5, 4, 2, 1,

�

�

�

�

3 6, 5, 4, 2, ,1

3 6, 5, 4, ,1 2,

3 6, 5, ,1 4, 2,

3 6, ,1 5, 4, 2,

3 ,1 6, 5, 4, 2,

�

�

�

�

�key

In the key outside of array.

How to calculate running time?

• Each “line” of pseudocode requires a constant time. (In RAM model)

for.-end

;]1[

while;-end

;1

];[]1[

begin do

][ and 0 ile wh

;1

];[

begin do

][ to2for

keyiA

ii

iAiA

keyiAi

ji

jAkey

Alengthj

This loop runs at mostj-1 times and each timeruns at most 3 lines.

This loop runs n-1 timesand each time runs at most4+3(j-1) lines.

2

)2)(1(31

))1(34()(2

nnn

jnTn

j

Remark on Running Time

• Running time is a function of input size.• In Turing machine model, the input size

of sorting is

ime.constant tin not runs "][" Hence,

logloglog 22212

keyiA

aaa n

Divide and Conquer

• Divide the problem into subproblems.

• Conquer the subproblems by solving them recursively.

• Combine the solutions to subproblems into the solution for original problem.

Merge Sort

end

);,1,(Sort-Merge

begin

ProgramMain

]....[subarray in elements sortswhich

procedure a be ),,(Sort-MergeLet

nA

rpA

rpA

Procedure

then.-end

);,,,(Merge

);,1,(Sort-Merge

);,,(Sort-Merge

;2/)(begin then

if

),,(Sort-Merge

rqpA

rqA

qpA

rpq

rp

rpA

];[][ do

to1for

];1[][ do

to1for

1];..1[ and 1]..1[array create

;

;1

),,,(Merge

2

1

21

2

1

jqAjR

nj

ipAiL

ni

nRnL

qrn

pqn

rqpA

;1 and ][][ else

1 and ][][hen t

][][ if do

tofor

;1

;1

;)1( ;)1( 21

jjjRkA

iiiLkA

jRiA

rpk

j

i

nRnL

5, 4, 3, 2, 1, :

; 8, ,6 4, 1, : ; 7, ,5 3, 2, :

4, 3, 2, 1, :

; 8, 6, ,4 1, : ; 7, ,5 3, 2, :

3, 2, 1, :

; 8, 6, ,4 1, : ; 7, 5, ,3 2, :

2, 1, :

; 8, 6, ,4 1, : ; 7, 5, 3, ,2 :

,1:

; 8, 6, 4, ,1 : ; 7, 5, 3, ,2 :

A

RL

A

RL

A

RL

A

RL

A

RL

Example

; 8, 6, 4, 1, : ; 7, 5, 3, 2, :

8, 7, 6, 5, 4, 3, 2, 1, :

; ,8 6, 4, 1, : ; 7, 5, 3, 2, :

7, 6, 5, 4, 3, 2, 1, :

; ,8 6, 4, 1, : ; ,7 5, 3, 2, :

6, 5, 4, 3, 2, ,1:

; 8, ,6 4, 1, : ; ,7 5, 3, 2, :

RL

A

RL

A

RL

A

RL

Procedure

then.-end

);,,,(Merge

);,1,(Sort-Merge

);,,(Sort-Merge

;2/)(begin then

if

),,(Sort-Merge

rqpA

rqA

qpA

rpq

rp

rpA

)(nT

)(n

)()2/(2)( nnTnT

)2/( nT

)2/( nT

))(()())(()(

.for )()()(0

such that

0 and 0 constantsexist there

means ))(()(

012

021

nfngngnf

nnngcnfngc

ncc

ngnf

Symmetry

How to Solve Recurrences

• Recursion-tree method.

• Substitution method.

• Master method.

Recursion Tree)2( kT

)1(T

)2( 2kT

)2( 1kT)2( 1kT

)2( 2kT)2( 2kT )2( 2kT

)1(T)1(T)1(T

kc 22

kk cc 222 21

2

kk kcT 2)2( 2

Substitution Method

• Guess the form of the solution.

• Use math induction to find the constants and also show the solution works.

means )()2/(2)( nnTnT

ncnTnTncnT

nnncc

21

0021

)2/(2)()2/(2

)0)()(0,0(

have Then we

)).(),...,2(),1(max( Choose 02 nTTTc

.)2/(2)( )1( 2ncnTnTn

We work to start from here.

)()2/(2)( Solve nnTnT

.)10 (Take

)10 (require lg

)1.0(lg

))/)5.0(lg5.0((lg

)3for 5.0/lg( )5.0)(lg1(

)/lg1)(lg2/12/(2

2/lg2/2)( ,3for Then

.2/lg2/)2/( Assume )2(

.) (Require .22(2) .0)1( ,2For (1)

:(analysis)Induction by Prove

)lg()( i.e., ,lg)( :Guess

2

2

2

2

2

2

2

22

cc

ccncn

nccncn

ncnncncn

nnencnnc

ncnennc

ncnncnTn

nncnT

ccccTTn

nnOnTncnnT

nen

exen

nn

nn

xn

/)(lg1lg

)1:(note lg1lg

)/11lg()2/lg(

)2/12/lg(2/lg

/1

lg10

)3for 5.0/lg( )5.0)(lg1(10

)/lg1)(lg2/12/(20

2/lg2/20)( ,3for Then

.2/lg2/10)2/( Assume )2(

.2(2) .0)1( ,2For (1)

:induction almathematicby lg10)( prove willWe

2

22

22

22

2

2

2

nnc

nnencnnc

ncnennc

ncnncnTn

nncnT

cTTn

nncnT

Formal Proof

Remark

).lg()(then

)()2/(2)( If

nnOnT

nOnTnT

Master Theorem

)).(()( then , largely sufficient

and 1constant afor )()/( if and

,0constant somefor )()( If

).lg()( then ),()( If

).()(then

,0constant somefor )()( If

./or / means / and ,1,1

constants where)()/()(Let

log

loglog

log

log

nfnTn

cncfbnaf

nnf

nnnTnnf

nnT

nOnf

bnbnbnba

nfbnaTnT

a

aa

a

a

b

bb

b

b

))(()())(()(

.for )()(0

such that

0 and 0 constantsexist there

means ))(()(

0

0

nfOngngnf

nnnfncg

nc

ngnf

Relationship

))(()())(()(

).()( )0)()(0(

means ))(()(

).()()( )0)()(0,0(

means ))(()(

).()( )0)()(0(

means ))(()(

1001

210021

2002

nfOngngnf

nfngcnnnc

ngnf

ngcnfngcnnncc

ngnf

ngcnfnnnc

ngOnf

Summary

What we learnt in this lecture?

• How to calculate running time.

• How to solve recurrences.

• Insertion sort and Merge sort.

• Divide and conquer

• Lecture Notes give you key points in each lecture.

• You must read textbook after lectures in order to study well.

)).)(,max( (Take

lg

.)(lg

)2/lg()2/(2)(Then

).2/lg()2/()2/( Assume )2(

).()( ,For (1)

:(analysis) ProofInduction

)lg()( i.e., ,lg)( :Guess

.,0 constants somefor

)2/(2)()2/(2 :Note

02

2

2

00

012

21

nTcc

ncn

nccncn

ncnncnT

nncnT

nTnTnn

nnOnTncnnT

nncc

ncnTnTncnT