Post on 01-Jan-2016
description
Unit 1. Sorting and Divide and Conquer
Lecture 1 Introduction to Algorithm and Sorting
What is an Algorithm?
• An algorithm is a computational procedure that takes some value, or a set of values, as input and produces some values, or a set of values, as output.
• So, it is a sequence of computational steps that transform the input into the output.
• Correction: sequence -> combination
Sequential and Parallel
• In sequential computation, an algorithm is a sequence of computational steps.
• However, it is not true in parallel computation.
• In this course, we study only algorithms in sequential computation.
Algorithms for Sorting
.'''such that sequence
input of }',...,','{n permutatioA :Output
}.,...,,{ numbers of sequenceA :Input
21
21
21
n
n
n
aaa
aaa
aaan
Insertion Sort, Merge Sort
6. 5, 4, 3, 2, 1, :Output
3. 1, 6, 4, 2, 5, :Input
e.g.,
Efficiency
• Running time from receiving the input to producing the output.
)log(
)( 2
nnO
nOInsert Sort
Merge Sort
Running time
.for )()(0
such that
0 and 0 constantsexist there
means ))(()(
0
0
nnncgnf
nc
ngOnf
function!
theofgrowth theisabout carereally what weTherefore,
.log
large,ly sufficient is When
bigger? is who,log and
22
1
22
1
nncnc
n
nncnc
keyiA
ii
iAiA
keyiAi
ji
jAkey
Alengthj
]1[
1
][]1[ do
][ and 0 while
1
][ do
][ to2for
Insertion SortAarray input
for.-end
;]1[
while;-end
;1
];[]1[
begin do
][ and 0 while
;1
];[
begin do
][ to2for
keyiA
ii
iAiA
keyiAi
ji
jAkey
Alengthj
3 1, ,6 5, 4, 2,
3 1, 6, 5, ,4 2,
3 1, 6, ,4 5, 2,
3 1, 6, 4, 5, ,2
3 1, 6, 4, ,2 5,
�
�
�
�
�
6 5, 4, ,3 2, 1,
6 5, ,3 4, 2, 1,
6 ,3 5, 4, 2, 1,
3 6, 5, 4, 2, 1,
�
�
�
�
3 6, 5, 4, 2, ,1
3 6, 5, 4, ,1 2,
3 6, 5, ,1 4, 2,
3 6, ,1 5, 4, 2,
3 ,1 6, 5, 4, 2,
�
�
�
�
�key
In the key outside of array.
How to calculate running time?
• Each “line” of pseudocode requires a constant time. (In RAM model)
for.-end
;]1[
while;-end
;1
];[]1[
begin do
][ and 0 ile wh
;1
];[
begin do
][ to2for
keyiA
ii
iAiA
keyiAi
ji
jAkey
Alengthj
This loop runs at mostj-1 times and each timeruns at most 3 lines.
This loop runs n-1 timesand each time runs at most4+3(j-1) lines.
2
)2)(1(31
))1(34()(2
nnn
jnTn
j
Remark on Running Time
• Running time is a function of input size.• In Turing machine model, the input size
of sorting is
ime.constant tin not runs "][" Hence,
logloglog 22212
keyiA
aaa n
Divide and Conquer
• Divide the problem into subproblems.
• Conquer the subproblems by solving them recursively.
• Combine the solutions to subproblems into the solution for original problem.
Merge Sort
end
);,1,(Sort-Merge
begin
ProgramMain
]....[subarray in elements sortswhich
procedure a be ),,(Sort-MergeLet
nA
rpA
rpA
Procedure
then.-end
);,,,(Merge
);,1,(Sort-Merge
);,,(Sort-Merge
;2/)(begin then
if
),,(Sort-Merge
rqpA
rqA
qpA
rpq
rp
rpA
];[][ do
to1for
];1[][ do
to1for
1];..1[ and 1]..1[array create
;
;1
),,,(Merge
2
1
21
2
1
jqAjR
nj
ipAiL
ni
nRnL
qrn
pqn
rqpA
;1 and ][][ else
1 and ][][hen t
][][ if do
tofor
;1
;1
;)1( ;)1( 21
jjjRkA
iiiLkA
jRiA
rpk
j
i
nRnL
5, 4, 3, 2, 1, :
; 8, ,6 4, 1, : ; 7, ,5 3, 2, :
4, 3, 2, 1, :
; 8, 6, ,4 1, : ; 7, ,5 3, 2, :
3, 2, 1, :
; 8, 6, ,4 1, : ; 7, 5, ,3 2, :
2, 1, :
; 8, 6, ,4 1, : ; 7, 5, 3, ,2 :
,1:
; 8, 6, 4, ,1 : ; 7, 5, 3, ,2 :
A
RL
A
RL
A
RL
A
RL
A
RL
Example
; 8, 6, 4, 1, : ; 7, 5, 3, 2, :
8, 7, 6, 5, 4, 3, 2, 1, :
; ,8 6, 4, 1, : ; 7, 5, 3, 2, :
7, 6, 5, 4, 3, 2, 1, :
; ,8 6, 4, 1, : ; ,7 5, 3, 2, :
6, 5, 4, 3, 2, ,1:
; 8, ,6 4, 1, : ; ,7 5, 3, 2, :
RL
A
RL
A
RL
A
RL
Procedure
then.-end
);,,,(Merge
);,1,(Sort-Merge
);,,(Sort-Merge
;2/)(begin then
if
),,(Sort-Merge
rqpA
rqA
qpA
rpq
rp
rpA
)(nT
)(n
)()2/(2)( nnTnT
)2/( nT
)2/( nT
))(()())(()(
.for )()()(0
such that
0 and 0 constantsexist there
means ))(()(
012
021
nfngngnf
nnngcnfngc
ncc
ngnf
Symmetry
How to Solve Recurrences
• Recursion-tree method.
• Substitution method.
• Master method.
Recursion Tree)2( kT
)1(T
)2( 2kT
)2( 1kT)2( 1kT
)2( 2kT)2( 2kT )2( 2kT
)1(T)1(T)1(T
kc 22
kk cc 222 21
2
kk kcT 2)2( 2
Substitution Method
• Guess the form of the solution.
• Use math induction to find the constants and also show the solution works.
means )()2/(2)( nnTnT
ncnTnTncnT
nnncc
21
0021
)2/(2)()2/(2
)0)()(0,0(
have Then we
)).(),...,2(),1(max( Choose 02 nTTTc
.)2/(2)( )1( 2ncnTnTn
We work to start from here.
)()2/(2)( Solve nnTnT
.)10 (Take
)10 (require lg
)1.0(lg
))/)5.0(lg5.0((lg
)3for 5.0/lg( )5.0)(lg1(
)/lg1)(lg2/12/(2
2/lg2/2)( ,3for Then
.2/lg2/)2/( Assume )2(
.) (Require .22(2) .0)1( ,2For (1)
:(analysis)Induction by Prove
)lg()( i.e., ,lg)( :Guess
2
2
2
2
2
2
2
22
cc
ccncn
nccncn
ncnncncn
nnencnnc
ncnennc
ncnncnTn
nncnT
ccccTTn
nnOnTncnnT
nen
exen
nn
nn
xn
/)(lg1lg
)1:(note lg1lg
)/11lg()2/lg(
)2/12/lg(2/lg
/1
lg10
)3for 5.0/lg( )5.0)(lg1(10
)/lg1)(lg2/12/(20
2/lg2/20)( ,3for Then
.2/lg2/10)2/( Assume )2(
.2(2) .0)1( ,2For (1)
:induction almathematicby lg10)( prove willWe
2
22
22
22
2
2
2
nnc
nnencnnc
ncnennc
ncnncnTn
nncnT
cTTn
nncnT
Formal Proof
Remark
).lg()(then
)()2/(2)( If
nnOnT
nOnTnT
Master Theorem
)).(()( then , largely sufficient
and 1constant afor )()/( if and
,0constant somefor )()( If
).lg()( then ),()( If
).()(then
,0constant somefor )()( If
./or / means / and ,1,1
constants where)()/()(Let
log
loglog
log
log
nfnTn
cncfbnaf
nnf
nnnTnnf
nnT
nOnf
bnbnbnba
nfbnaTnT
a
aa
a
a
b
bb
b
b
))(()())(()(
.for )()(0
such that
0 and 0 constantsexist there
means ))(()(
0
0
nfOngngnf
nnnfncg
nc
ngnf
Relationship
))(()())(()(
).()( )0)()(0(
means ))(()(
).()()( )0)()(0,0(
means ))(()(
).()( )0)()(0(
means ))(()(
1001
210021
2002
nfOngngnf
nfngcnnnc
ngnf
ngcnfngcnnncc
ngnf
ngcnfnnnc
ngOnf
Summary
What we learnt in this lecture?
• How to calculate running time.
• How to solve recurrences.
• Insertion sort and Merge sort.
• Divide and conquer
• Lecture Notes give you key points in each lecture.
• You must read textbook after lectures in order to study well.
)).)(,max( (Take
lg
.)(lg
)2/lg()2/(2)(Then
).2/lg()2/()2/( Assume )2(
).()( ,For (1)
:(analysis) ProofInduction
)lg()( i.e., ,lg)( :Guess
.,0 constants somefor
)2/(2)()2/(2 :Note
02
2
2
00
012
21
nTcc
ncn
nccncn
ncnncnT
nncnT
nTnTnn
nnOnTncnnT
nncc
ncnTnTncnT