Algorithms Lecture 1. Introduction The methods of algorithm design form one of the core practical...
-
Upload
brook-logan -
Category
Documents
-
view
215 -
download
0
Transcript of Algorithms Lecture 1. Introduction The methods of algorithm design form one of the core practical...
Algorithms
Lecture 1
Introduction
• The methods of algorithm design form one of the core practical technologies of computer science.
• The main aim of this lecture is to familiarize the student with the framework we shall use through the course about the design and analysis of algorithms.
• We start with a discussion of the algorithms needed to
solve computational problems. The problem of sorting is used as a running example.
• We introduce a pseudocode to show how we shall
specify the algorithms.
Algorithms
• The word algorithm comes from the name of a Persian mathematician Abu Ja’far Mohammed ibn-i Musa al Khowarizmi.
• In computer science, this word refers to a special method useable by a computer for solution of a problem. The statement of the problem specifies in general terms the desired input/output relationship.
• For example, sorting a given sequence of numbers into nondecreasing order provides fertile ground for introducing many standard design techniques and analysis tools.
The problem of sorting
Insertion Sort
Example of Insertion Sort
Example of Insertion Sort
Example of Insertion Sort
Example of Insertion Sort
Example of Insertion Sort
Example of Insertion Sort
Example of Insertion Sort
Example of Insertion Sort
Example of Insertion Sort
Example of Insertion Sort
Example of Insertion Sort
Analysis of algorithms
The theoretical study of computer-programperformance and resource usage.
What’s more important than performance?• modularity
• correctness• maintainability• functionality• robustness
• user-friendliness• programmer time
• simplicity• extensibility
• reliability
Analysis of algorithms
Why study algorithms andperformance?
• Algorithms help us to understand scalability.• Performance often draws the line between whatis feasible and what is impossible.• Algorithmic mathematics provides a languagefor talking about program behavior.• The lessons of program performance generalizeto other computing resources.• Speed is fun!
Running Time
• The running time depends on the input: analready sorted sequence is easier to sort.
• Parameterize the running time by the size ofthe input, since short sequences are easier to
sort than long ones.
• Generally, we seek upper bounds on therunning time, because everybody likes a
guarantee.
Kinds of analyses
Worst-case: (usually)• T(n) = maximum time of algorithm
on any input of size n.
Average-case: (sometimes)• T(n) = expected time of algorithm
over all inputs of size n.• Need assumption of statistical
distribution of inputs.
Best-case:• Cheat with a slow algorithm that
works fast on some input.
Machine-independent time
What is insertion sort’s worst-case time?
• It depends on the speed of our computer:• relative speed (on the same machine),
• absolute speed (on different machines).
BIG IDEA:• Ignore machine-dependent constants.
• Look at growth of “Asymptotic Analysis”
nnT as )(
Machine-independent time: An example
A pseudocode for insertion sort ( INSERTION SORT ). INSERTION-SORT(A)
1 for j 2 to length [A]2 do key A[ j] 3 Insert A[j] into the sortted sequence A[1,..., j-1].4 i j – 15 while i > 0 and A[i] > key6 do A[i+1] A[i]7 i i – 18 A[i +1] key
Analysis of INSERTION-SORT(contd.)
1]1[8
)1(17
)1(][]1[6
][05
114
10]11[ sequence
sorted theinto][Insert 3
1][2
][21
timescost SORT(A)-INSERTION
8
27
26
25
4
2
1
nckeyiA
tcii
tciAiA
tckeyiAandi
ncji
njA
jA
ncjAkey
ncAlengthj
nj j
nj j
nj j
do
while
do
tofor
Analysis of INSERTION-SORT(contd.)
)1()1()1()(2
62
5421
n
jj
n
jj tctcncnccnT
).1()1( 82
7
nctcn
jj
•The total running time is
Analysis of INSERTION-SORT(contd.)
The best case: The array is already sorted. (tj =1 for j=2,3, ...,n)
)1()1()1()1()( 85421 ncncncncncnT
).()( 854285421 ccccnccccc
Analysis of INSERTION-SORT(contd.)
•The worst case: The array is reverse sorted
(tj =j for j=2,3, ...,n).
)12/)1(()1()( 521 nncncncnT
)1()2/)1(()2/)1(( 876 ncnncnnc
ncccccccnccc )2/2/2/()2/2/2/( 87654212
765
2
)1(1
nnj
n
j
cbnannT 2)(
Growth of Functions
Although we can sometimes determine the exact running time of an algorithm, the extra precision is not usually worth the effort of computing it.
For large inputs, the multiplicative constants and lower order terms of an exact running time are dominated by the effects of the input size itself.
Asymptotic Notation
The notation we use to describe the asymptotic running time of an algorithm are defined in terms of functions whose domains are the set of natural numbers
...,2,1,0N
-notation
• For a given function , we denote by the set of functions
• A function belongs to the set if there exist positive constants and such that it can be “sandwiched” between and for sufficienly large n.
)(ng ))(( ng
021
021
allfor )()()(c0
s.t.and,, constants positiveexist there:)())((
nnngcnfng
nccnfng
)(nf ))(( ng1c 2c
)(1 ngc)(2 ngc
Θ
O-notation
• For a given function , we denote by the set of functions
• We use O-notation to give an asymptotic upper bound on a function, to within a constant factor.
)(ng ))(( ngO
0
0
allfor )()(0
s.t.and constants positiveexist there:)())((
nnncgnf
ncnfngO
Ω-notation
• For a given function , we denote by the set of functions
• We use O-notation to give an asymptotic lower bound on a function, to within a constant factor.
)(ng ))(( ng
0
0
allfor )()(0
s.t.and constants positiveexist there:)())((
nnnfncg
ncnfng
Asymptotic notation
Graphic examples of and . ,, O
22
221 3
2
1ncnnnc
213
2
1c
nc
Example
Show that
We must find c1 and c2 such that
Dividing bothsides by n2 yields
For
)(32
1)( 22 nnnnf
)(32
17 22
0 nnnn
Theorem
• For any two functions and , we have
if and only if
)(ng
))(()( ngnf
)(nf
)).(()( and ))(()( ngnfngOnf
Olması
olduğunu gösterir
)2(5223 nnn
Example
)2(5223)( nnnnf
)2(5223 nOnn
o-notation
• We use o-notation to denote an upper bound that is not asymptotically tight.
• We formally define as the set))(( ngo
0
0
allfor )()(0
s.t.0 constants aexist there
0constant positiveany for :)(
))((
nnncgnf
n
cnf
ngo
0)(
)(lim
ng
nf
n
ω-notation
• We use ω-notation to denote an upper bound that is not asymptotically tight.
• We formally define as the set))(( ng
0
0
allfor )()(0
s.t.0 constants aexist there
0constant positiveany for :)(
))((
nnnfncg
n
cnf
ng
)(
)(lim ng
nf
n
Insertion sort analysis
Merge Sort
Merging two sorted arrays
Merging two sorted arrays
Merging two sorted arrays
Merging two sorted arrays
Merging two sorted arrays
Merging two sorted arrays
Merging two sorted arrays
Merging two sorted arrays
Merging two sorted arrays
Merging two sorted arrays
Merging two sorted arrays
Merging two sorted arrays
Merging two sorted arrays
Analyzing merge sort
Recurrence for merge sort
Recursion tree
Recursion tree
Recursion tree
Recursion tree
Recursion tree
Recursion tree
Recursion tree
Recursion tree
Recursion tree
Recursion tree