CSC2105: Algorithms Mashiour Rahman [email protected] American International University Bangladesh.
-
Upload
griffin-asher-pitts -
Category
Documents
-
view
219 -
download
0
Transcript of CSC2105: Algorithms Mashiour Rahman [email protected] American International University Bangladesh.
CSC2105: AlgorithmsCSC2105: Algorithms
Mashiour RahmanMashiour [email protected]@aiub.edu
American International University BangladeshAmerican International University Bangladesh
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 22
LiteratureLiterature
Introduction to Algorithms, Second Edition, Introduction to Algorithms, Second Edition, Thomas H. Cormen, Charle E. Leiserson, Thomas H. Cormen, Charle E. Leiserson, Ronald L. Rivest, Clifford Stein (CLRS). Ronald L. Rivest, Clifford Stein (CLRS).
Fundamental of Computer Algorithms, Ellis Fundamental of Computer Algorithms, Ellis Horowitz, Sartaj Sahni, Sanguthevar Horowitz, Sartaj Sahni, Sanguthevar Rajasekaran (HSR).Rajasekaran (HSR).
Helpful link for Problem Solving : Helpful link for Problem Solving : http://acm.uva.es/problemset/http://acm.uva.es/problemset/
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 33
The Goals of this CourseThe Goals of this Course
The main things we will learn in this course:The main things we will learn in this course: To To thinkthink algorithmically algorithmically and get the spirit of how and get the spirit of how
algorithms are designed.algorithms are designed. To get to know a To get to know a toolboxtoolbox of of classical classical algorithms.algorithms. To learn a number of algorithm design To learn a number of algorithm design techniquestechniques
(such as divide-and-conquer).(such as divide-and-conquer). To reason (in a precise and formal way) about the To reason (in a precise and formal way) about the
efficiencyefficiency and the and the correctnesscorrectness of algorithms. of algorithms.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 44
GeneralGeneral
Algorithms are first solved on paper and later Algorithms are first solved on paper and later keyed in on the computer.keyed in on the computer.
The most important thing is to be The most important thing is to be simple simple andand preciseprecise..
During lectures:During lectures: Interaction is welcome; ask questions.Interaction is welcome; ask questions. Additional explanations and examples if desired.Additional explanations and examples if desired. Speed up/slow down the progress.Speed up/slow down the progress.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 55
PrerequisitesPrerequisites
Introduction to programmingIntroduction to programming Data types, operationsData types, operations Conditional statementsConditional statements LoopsLoops Procedures and functionsProcedures and functions C/C++/JavaC/C++/Java
Computer lab (edit, compile, execute)Computer lab (edit, compile, execute)
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 66
HistoryHistory
Name: Persian mathematician Mohammed al-Name: Persian mathematician Mohammed al-Khowarizmi, in Latin became Algorismus.Khowarizmi, in Latin became Algorismus.
First algorithm: Euclidean Algorithm, greatest First algorithm: Euclidean Algorithm, greatest common divisor, 400-300 B.C.common divisor, 400-300 B.C.
1919thth century – Charles Babbage, Ada Lovelace. century – Charles Babbage, Ada Lovelace. 2020thth century – Alan Turing, Alonzo Church, century – Alan Turing, Alonzo Church,
John von Neumann.John von Neumann.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 77
Data Structures and AlgorithmsData Structures and Algorithms
Data structureData structure Organization of data to solve the problem at hand.Organization of data to solve the problem at hand.
AlgorithmAlgorithm Outline, the essence of a computational procedure, Outline, the essence of a computational procedure,
step-by-step instructions.step-by-step instructions. ProgramProgram
Implementation of an algorithm in some Implementation of an algorithm in some programming language. programming language.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 88
Overall PictureOverall Picture
Using a computer to Using a computer to help solve problems.help solve problems. Precisely specify the Precisely specify the
problem.problem. Designing programsDesigning programs
architecture architecture algorithmsalgorithms
Writing programsWriting programs Verifying (testing) Verifying (testing)
programsprograms
Data Structure and Algorithm Design Goals
Implementation Goals
Correctness Efficiency
Robustness
Adaptability
Reusability
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 99
Overall Picture/2Overall Picture/2
This course is This course is notnot about: about: Programming languagesProgramming languages Computer architectureComputer architecture Software architectureSoftware architecture Software design and implementation principlesSoftware design and implementation principles
Issues concerning small and large scale programming.Issues concerning small and large scale programming. We will only touch upon the theory of We will only touch upon the theory of
complexity and computability.complexity and computability.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 1010
Algorithmic problemAlgorithmic problem
Infinite number of input Infinite number of input instancesinstances satisfying the satisfying the specification. For example:specification. For example:
A sorted, non-decreasing sequence of natural numbers. A sorted, non-decreasing sequence of natural numbers. The sequence is of non-zero, finite length:The sequence is of non-zero, finite length:
1, 20, 908, 909, 100000, 1000000000.1, 20, 908, 909, 100000, 1000000000. 3. 3.
Specification of input
?Specification of output as a function of input
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 1111
Algorithmic SolutionAlgorithmic Solution
Algorithm describes actions on the input instanceAlgorithm describes actions on the input instance.. There may be many correct algorithms for the There may be many correct algorithms for the
same algorithmic problemsame algorithmic problem..
Input instance, adhering to the specification
Algorithm Output related to the input as required
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 1212
Definition of an AlgorithmDefinition of an Algorithm
An An algorithmalgorithm is a sequence of is a sequence of unambiguousunambiguous instructions for solving a problem, i.e., for instructions for solving a problem, i.e., for obtaining a obtaining a required outputrequired output for any for any legitimate legitimate inputinput in a finite amount of time. in a finite amount of time.
Properties:Properties: PrecisionPrecision DeterminismDeterminism FinitenessFiniteness
EfficiencyEfficiency CorrectnessCorrectness GeneralityGenerality
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 1313
How to Develop an AlgorithmHow to Develop an Algorithm
Precisely definePrecisely define the problem. Precisely specify the the problem. Precisely specify the input and output. Consider all cases. input and output. Consider all cases.
Come up with a Come up with a simple plansimple plan to solve the problem at to solve the problem at hand.hand. The plan is language independent.The plan is language independent. The precise problem specification influences the plan.The precise problem specification influences the plan.
Turn the plan into an implementationTurn the plan into an implementation The problem representation (data structure) influences the The problem representation (data structure) influences the
implementation.implementation.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 1414
Preconditions, PostconditionsPreconditions, Postconditions
It is important to specify the preconditions and It is important to specify the preconditions and the postconditions of algorithms:the postconditions of algorithms: INPUTINPUT: precise specifications of what the : precise specifications of what the
algorithm gets as an input.algorithm gets as an input. OUTPUTOUTPUT: precise specifications of what the : precise specifications of what the
algorithm produces as an output, and how this algorithm produces as an output, and how this relates to the input. The handling of special cases relates to the input. The handling of special cases of the input should be described.of the input should be described.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 1515
Analysis of AlgorithmsAnalysis of Algorithms
Efficiency:Efficiency: Running timeRunning time Space usedSpace used
Efficiency as a function of the input size:Efficiency as a function of the input size: Number of data elements (numbers, points).Number of data elements (numbers, points). The number of bits of an input number .The number of bits of an input number .
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 1616
The RAM modelThe RAM model
It is important to choose the level of detail.It is important to choose the level of detail. The RAM model:The RAM model:
Instructions (each taking constant time), we Instructions (each taking constant time), we usually choose one type of instruction as a usually choose one type of instruction as a characteristiccharacteristic operation that is counted: operation that is counted:
Arithmetic (add, subtract, multiply, etc.)Arithmetic (add, subtract, multiply, etc.) Data movement (assign)Data movement (assign) Control flow (branch, subroutine call, return)Control flow (branch, subroutine call, return) ComparisonComparison
Data types – integers, characters, and floats Data types – integers, characters, and floats
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 1717
Analysis of Insertion SortAnalysis of Insertion Sort
for j 2 to n do keyA[j] // Insert A[j] into A[1..j-1] ij-1 while i>0 and A[i]>key do A[i+1]A[i] i-- A[i+1]:=key
costc1
c2
0 c3
c4
c5
c6
c7
timesnn-1n-1n-1
n-1
2
n
jjt
2( 1)
n
jjt
2( 1)
n
jjt
Time to compute the Time to compute the running timerunning time as a function of as a function of the the input size input size (exact analysis).(exact analysis).
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 1818
Analysis of Insertion Sort/2Analysis of Insertion Sort/2
The running time of an algorithm is the sum of The running time of an algorithm is the sum of the running times of each state-ment.the running times of each state-ment.
A statement with cost c that is executed n A statement with cost c that is executed n times contributes c*n to the running time.times contributes c*n to the running time.
The total running time T(n) of insertion sort isThe total running time T(n) of insertion sort is T(n) = c1*n + c2(n-1) + c3(n-1) + c4 +T(n) = c1*n + c2(n-1) + c3(n-1) + c4 + c5 + c6 + c7(n-1)c5 + c6 + c7(n-1)
2
n
jjt
2( 1)
n
jjt
2
( 1)n
jjt
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 1919
Analysis of Insertion Sort/3Analysis of Insertion Sort/3
Often the performance depends on the details Often the performance depends on the details of the input (not only the length n).of the input (not only the length n).
This is modeled by tThis is modeled by tj.j.
In the case of insertion sort the time tIn the case of insertion sort the time t jj depends depends
on the original sorting of the input array.on the original sorting of the input array.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 2020
Performance AnalysisPerformance Analysis
Often it is sufficient to count the number of iterations Often it is sufficient to count the number of iterations of the core (innermost) part.of the core (innermost) part. No distinction between comparisons, assignments, etc (that No distinction between comparisons, assignments, etc (that
means roughly the same cost for all of them).means roughly the same cost for all of them). Gives precise enough results.Gives precise enough results.
In some cases the cost of selected operations In some cases the cost of selected operations dominates all other costs.dominates all other costs. Disk I/O versus RAM operations.Disk I/O versus RAM operations. Database systems.Database systems.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 2121
Best/Worst/Average CaseBest/Worst/Average Case
Analyzing insertion sort’sAnalyzing insertion sort’s Best caseBest case: elements already sorted, : elements already sorted, ttjj=1, =1, running running
time = n-1, i.e., time = n-1, i.e., linearlinear time. time. Worst caseWorst case: elements are sorted in inverse order, : elements are sorted in inverse order,
ttjj=j-1=j-1, running time = , running time =
(n(n22-n)/2, -n)/2, i.e.,i.e., quadratic quadratic time. time. Average caseAverage case: : ttjj=j/2, =j/2, running time = running time =
(n(n22+n-2)/4, +n-2)/4, i.e.,i.e., quadratic quadratic time. time.
2( 1)
n
jjt
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 2222
Best/Worst/Average Case/3Best/Worst/Average Case/3
For inputs of all sizes:For inputs of all sizes:
1n
2n
3n
4n
5n
6n
Input instance size
Run
ning
tim
e
1 2 3 4 5 6 7 8 9 10 11 12 …..
best-case
average-case
worst-case
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 2323
Best/Worst/Average Case/4Best/Worst/Average Case/4
Worst case Worst case is usually used:is usually used: It is an upper-bound.It is an upper-bound. In certain application domains (e.g., air traffic In certain application domains (e.g., air traffic
control, surgery) knowing the control, surgery) knowing the worst-caseworst-case time time complexity is of crucial importance.complexity is of crucial importance.
For some algorithms For some algorithms worst caseworst case occurs fairly often- occurs fairly often- The The average caseaverage case is often as bad as the is often as bad as the worst caseworst case.. Finding the Finding the average case average case can be very difficult.can be very difficult.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 2424
Analysis of Linear SearchAnalysis of Linear Search
INPUTINPUT: A[1..n] – a sorted array of integers, : A[1..n] – a sorted array of integers, q – q – an integer.an integer. OUTPUTOUTPUT: an index : an index j j such that A[such that A[jj] = ] = q. NILq. NIL if if j j (1(1jjnn): A[): A[jj] ] qq j := 1j := 1while while j j nn and and A[j] A[j] q q dodo j++j++if if j j nn thenthen returnreturn j jelse return else return NILNIL
INPUTINPUT: A[1..n] – a sorted array of integers, : A[1..n] – a sorted array of integers, q – q – an integer.an integer. OUTPUTOUTPUT: an index : an index j j such that A[such that A[jj] = ] = q. NILq. NIL if if j j (1(1jjnn): A[): A[jj] ] qq j := 1j := 1while while j j nn and and A[j] A[j] q q dodo j++j++if if j j nn thenthen returnreturn j jelse return else return NILNIL
Worst case running time: n Average case running time: n/2 Best case running time: 0
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 2525
Binary SearchBinary Search
INPUTINPUT: A[1..n] – a sorted array of (increasing) integers, : A[1..n] – a sorted array of (increasing) integers, q – q – an integer.an integer. OUTPUTOUTPUT: an index : an index j j such that A[such that A[jj] = ] = q. NILq. NIL, if , if j j (1(1jjnn): A[): A[jj] ] qq l := 1; r := l := 1; r := nndodo
m := m := (l+r)/2(l+r)/2if if A[m] = q A[m] = q thenthen return return mmelse if else if A[m] > q A[m] > q then then r := m-1r := m-1else else l := m+1 l := m+1
while while l <= rl <= rreturn return NILNIL
INPUTINPUT: A[1..n] – a sorted array of (increasing) integers, : A[1..n] – a sorted array of (increasing) integers, q – q – an integer.an integer. OUTPUTOUTPUT: an index : an index j j such that A[such that A[jj] = ] = q. NILq. NIL, if , if j j (1(1jjnn): A[): A[jj] ] qq l := 1; r := l := 1; r := nndodo
m := m := (l+r)/2(l+r)/2if if A[m] = q A[m] = q thenthen return return mmelse if else if A[m] > q A[m] > q then then r := m-1r := m-1else else l := m+1 l := m+1
while while l <= rl <= rreturn return NILNIL
Idea: Have a left and right bound. Elements to the right of r are bigger than the search element. Equivalent for l.
In each step reduce the range of the search space by half.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 2626
Analysis of Binary SearchAnalysis of Binary Search
How many times the loop is executed?How many times the loop is executed? With each execution the difference between With each execution the difference between ll and and rr is cut in half. is cut in half.
Initially the difference is Initially the difference is n.n. The loop stops when the difference becomes 0 (less than The loop stops when the difference becomes 0 (less than
1)1) . . How many times do you have to cut How many times do you have to cut n n in half to get in half to get
0?0? log n log n – better than the brute-force approach of – better than the brute-force approach of
linear search (linear search (nn).).
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 2727
Linear Search vs Binary SearchLinear Search vs Binary Search
Costs of linear search: nCosts of linear search: n Costs of binary search: log(n)Costs of binary search: log(n) Should we care?Should we care? Phone book with 200’000 entries:Phone book with 200’000 entries:
n = 200’000n = 200’000 log n = log 200’000 = 17.6log n = log 200’000 = 17.6
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 2828
Asymptotic AnalysisAsymptotic Analysis
Goal: Goal: tto simplify the analysis of the running time by o simplify the analysis of the running time by getting rid ofgetting rid of details, which are affected by specific details, which are affected by specific implementation and hardware implementation and hardware ““rounding” of numbers: rounding” of numbers: 1,000,0011,000,001 1,000,0001,000,000 ““rounding” of functionsrounding” of functions: 3: 3nn22 nn22
Capturing the essence: how the running time of an Capturing the essence: how the running time of an algorithm increases with the size of the input algorithm increases with the size of the input in the in the limitlimit.. Asymptotically more efficient algorithms are best for all Asymptotically more efficient algorithms are best for all
but small inputs but small inputs
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 2929
Asymptotic NotationAsymptotic Notation
The “big-Oh” The “big-Oh” OO--NotationNotation asymptotic upper boundasymptotic upper bound f(n) = O(g(n)),f(n) = O(g(n)), if there exists if there exists
constants constants c>0c>0 and and nn00>0, s.t. >0, s.t. f(n) f(n) c g(n) c g(n) for for n n nn00
f(n)f(n) and and g(n)g(n) are functions are functions over non-negative integersover non-negative integers
Used for Used for worst-caseworst-case analysisanalysis
)(nf( )c g n
0n Input Size
Run
ning
Tim
e
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 3030
The “big-Omega” The “big-Omega” NotationNotation asymptotic lower boundasymptotic lower bound f(n) = f(n) = (g(n))(g(n)) if there exists if there exists
constants constants c>0c>0 and and nn00>0, s.t. >0, s.t. c g(n) c g(n) f(n) f(n) for for n n nn00
Used to describe Used to describe best-case best-case running times or lower bounds running times or lower bounds of algorithmic problems.of algorithmic problems. E.g., lower-bound of searching in E.g., lower-bound of searching in
an unsorted array is an unsorted array is (n). (n). Input Size
Run
ning
Tim
e
)(nf
( )c g n
0n
Asymptotic Notation/2Asymptotic Notation/2
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 3131
Asymptotic Notation/3Asymptotic Notation/3
Simple Rule: Drop lower order terms and Simple Rule: Drop lower order terms and constant factors.constant factors. 50 50 n n log log n n is O(is O(n n log log n)n) 77n n - 3 is O(- 3 is O(nn)) 88nn22 log log n n + 5+ 5nn22 + + n n is O(is O(nn22 log log nn))
Note: Although (50 Note: Although (50 n n log log nn) is) is O(O(nn55), it is ), it is expected that an approximation is of the expected that an approximation is of the smallest possible order.smallest possible order.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 3232
The “big-Theta” The “big-Theta” NotationNotation asymptoticly tight boundasymptoticly tight bound f(n) =f(n) = (g(n))(g(n)) if there exists if there exists
constants constants cc11>0, c>0, c22>0,>0, and and nn00>0, >0, s.t. s.t. for for n n nn00 cc11 g(n) g(n) f(n) f(n) c c22 g(n) g(n)
f(n)f(n) == (g(n)) (g(n)) if and only if if and only if f(n)f(n) == (g(n)) (g(n)) and and f(n)f(n) == (g(n))(g(n))
O(f(n)) O(f(n)) is often abused instead is often abused instead of of (f(n)) (f(n))
Input Size
Run
ning
Tim
e
)(nf
0n
Asymptotic Notation/4Asymptotic Notation/4
)(ngc 2
)(ngc 1
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 3333
A Quick Math Refresher A Quick Math Refresher
Arithmetic progressionArithmetic progression
Geometric progressionGeometric progression given an integer given an integer nn00 and a real number and a real number 0<a0<a11
geometric progressions exhibit exponential growthgeometric progressions exhibit exponential growth
12
0
11 ...
1
nni n
i
aa a a a
a
0
(1 )1 2 3 ...
2
n
i
n ni n
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 3434
MiscellaneousMiscellaneous
Manipulating logarithms:Manipulating logarithms:
aalog b = log b / log alog b = log b / log a
log alog abb = b log a = b log a
Manipulating summations:Manipulating summations:
j
jj
jj
jj baba )(
j
jj
j acca
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 3535
SummationsSummations
The running time of insertion sort is The running time of insertion sort is determined by a nested loop.determined by a nested loop.
Nested loops correspond to summations:Nested loops correspond to summations:
for j := 2 to n key := A[j] i := j-1 while i>0 and A[i]>key A[i+1] := A[i] i := i-1 A[i+1] := key
for j := 2 to n key := A[j] i := j-1 while i>0 and A[i]>key A[i+1] := A[i] i := i-1 A[i+1] := key
n
jj
2)1(
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 3636
Proof by InductionProof by Induction
We want to show that property We want to show that property P P is true for all is true for all integers integers n n nn00..
BasisBasis: prove that : prove that PP is true for is true for nn00.. Inductive stepInductive step: prove that if : prove that if PP is true for all is true for all kk such such
that that nn0 0 k k n n – 1 then – 1 then PP is also true for is also true for n.n. ExampleExample
BasisBasis
0
( 1)( ) for 1
2
n
i
n nS n i n
1
0
1(1 1)(1)
2i
S i
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 3737
Proof by Induction/2Proof by Induction/2
0
1
0 0
2
( 1)( ) for 1 k 1
2
( ) ( 1)
( 1 1) ( 2 )( 1)
2 2( 1)
2
k
i
n n
i i
k kS k i n
S n i i n S n n
n n n nn n
n n
Inductive StepInductive Step
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 3838
Correctness of AlgorithmsCorrectness of Algorithms
An algorithm is An algorithm is correctcorrect if for any legal input it if for any legal input it terminates and produces the desired output.terminates and produces the desired output.
Automatic proof of correctness is not possible.Automatic proof of correctness is not possible. There are practical techniques and rigorous There are practical techniques and rigorous
formalisms that help to reason about the formalisms that help to reason about the correctness of (parts of) algorithms.correctness of (parts of) algorithms.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 3939
Partial and Total CorrectnessPartial and Total Correctness
Partial correctnessPartial correctness
Any legal input
Algorithm Output
IF this point is reached, THEN this is the desired output
Total correctness
Any legal input
Algorithm Output
INDEED this point is reached, AND this is the desired output
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 4040
AssertionsAssertions
To prove partial correctness we associate a number of To prove partial correctness we associate a number of assertionsassertions (statements about the state of the (statements about the state of the execution) with specific checkpoints in the algorithm.execution) with specific checkpoints in the algorithm. E.g., E.g., AA[1], …, [1], …, AA[j][j] form an increasing sequenceform an increasing sequence
PreconditionsPreconditions – assertions that must be valid – assertions that must be valid beforebefore the execution of an algorithm or a subroutine the execution of an algorithm or a subroutine ((INPUTINPUT).).
PostconditionsPostconditions – assertions that must be valid – assertions that must be valid afterafter the execution of an algorithm or a subroutine the execution of an algorithm or a subroutine ((OUTPUTOUTPUT).).
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 4141
Pre/post-conditionsPre/post-conditions
Example:Example: Write a pseudocode algorithm to find the two Write a pseudocode algorithm to find the two
smallest numbers in a sequence of numbers (given smallest numbers in a sequence of numbers (given as an array).as an array).
INPUT: an array of integers A[1..n], n > 0INPUT: an array of integers A[1..n], n > 0 OUTPUT: (m1, m2) such thatOUTPUT: (m1, m2) such that
m1<m2 and for each im1<m2 and for each i[1..n]: m1 [1..n]: m1 A[i] and, if A[i] and, if A[i] A[i] m1, then m2 m1, then m2 A[i]. A[i].
m2 = m1 = A[1] if m2 = m1 = A[1] if j,ij,i[1..n]: A[i]=A[j][1..n]: A[i]=A[j]
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 4242
Loop InvariantsLoop Invariants
Invariants:Invariants: assertions that are valid any time they are assertions that are valid any time they are reached (many times during the execution of an reached (many times during the execution of an algorithm, e.g., in loops)algorithm, e.g., in loops)
We must show three things about loop invariants:We must show three things about loop invariants: Initialization:Initialization: it is true prior to the first iteration. it is true prior to the first iteration. Maintenance:Maintenance: ifif it is true before an iteration, it is true before an iteration, thenthen it is true it is true
after the iteration.after the iteration. Termination:Termination: when a loop terminates the invariant gives a when a loop terminates the invariant gives a
useful property to show the correctness of the algorithmuseful property to show the correctness of the algorithm
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 4343
Example: Binary Search/1Example: Binary Search/1
InitializationInitialization: : l = 1, r = n l = 1, r = n the invariant holds because there are the invariant holds because there are no elements to the left of no elements to the left of l l or to the right of or to the right of r.r.
l=1 yields l=1 yields j,i j,i [1..0]: A[i]<q [1..0]: A[i]<q this holds because [1..0] is emptythis holds because [1..0] is empty
r=n yields r=n yields j,i j,i [n+1..n]: A[i]>q [n+1..n]: A[i]>q this holds because [n+1..n] is emptythis holds because [n+1..n] is empty
l := 1r := ndo m := (l+r)/2 if A[m] = q then return m else if A[m] > q then r := m-1 else l := m+1 while l <= rreturn NIL
l := 1r := ndo m := (l+r)/2 if A[m] = q then return m else if A[m] > q then r := m-1 else l := m+1 while l <= rreturn NIL
We want to show that q is not in A if NIL is returned.
Invariant: i[1..l-1]: A[i]<q (Ia) i[r+1..n]: A[i]>q (Ib)
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 4444
Example: Binary Search/2Example: Binary Search/2
MaintenanceMaintenance: l, r, m = : l, r, m = (l+r)/2(l+r)/2 A[m]!=q & A[m]>q, r=m-1, A sorted implies A[m]!=q & A[m]>q, r=m-1, A sorted implies
kk[r+1..n]: A[k]>q (Ib)[r+1..n]: A[k]>q (Ib) A[m]!=q & A[m]<q, l=m+1, A sorted implies A[m]!=q & A[m]<q, l=m+1, A sorted implies
kk[1..l-1]: A[k]<q (Ib)[1..l-1]: A[k]<q (Ib)
Invariant: i[1..l-1]: A[i]<q (Ia) i[r+1..n]: A[i]>q (Ib)
l := 1r := ndo m := (l+r)/2 if A[m] = q then return m else if A[m] > q then r := m-
1 else l := m+1 while l <= rreturn NIL
l := 1r := ndo m := (l+r)/2 if A[m] = q then return m else if A[m] > q then r := m-
1 else l := m+1 while l <= rreturn NIL
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 4545
Example: Binary Search/3Example: Binary Search/3
TerminationTermination: l, r, l<=r: l, r, l<=r Two cases:Two cases:
l:=m+1 we get l:=m+1 we get (l+r)/2(l+r)/2+1 > l+1 > l r:=m-1 we get r:=m-1 we get (l+r)/2(l+r)/2-1 < r-1 < r
The range gets smaller during each iteration and the loop will The range gets smaller during each iteration and the loop will terminate when l<=r no longer holds.terminate when l<=r no longer holds.
Invariant: i[1..l-1]: A[i]<q(Ia) i[r+1..n]: A[i]>q (Ib)
l := 1r := ndo m := (l+r)/2 if A[m] = q then return m else if A[m] > q then r := m-1 else l := m+1 while l <= rreturn NIL
l := 1r := ndo m := (l+r)/2 if A[m] = q then return m else if A[m] > q then r := m-1 else l := m+1 while l <= rreturn NIL
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 4646
Example: Insertion Sort/1Example: Insertion Sort/1
InvariantInvariant:: outside while loopoutside while loop
A[1...j-1] is sortedA[1...j-1] is sorted A[1...j-1] A[1...j-1] A Aorigorig
inside while loop:inside while loop: A[1...i], key, A[i+1…j-1]A[1...i], key, A[i+1…j-1] A[1...i] is sortedA[1...i] is sorted A[i+1…j-1] is sortedA[i+1…j-1] is sorted A[k] > key, i+1<=k<=j-1A[k] > key, i+1<=k<=j-1
for j := 2 to n do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-- A[i+1] := key
for j := 2 to n do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-- A[i+1] := key
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 4747
Example: Insertion Sort/2Example: Insertion Sort/2
outside while loopoutside while loop A[1...j-1] is sortedA[1...j-1] is sorted A[1...j-1] A[1...j-1] A Aorigorig
inside while loop:inside while loop: A[1...i], key, A[i+1…j-1]A[1...i], key, A[i+1…j-1] A[1...i] is sortedA[1...i] is sorted A[i+1…j-1] is sortedA[i+1…j-1] is sorted A[k] > key, i+1<=k<=j-1A[k] > key, i+1<=k<=j-1
Initialization: j=2: the invariant holds, A[1…1] is trivially sorted. i=j-1: A[1...j-1], key, A[j…j-1] where key=A[j]
A[j…j-1] is empty (and thus trivially sorted) A[1…j-1] is sorted (invariant of outer loop)
for j := 2 to length(A)do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-- A[i+1] := key
for j := 2 to length(A)do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-- A[i+1] := key
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 4848
Example: Insertion Sort/2Example: Insertion Sort/2
outside while loopoutside while loop A[1…j-1] is sortedA[1…j-1] is sorted A[1...j-1] A[1...j-1] A Aorigorig
inside while loop:inside while loop: A[1...i], key, A[i+1…j-1]A[1...i], key, A[i+1…j-1] A[1...i] is sortedA[1...i] is sorted A[i+1…j-1] is sortedA[i+1…j-1] is sorted A[k] > key, i+1<=k<=j-1A[k] > key, i+1<=k<=j-1
Maintenance: (A[1…j-1] sorted) + (insert A[j]) A[1…j]
sorted). A[1...i-1], key, A[i,i+1…j-1] satisfies conditions
because of condition A[i]>key and A[1...j-1] being sorted.
for j := 2 to length(A)do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-- A[i+1] := key
for j := 2 to length(A)do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-- A[i+1] := key
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 4949
Example: Insertion Sort/2Example: Insertion Sort/2
outside while loopoutside while loop A[1…j-1] is sortedA[1…j-1] is sorted A[1...j-1] A[1...j-1] A Aorigorig
inside while loop:inside while loop: A[1...i], key, A[i+1…j-1]A[1...i], key, A[i+1…j-1] A[1...i] is sortedA[1...i] is sorted A[i+1…j-1] is sortedA[i+1…j-1] is sorted A[k] > key, i+1<=k<=j-1A[k] > key, i+1<=k<=j-1
Termination: main loop, j=n+1: A[1…n] sorted. A[i]key: (A[1...i], key, A[i+1…j-1]) = A[1…j-1] is
sorted i=0: (key, A[1…j-1]) = A[1…j-1] is sorted.
for j := 2 to length(A)do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-- A[i+1] := key
for j := 2 to length(A)do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-- A[i+1] := key
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 5050
RecursionRecursion
An object is recursiveAn object is recursive if if it contains itself as part of it, orit contains itself as part of it, or iit is defined in terms of itself.t is defined in terms of itself.
Factorial: n!Factorial: n! How do you compute 10!?How do you compute 10!? n! = 1 * 2 * 3 *...* nn! = 1 * 2 * 3 *...* n n! = n * (n-1)!n! = n * (n-1)!
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 5151
Factorial FunctionFactorial Function
Pseudocode of factorial:Pseudocode of factorial:fac1fac1INPUTINPUT: n: n – – a natural number.a natural number. OUTPUTOUTPUT: n! (factorial of n) : n! (factorial of n) fac1(n)fac1(n) if if n < n < 22 thenthen returnreturn 1 1 else return else return n * fac1(n-1)n * fac1(n-1)
fac1fac1INPUTINPUT: n: n – – a natural number.a natural number. OUTPUTOUTPUT: n! (factorial of n) : n! (factorial of n) fac1(n)fac1(n) if if n < n < 22 thenthen returnreturn 1 1 else return else return n * fac1(n-1)n * fac1(n-1)
A recursive procedure includes aA recursive procedure includes a Termination condition (determines when and how Termination condition (determines when and how
to stop the recursion).to stop the recursion). One (or more) recursive calls.One (or more) recursive calls.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 5252
Tracing the ExecutionTracing the Execution
fac(3)
fac(1) 1
fac(2) 2 * fac(1)
1
2
6
3 * fac(2)
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 5353
BookkeepingBookkeeping
fac(5) 5*fac(4)
fac(4) 4*fac(3)
fac(3) 3*fac(2)
fac(2) 2*fac(1)
fac(1) 1
1
2
6
24
120
The computer maintains an activation stack for The computer maintains an activation stack for active procedure calls (-> compiler active procedure calls (-> compiler construction). Example for fac(5).construction). Example for fac(5).
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 5454
Variants of FactorialVariants of Factorial
fac2fac2INPUTINPUT: n: n – – a natural number.a natural number. OUTPUTOUTPUT: n! (factorial of n) : n! (factorial of n) facfac22(n)(n) if if n n 22 thenthen returnreturn 1 1 return return n * fac2(n-1)n * fac2(n-1)
fac2fac2INPUTINPUT: n: n – – a natural number.a natural number. OUTPUTOUTPUT: n! (factorial of n) : n! (factorial of n) facfac22(n)(n) if if n n 22 thenthen returnreturn 1 1 return return n * fac2(n-1)n * fac2(n-1)
fac3fac3INPUTINPUT: n: n – – a natural number.a natural number. OUTPUTOUTPUT: n! (factorial of n) : n! (factorial of n) facfac33(n)(n) return return n * fac3(n-1)n * fac3(n-1) if if n n 22 thenthen returnreturn 1 1
fac3fac3INPUTINPUT: n: n – – a natural number.a natural number. OUTPUTOUTPUT: n! (factorial of n) : n! (factorial of n) facfac33(n)(n) return return n * fac3(n-1)n * fac3(n-1) if if n n 22 thenthen returnreturn 1 1
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 5555
Analysis of the solutionsAnalysis of the solutions
fac2 is correct fac2 is correct The return statement in the if clause terminates the The return statement in the if clause terminates the
function and, thus, the entire recursion.function and, thus, the entire recursion. fac3 is incorrectfac3 is incorrect
Infinite recursion. The termination condition is Infinite recursion. The termination condition is never reached.never reached.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 5656
Fibonacci NumbersFibonacci Numbers
DefinitionDefinition ffib(ib(11) = 1) = 1 ffib(ib(22) = 1) = 1 ffib(n) = ib(n) = ffib(n-ib(n-11) + ) + ffib(n-ib(n-22)), n>2, n>2
Numbers in the series:Numbers in the series: 1, 1, 2, 3, 5, 8, 13, 21, 34, ...1, 1, 2, 3, 5, 8, 13, 21, 34, ...
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 5757
Fibonacci Fibonacci ImplementationImplementation
fibfibINPUTINPUT: n: n – – a natural numbera natural number larger than 0 larger than 0.. OUTPUTOUTPUT: : fib(n), the nth Fibonacci number.fib(n), the nth Fibonacci number. fibfib(n)(n) if if n n 22 thenthen returnreturn 1 1 else return else return fib(n-1) + fib(n-2) fib(n-1) + fib(n-2)
fibfibINPUTINPUT: n: n – – a natural numbera natural number larger than 0 larger than 0.. OUTPUTOUTPUT: : fib(n), the nth Fibonacci number.fib(n), the nth Fibonacci number. fibfib(n)(n) if if n n 22 thenthen returnreturn 1 1 else return else return fib(n-1) + fib(n-2) fib(n-1) + fib(n-2)
Multiple recursive calls are possible.Multiple recursive calls are possible.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 5858
Fibonacci Fibonacci Implementation/2Implementation/2
intint f fibib((intint i) { i) { ifif ( (i i <<= = 2) {2) { returnreturn 1;} 1;} elseelse { { returnreturn f fibib(i-1)(i-1) + fib(i-2) + fib(i-2); }; }}}
intint main() { main() { printf(“Fibonacci of 5 is %d\n", fac(5));printf(“Fibonacci of 5 is %d\n", fac(5)); }}
intint f fibib((intint i) { i) { ifif ( (i i <<= = 2) {2) { returnreturn 1;} 1;} elseelse { { returnreturn f fibib(i-1)(i-1) + fib(i-2) + fib(i-2); }; }}}
intint main() { main() { printf(“Fibonacci of 5 is %d\n", fac(5));printf(“Fibonacci of 5 is %d\n", fac(5)); }}
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 5959
Tracing fib(4)Tracing fib(4)
fib(4) fib(3) + fib(2)
fib(2) 1
fib(3) fib(2) + fib(1)
11
3
fib(2) 1
fib(1) 1
21
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 6060
BookkeepingBookkeeping
fib(4) fib(3) + fib(2)
fib(2) 1fib(1) 1
2
3
Activation stack for fib(4).Activation stack for fib(4).
fib(3) fib(2) + fib(1) 11
1
fib(2) 1
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 6161
Mutual RecursionMutual Recursion
Recursion doesRecursion does not not always occur because a always occur because a procedureprocedure calls itself. calls itself.
Mutual recursion Mutual recursion occurs ifoccurs if two two proceduresprocedures call call each other.each other.
B A
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 6262
Mutual Recursion ExampleMutual Recursion Example
Problem: Determine whether a Problem: Determine whether a natural natural numbernumber is even.is even.
Definition of even:Definition of even: 0 is even0 is even N is odd if N-1 isN is odd if N-1 is eveneven N is even if N-1 is oddN is even if N-1 is odd
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 6363
ImplementationImplementation of even of even
Can it be used to determine whether a number Can it be used to determine whether a number is odd?is odd?
evenevenINPUTINPUT: n: n – – a natural numbera natural number..OUTPUTOUTPUT: : true if n is even; false otherwisetrue if n is even; false otherwise oddodd(n)(n) if if n = 0n = 0 then return then return TRUETRUE return !return !even(n-1)even(n-1)
even(n)even(n) if if n = 0 n = 0 thenthen returnreturn TRUE TRUE else return !else return !odd(n-1)odd(n-1)
evenevenINPUTINPUT: n: n – – a natural numbera natural number..OUTPUTOUTPUT: : true if n is even; false otherwisetrue if n is even; false otherwise oddodd(n)(n) if if n = 0n = 0 then return then return TRUETRUE return !return !even(n-1)even(n-1)
even(n)even(n) if if n = 0 n = 0 thenthen returnreturn TRUE TRUE else return !else return !odd(n-1)odd(n-1)
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 6464
Is Recursion Necessary?Is Recursion Necessary?
Theory: You can always resort to iteration and Theory: You can always resort to iteration and explicitly maintain a recursion stack.explicitly maintain a recursion stack.
Practice: Recursion is elegant and in some cases the Practice: Recursion is elegant and in some cases the best solution by far.best solution by far.
In the previous examples recursion was never In the previous examples recursion was never appropriate since there exist simple iterative appropriate since there exist simple iterative solutions.solutions.
Recursion is more expensive than corresponding Recursion is more expensive than corresponding iterative solutions since bookkeeping is necessary.iterative solutions since bookkeeping is necessary.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 6565
SortingSorting
Sorting is a classical and important algorithmic Sorting is a classical and important algorithmic problem.problem.
We look at sorting arrays (in contrast to files, which We look at sorting arrays (in contrast to files, which restrict random access).restrict random access).
A key constraint is the efficient management of the A key constraint is the efficient management of the spacespace In-place sorting algorithmsIn-place sorting algorithms
The efficiency comparison is based on the number of The efficiency comparison is based on the number of comparisons (C) and the number of movements (M).comparisons (C) and the number of movements (M).
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 6666
SortingSorting
Simple sorting methods use roughly n * n Simple sorting methods use roughly n * n comparisonscomparisons Insertion sortInsertion sort Selection sortSelection sort Bubble sortBubble sort
Fast sorting methods use roughly n * log n Fast sorting methods use roughly n * log n comparisons.comparisons. Merge sortMerge sort Heap sortHeap sort QuicksortQuicksort
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 6767
Sort
Example 2: SortingExample 2: SortingINPUTsequence of n numbers
a1, a2, a3,….,anb1,b2,b3,….,bn
OUTPUTa permutation of the input sequence of numbers
2 5 4 10 7 2 4 5 7 10
Correctness (requirements for the output)For any given input the algorithm halts with the output:
• b1 < b2 < b3 < …. < bn
• b1, b2, b3, …., bn is a permutation of a1, a2, a3,….,an
Correctness (requirements for the output)For any given input the algorithm halts with the output:
• b1 < b2 < b3 < …. < bn
• b1, b2, b3, …., bn is a permutation of a1, a2, a3,….,an
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 6868
InsertionInsertion Sort Sort
A1 nj
3 6 84 9 7 2 5 1
i
Strategy• Start with one sorted card.• Insert an unsorted card at the
correct position in the sorted part.
• Continue until all unsorted cards are inserted/sorted.
Strategy• Start with one sorted card.• Insert an unsorted card at the
correct position in the sorted part.
• Continue until all unsorted cards are inserted/sorted.
A 44 55 12 42 94 18 06 67 44 55 12 42 94 18 06 67 12 44 55 42 94 18 06 67 12 42 44 55 94 18 06 67 12 42 44 55 94 18 06 67 12 18 42 44 55 94 06 67 06 12 18 42 44 55 94 67 06 12 18 42 44 55 67 94
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 6969
Insertion Sort/2Insertion Sort/2
n
j
j2
The number of comparisons during the jth The number of comparisons during the jth iteration is iteration is at least 1: Cmin = = n-1 at least 1: Cmin = = n-1 at most j-1: Cmax = = (n*n-n-2)/2at most j-1: Cmax = = (n*n-n-2)/2
INPUT: A[1..n] – an array of integersOUTPUT: a permutation of A such that A[1]A[2]…A[n]for j := 2 to n do key = A[j] i := j-1 while i > 0 and A[i] > key do A[i+1] := A[i]; i-- A[j+1] := key
INPUT: A[1..n] – an array of integersOUTPUT: a permutation of A such that A[1]A[2]…A[n]for j := 2 to n do key = A[j] i := j-1 while i > 0 and A[i] > key do A[i+1] := A[i]; i-- A[j+1] := key
n
j 21
n
jj
21
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 7070
Insertion Sort/3Insertion Sort/3
The number of comparisons during the jth The number of comparisons during the jth iteration is:iteration is: j/2 in average: Cavg = = (n*n+n–2)/4j/2 in average: Cavg = = (n*n+n–2)/4
The number of movements is Ci+1: The number of movements is Ci+1: Mmin = = 2*(n-1), Mmin = = 2*(n-1),
Mavg = = (n*n+5n-6)/4 Mavg = = (n*n+5n-6)/4
Mmax = = (n*n+n-2)/2Mmax = = (n*n+n-2)/2
n
j 22
2/2
n
jj
n
jj
2
n
jj
212/
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 7171
SelectionSelection Sort Sort
A1 nj
1 3 42 5 7 8 9 6
i
Strategy• Start empty handed.• Enlarge the sorted part by
switching the first element of the unsorted part with the smallest element of the unsorted part.
• Continue until the unsorted part consists of one element only.
Strategy• Start empty handed.• Enlarge the sorted part by
switching the first element of the unsorted part with the smallest element of the unsorted part.
• Continue until the unsorted part consists of one element only.
A 44 55 12 42 94 18 06 67 06 55 12 42 94 18 44 67 06 12 55 42 94 18 44 67 06 12 18 42 94 55 44 67 06 12 18 42 94 55 44 67 06 12 18 42 44 55 94 67 06 12 18 42 44 55 94 67 06 12 18 42 44 55 67 94
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 7272
Selection Sort/2Selection Sort/2
The number of comparisons is indepen-dent The number of comparisons is indepen-dent of the original ordering (this is less natural of the original ordering (this is less natural behavior than insertion sort):behavior than insertion sort): C = = (n*n-n)/2C = = (n*n-n)/2
INPUT: A[1..n] – an array of integersOUTPUT: a permutation of A such that A[1]A[2]…A[n]for j := 1 to n-1 do key := A[j]; ptr := j for i := j+1 to n do if A[i] < key then ptr := i; key := A[i]; A[ptr] := A[j]; A[j] := key
INPUT: A[1..n] – an array of integersOUTPUT: a permutation of A such that A[1]A[2]…A[n]for j := 1 to n-1 do key := A[j]; ptr := j for i := j+1 to n do if A[i] < key then ptr := i; key := A[i]; A[ptr] := A[j]; A[j] := key
1
1
n
jj
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 7373
Selection Sort/3Selection Sort/3
The number of movements is: The number of movements is: Mmin = = 3*(n-1) Mmin = = 3*(n-1)
Mmax = = (n*n–n)/4 + 3*(n-1)Mmax = = (n*n–n)/4 + 3*(n-1)
1
13
n
j
1
13
n
jj
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 7474
BubbleBubble Sort Sort
A1 nj
1 3 42 5 7 9 8 6
i
Strategy• Start from the back and
compare pairs of adjacent elements.
• Switch the elements if the larger comes before the smaller.
• In each step the smallest element of the unsorted part is moved to the beginning of the unsorted part and the sorted part grows by one.
Strategy• Start from the back and
compare pairs of adjacent elements.
• Switch the elements if the larger comes before the smaller.
• In each step the smallest element of the unsorted part is moved to the beginning of the unsorted part and the sorted part grows by one.
A 44 55 12 42 94 18 06 67 06 44 55 12 42 94 18 67 06 12 44 55 18 42 94 67 06 12 18 44 55 42 67 94 06 12 18 42 44 55 67 94 06 12 18 42 44 55 67 94 06 12 18 42 44 55 67 94 06 12 18 42 44 55 67 94
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 7575
Bubble Sort/2Bubble Sort/2
The number of comparisons is indepen-dent of The number of comparisons is indepen-dent of the original ordering:the original ordering: C = = (n*n-n)/2C = = (n*n-n)/2
INPUT: A[1..n] – an array of integersOUTPUT: a permutation of A such that A[1]A[2]…A[n]for j := 2 to n do for i := n to j do if A[i-1] < A[i] then key := A[i-1]; A[i-1] := A[i]; A[i]:=key
INPUT: A[1..n] – an array of integersOUTPUT: a permutation of A such that A[1]A[2]…A[n]for j := 2 to n do for i := n to j do if A[i-1] < A[i] then key := A[i-1]; A[i-1] := A[i]; A[i]:=key
n
jj
21
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 7676
Bubble Sort/3Bubble Sort/3
The number of movements is: The number of movements is: Mmin = 0 Mmin = 0
Mmax = = 3*n*(n-1)/2Mmax = = 3*n*(n-1)/2
Mavg = = 3*n*(n-1)/4 Mavg = = 3*n*(n-1)/4
n
jj
2)1(3
2/)1(32
n
jj
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 7777
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 7878
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 7979
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 8080
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 8181
The divide- and- conquer design The divide- and- conquer design paradigmparadigm
1. Divide 1. Divide the problem (instance) into the problem (instance) into subproblems.subproblems.
2. Conquer 2. Conquer the subproblems by solving them the subproblems by solving them recursively.recursively.
3. Combine 3. Combine subproblem solutions.subproblem solutions.
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 8282
The divide- and- conquer design The divide- and- conquer design paradigmparadigm
Example: Example: merge sortmerge sort1. Divide: 1. Divide: Trivial.Trivial.2. Conquer: 2. Conquer: Recursively sort 2 subarrays.Recursively sort 2 subarrays.3. Combine: 3. Combine: Linear- time merge.Linear- time merge.Recurrence for merge sortRecurrence for merge sort T( n) T( n) = = 2 2 T( n/ 2) T( n/ 2) # subproblems# subproblems subproblem sizesubproblem size
++ O( n) O( n) work dividing and combiningwork dividing and combining
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 8383
The divide- and- conquer design The divide- and- conquer design paradigmparadigm
Binary searchBinary searchFind an element in a sorted array:Find an element in a sorted array:1. Divide: 1. Divide: Check middle element.Check middle element.2. Conquer: 2. Conquer: Recursively search 1 subarray.Recursively search 1 subarray.3. Combine: 3. Combine: Trivial.Trivial.Recurrence for binary searchRecurrence for binary searchT(n)T(n) = = 11 T(n/2)T(n/2) ++ ΘΘ(1)(1) # subproblems# subproblems subproblem subproblem work work size dividing and size dividing and combiningcombining
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 8484
The divide- and- conquer design The divide- and- conquer design paradigmparadigm
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 8585
The divide- and- conquer design The divide- and- conquer design paradigmparadigm
ConclusionConclusion
• • Divide and conquer is just one of several Divide and conquer is just one of several powerful techniques for algorithm design.powerful techniques for algorithm design.
• • Divide- and- conquer algorithms can be Divide- and- conquer algorithms can be analyzed using recurrences and the master analyzed using recurrences and the master method (so practice this math).method (so practice this math).
• • Can lead to more efficient algorithmsCan lead to more efficient algorithms
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 8686
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 8787
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 8888
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 8989
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 9090
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 9191
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 9292
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 9393
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 9494
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 9595
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 9696
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 9797
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 9898
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 9999
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 100100
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 101101
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 102102
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 103103
CSC2105CSC2105 Mashiour RahmanMashiour Rahman 104104