Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1...
Transcript of Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1...
![Page 1: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/1.jpg)
©Yu Chen
Design and Analysis of AlgorithmsIntroduction to Algorithms
1 A Taste of Algorithm DesignSingle Machine Scheduling ProblemReturn on Investment (ROI) Problem
2 A Taste of Algorithm AnalysisSorting Problem
3 Complexity TheoryTravelling Salesman ProblemKnapsack Problem
1 / 40
![Page 2: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/2.jpg)
©Yu Chen
What is Algorithm?
“An algorithm is a finite, definite, effective procedure, with someinput and some output.”
– Donald Knuth
2 / 40
![Page 3: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/3.jpg)
©Yu Chen
1 A Taste of Algorithm DesignSingle Machine Scheduling ProblemReturn on Investment (ROI) Problem
2 A Taste of Algorithm AnalysisSorting Problem
3 Complexity TheoryTravelling Salesman ProblemKnapsack Problem
3 / 40
![Page 4: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/4.jpg)
©Yu Chen
Single Machine Scheduling Problem
Problem. n tasks, each task i requires time ti to process (withoutwaiting), refereed to minimum processing time. We have to assignn tasks on a single machine.
flowtime of task i: endi − starti ≥ ti, starti = 0
Performance goal. find an assignment such that the total flowtimeof all n tasks is shortest.
Instance exampletask set S = 1, 2, 3, 4, 5minimum processing time: t1 = 3, t2 = 8, t3 = 5, t4 = 10,t5 = 15
4 / 40
![Page 5: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/5.jpg)
©Yu Chen
Greedy Strategy
Algorithm. Shortest processing time (SPT) firstsort (3, 8, 5, 10, 15) in an increasing order
Solution. 1, 3, 2, 4, 5
3 5 8 10 15
0 3 8 16 26 41
overall flowtime
t = 3 + (3 + 5) + (3 + 5 + 8) + (3 + 5 + 8 + 10)
+ (3 + 5 + 8 + 10 + 15)
= 3× 5 + 5× 4 + 8× 3 + 10× 2 + 15
= 94
5 / 40
![Page 6: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/6.jpg)
©Yu Chen
Modeling
Input.task set: S = 1, 2, . . . , nprocessing time of task j: tj ∈ Z+, j ∈ [n]
Output. Schedule I, a permutation of S, i.e., (i1, i2, . . . , in)Objective function. the flowtime of I:
t(I) =
n∑k=1
(n− k + 1)tik
Solution. I∗ — minimize t(I∗)
t(I∗) = mint(I) | I ∈ Permutation(S)
6 / 40
![Page 7: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/7.jpg)
©Yu Chen
Greedy Algorithm
Strategy. shortest processing time (SPT) firstAlgorithm. sort the processing time in an increasing order, thenprocess them sequentiallyCorrectness. yield the optimal solutions for all instances
Proof. If not ⇒ ∃ optimial schedule I∗ with at least one reverseorder, i.e., task i and j are adjacent but ti > tj . Switch task i andj in I∗ ; schedule I ′
I∗ ti tj
I ′ tj ti
flowtime comparison: t(I ′)− t(I∗) = tj − ti < 0 ⇒ contradicts tothe optimal property of I∗
7 / 40
![Page 8: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/8.jpg)
©Yu Chen
Greedy Algorithm
Strategy. shortest processing time (SPT) firstAlgorithm. sort the processing time in an increasing order, thenprocess them sequentiallyCorrectness. yield the optimal solutions for all instances
Proof. If not ⇒ ∃ optimial schedule I∗ with at least one reverseorder, i.e., task i and j are adjacent but ti > tj . Switch task i andj in I∗ ; schedule I ′
I∗ ti tj
I ′ tj ti
flowtime comparison: t(I ′)− t(I∗) = tj − ti < 0 ⇒ contradicts tothe optimal property of I∗
7 / 40
![Page 9: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/9.jpg)
©Yu Chen
Heuristics is not always correct
Counterexample.Knapsack problem: four items need to insert into a knapsack, withvalues and weights as below:
label 1 2 3 4weight wi 3 4 5 2value vi 7 9 9 2
the knapsack weight limit is 6.
How to choose items to maximize the total values in the backpack?
8 / 40
![Page 10: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/10.jpg)
©Yu Chen
Failure of Greedy Algorithm
Greedy strategy. highest value-weight ration comes first, withweight limit 6
sort vi/wi in a descending order: 1, 2, 3, 47
3>
9
4>
9
5>
2
2
greedy solution: 1, 4, weight = 5, value = 9better solution: 2, 4, weight = 6, value = 11
9 / 40
![Page 11: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/11.jpg)
©Yu Chen
Return on Investment (ROI) Problem
Problem. m coins to invest n project.profit function fi(x) denotes the return on investing project iwith x coins, i = 1, 2, . . . , n.
How to maximize the overall return?Instance example: 50000 coins, 4 projects:
x f1(x) f2(x) f3(x) f4(x)
0 0 0 0 01 11 0 2 202 12 5 10 213 13 10 30 224 14 15 32 235 15 20 40 24
10 / 40
![Page 12: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/12.jpg)
©Yu Chen
Modeling
Input. n, m, fi(x), i ∈ [n], x ∈ 0, . . . ,mSolution. vector ⟨x1, x2, . . . , xn⟩, xi is the num of coins investedon project i satisfying:
objective function: maxn∑
i=1
fi(xi)
constraints:n∑
i=1
xi = m,xi ∈ 0, . . . ,m
11 / 40
![Page 13: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/13.jpg)
©Yu Chen
Brute Force Algorithm
Definition 1 (Brute Force Algorithms)A programming style that does not include any shortcuts toimprove performance, but instead relies on sheer computing powerto try all possibilities until the solution to a problem is found.
∀ n-dimension vector ⟨x1, x2, . . . , xn⟩ satisfying
x1 + x2 + · · ·+ xn = m,xi ∈ 0, . . . ,m
compute the sum of return
f1(x1) + f2(x2) + . . . , fn(xn)
find the solution with highest return
12 / 40
![Page 14: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/14.jpg)
©Yu Chen
Brute Force Algorithm
Definition 1 (Brute Force Algorithms)A programming style that does not include any shortcuts toimprove performance, but instead relies on sheer computing powerto try all possibilities until the solution to a problem is found.
∀ n-dimension vector ⟨x1, x2, . . . , xn⟩ satisfying
x1 + x2 + · · ·+ xn = m,xi ∈ 0, . . . ,m
compute the sum of return
f1(x1) + f2(x2) + . . . , fn(xn)
find the solution with highest return
12 / 40
![Page 15: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/15.jpg)
©Yu Chen
Example
x f1(x) f2(x) f3(x) f4(x)
0 0 0 0 01 11 0 2 202 12 5 10 213 13 10 30 224 14 15 32 235 15 20 40 24
x1 + x2 + x3 + x4 = 5
s1 = ⟨0, 0, 0, 5⟩, v(s1) = 24
s2 = ⟨0, 0, 1, 4⟩, v(s2) = 25
s3 = ⟨0, 0, 2, 3⟩, v(s3) = 32
. . .
s56 = ⟨5, 0, 0, 0⟩, v(s56) = 15
Solution. s = ⟨1, 0, 3, 1⟩Highest return. 11+30+20 = 61
13 / 40
![Page 16: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/16.jpg)
©Yu Chen
Efficiency of Brute Force Algorithm
each possible solution vector is a non-negative integer solution ofequation
x1 + x2 + · · ·+ xn = m
How to estimate the number of possible ⟨x1, x2, . . . , xn⟩solution can be expressed as 0-1 sequence with the followingformat: # 1 = m, # 0 = n− 1
1 . . . 1︸ ︷︷ ︸x1
0 1 . . . 1︸ ︷︷ ︸x2
0 . . . 0 1 . . . 1︸ ︷︷ ︸xn
n = 4, m = 7
candidate solution ⟨1, 2, 3, 1⟩ corresponds to:1 0 1 1 0 1 1 1 0 1
14 / 40
![Page 17: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/17.jpg)
©Yu Chen
Efficiency of Brute Force Algorithm
each possible solution vector is a non-negative integer solution ofequation
x1 + x2 + · · ·+ xn = m
How to estimate the number of possible ⟨x1, x2, . . . , xn⟩solution can be expressed as 0-1 sequence with the followingformat: # 1 = m, # 0 = n− 1
1 . . . 1︸ ︷︷ ︸x1
0 1 . . . 1︸ ︷︷ ︸x2
0 . . . 0 1 . . . 1︸ ︷︷ ︸xn
n = 4, m = 7
candidate solution ⟨1, 2, 3, 1⟩ corresponds to:1 0 1 1 0 1 1 1 0 1
14 / 40
![Page 18: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/18.jpg)
©Yu Chen
Efficiency of Brute Force Algorithm
The number of such sequences is an exponential function of inputsize
C(m+ n− 1, n− 1) =(m+ n− 1)!
m!(n− 1)!(1)
= Ω((1 + ϵ)m+n−1) (2)
Better algorithm?
15 / 40
![Page 19: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/19.jpg)
©Yu Chen
Summary: Steps of Algorithm Design
1 Modeling. give formal description of input, output andobjective function
2 Design. Choose what algorithms? How to describe it?
3 Prove. Is the algorithm correct: yielding optimal solution forall instances.
If so, how to prove it?If not, can you find an counterexample?
4 Analysis. efficiency: time and space
16 / 40
![Page 20: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/20.jpg)
©Yu Chen
Summary: Steps of Algorithm Design
1 Modeling. give formal description of input, output andobjective function
2 Design. Choose what algorithms? How to describe it?
3 Prove. Is the algorithm correct: yielding optimal solution forall instances.
If so, how to prove it?If not, can you find an counterexample?
4 Analysis. efficiency: time and space
16 / 40
![Page 21: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/21.jpg)
©Yu Chen
Summary: Steps of Algorithm Design
1 Modeling. give formal description of input, output andobjective function
2 Design. Choose what algorithms? How to describe it?
3 Prove. Is the algorithm correct: yielding optimal solution forall instances.
If so, how to prove it?If not, can you find an counterexample?
4 Analysis. efficiency: time and space
16 / 40
![Page 22: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/22.jpg)
©Yu Chen
Summary: Steps of Algorithm Design
1 Modeling. give formal description of input, output andobjective function
2 Design. Choose what algorithms? How to describe it?
3 Prove. Is the algorithm correct: yielding optimal solution forall instances.
If so, how to prove it?If not, can you find an counterexample?
4 Analysis. efficiency: time and space
16 / 40
![Page 23: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/23.jpg)
©Yu Chen
1 A Taste of Algorithm DesignSingle Machine Scheduling ProblemReturn on Investment (ROI) Problem
2 A Taste of Algorithm AnalysisSorting Problem
3 Complexity TheoryTravelling Salesman ProblemKnapsack Problem
17 / 40
![Page 24: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/24.jpg)
©Yu Chen
Insertion Algorithm
Insertion sort iterates, consuming one input element eachrepetition, and growing a sorted output list.
At each iteration, insertion sort removes one element from theinput data, finds the location it belongs within the sorted list,and inserts it there.It repeats until no input elements remain.
5 7 1 3 6 2 4input
1 3 5 6 7 2 4middle state
1 2 3 5 6 7 4after inserting 2
18 / 40
![Page 25: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/25.jpg)
©Yu Chen
Demo of Insertion Sort
input 5 7 1 3 6 2 4
beginning 5 7 1 3 6 2 4
insert 7 5 7 1 3 6 2 4
insert 1 1 5 7 3 6 2 4
insert 3 1 3 5 7 6 2 4
insert 6 1 3 5 6 7 2 4
insert 2 1 2 3 5 6 7 4
insert 4 1 2 3 4 5 6 7
19 / 40
![Page 26: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/26.jpg)
©Yu Chen
Demo of Insertion Sort
input 5 7 1 3 6 2 4
beginning 5 7 1 3 6 2 4
insert 7 5 7 1 3 6 2 4
insert 1 1 5 7 3 6 2 4
insert 3 1 3 5 7 6 2 4
insert 6 1 3 5 6 7 2 4
insert 2 1 2 3 5 6 7 4
insert 4 1 2 3 4 5 6 7
19 / 40
![Page 27: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/27.jpg)
©Yu Chen
Demo of Insertion Sort
input 5 7 1 3 6 2 4
beginning 5 7 1 3 6 2 4
insert 7 5 7 1 3 6 2 4
insert 1 1 5 7 3 6 2 4
insert 3 1 3 5 7 6 2 4
insert 6 1 3 5 6 7 2 4
insert 2 1 2 3 5 6 7 4
insert 4 1 2 3 4 5 6 7
19 / 40
![Page 28: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/28.jpg)
©Yu Chen
Demo of Insertion Sort
input 5 7 1 3 6 2 4
beginning 5 7 1 3 6 2 4
insert 7 5 7 1 3 6 2 4
insert 1 1 5 7 3 6 2 4
insert 3 1 3 5 7 6 2 4
insert 6 1 3 5 6 7 2 4
insert 2 1 2 3 5 6 7 4
insert 4 1 2 3 4 5 6 7
19 / 40
![Page 29: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/29.jpg)
©Yu Chen
Demo of Insertion Sort
input 5 7 1 3 6 2 4
beginning 5 7 1 3 6 2 4
insert 7 5 7 1 3 6 2 4
insert 1 1 5 7 3 6 2 4
insert 3 1 3 5 7 6 2 4
insert 6 1 3 5 6 7 2 4
insert 2 1 2 3 5 6 7 4
insert 4 1 2 3 4 5 6 7
19 / 40
![Page 30: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/30.jpg)
©Yu Chen
Demo of Insertion Sort
input 5 7 1 3 6 2 4
beginning 5 7 1 3 6 2 4
insert 7 5 7 1 3 6 2 4
insert 1 1 5 7 3 6 2 4
insert 3 1 3 5 7 6 2 4
insert 6 1 3 5 6 7 2 4
insert 2 1 2 3 5 6 7 4
insert 4 1 2 3 4 5 6 7
19 / 40
![Page 31: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/31.jpg)
©Yu Chen
Demo of Insertion Sort
input 5 7 1 3 6 2 4
beginning 5 7 1 3 6 2 4
insert 7 5 7 1 3 6 2 4
insert 1 1 5 7 3 6 2 4
insert 3 1 3 5 7 6 2 4
insert 6 1 3 5 6 7 2 4
insert 2 1 2 3 5 6 7 4
insert 4 1 2 3 4 5 6 7
19 / 40
![Page 32: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/32.jpg)
©Yu Chen
Demo of Insertion Sort
input 5 7 1 3 6 2 4
beginning 5 7 1 3 6 2 4
insert 7 5 7 1 3 6 2 4
insert 1 1 5 7 3 6 2 4
insert 3 1 3 5 7 6 2 4
insert 6 1 3 5 6 7 2 4
insert 2 1 2 3 5 6 7 4
insert 4 1 2 3 4 5 6 7
19 / 40
![Page 33: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/33.jpg)
©Yu Chen
Analysis of Insertion Sort
Complexity analysisworst-case: O(n2) comparison and swapbest-case: O(n) comparison and O(1) swapaverage-case: O(n2) comparison and swap
replace array with linked list: reduce swap operation in each roundto constant time
Advantages:simple: Jon Bentley shows a three-line C versionadaptive: efficient for data sets that are already substantiallysortedstable: does not change the relative order of elements withequal keysin-place: only require constant additional memoryonline: can sort a data set as it receives it
20 / 40
![Page 34: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/34.jpg)
©Yu Chen
Analysis of Insertion Sort
Complexity analysisworst-case: O(n2) comparison and swapbest-case: O(n) comparison and O(1) swapaverage-case: O(n2) comparison and swap
replace array with linked list: reduce swap operation in each roundto constant time
Advantages:simple: Jon Bentley shows a three-line C versionadaptive: efficient for data sets that are already substantiallysortedstable: does not change the relative order of elements withequal keysin-place: only require constant additional memoryonline: can sort a data set as it receives it
20 / 40
![Page 35: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/35.jpg)
©Yu Chen
Bubble Sort
Bubble sort: compares adjacent elements and swaps them if theyare in the wrong order.
the pass through the list is repeated until the list is sorted.named for the way smaller or larger elements “bubble” to thetop of the list (another name is sinking sort)
before pass
5 1 6 2 8 3 4 7
one pass
1 5 2 4 3 6 7 8
21 / 40
![Page 36: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/36.jpg)
©Yu Chen
Demo of Bubble Sort
input 5 8 1 3 6 2 4 7
pass 1 5 1 3 6 2 4 7 8
pass 2 1 3 5 2 4 6 7 8
pass 3 1 3 2 4 5 6 7 8
pass 4 1 2 3 4 5 6 7 8
pass 5 1 2 3 4 5 6 7 8
22 / 40
![Page 37: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/37.jpg)
©Yu Chen
Demo of Bubble Sort
input 5 8 1 3 6 2 4 7
pass 1 5 1 3 6 2 4 7 8
pass 2 1 3 5 2 4 6 7 8
pass 3 1 3 2 4 5 6 7 8
pass 4 1 2 3 4 5 6 7 8
pass 5 1 2 3 4 5 6 7 8
22 / 40
![Page 38: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/38.jpg)
©Yu Chen
Demo of Bubble Sort
input 5 8 1 3 6 2 4 7
pass 1 5 1 3 6 2 4 7 8
pass 2 1 3 5 2 4 6 7 8
pass 3 1 3 2 4 5 6 7 8
pass 4 1 2 3 4 5 6 7 8
pass 5 1 2 3 4 5 6 7 8
22 / 40
![Page 39: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/39.jpg)
©Yu Chen
Demo of Bubble Sort
input 5 8 1 3 6 2 4 7
pass 1 5 1 3 6 2 4 7 8
pass 2 1 3 5 2 4 6 7 8
pass 3 1 3 2 4 5 6 7 8
pass 4 1 2 3 4 5 6 7 8
pass 5 1 2 3 4 5 6 7 8
22 / 40
![Page 40: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/40.jpg)
©Yu Chen
Demo of Bubble Sort
input 5 8 1 3 6 2 4 7
pass 1 5 1 3 6 2 4 7 8
pass 2 1 3 5 2 4 6 7 8
pass 3 1 3 2 4 5 6 7 8
pass 4 1 2 3 4 5 6 7 8
pass 5 1 2 3 4 5 6 7 8
22 / 40
![Page 41: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/41.jpg)
©Yu Chen
Demo of Bubble Sort
input 5 8 1 3 6 2 4 7
pass 1 5 1 3 6 2 4 7 8
pass 2 1 3 5 2 4 6 7 8
pass 3 1 3 2 4 5 6 7 8
pass 4 1 2 3 4 5 6 7 8
pass 5 1 2 3 4 5 6 7 8
22 / 40
![Page 42: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/42.jpg)
©Yu Chen
Analysis of Bubble Sort
Complexity analysisworst-case: O(n2) comparison and swapbest-case: O(n) comparison and O(1) swapaverage-case: O(n2) comparison and swap
Advantages. simple and stableDisadvantages. inefficient, only for education purpose
23 / 40
![Page 43: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/43.jpg)
©Yu Chen
Analysis of Bubble Sort
Complexity analysisworst-case: O(n2) comparison and swapbest-case: O(n) comparison and O(1) swapaverage-case: O(n2) comparison and swap
Advantages. simple and stableDisadvantages. inefficient, only for education purpose
23 / 40
![Page 44: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/44.jpg)
©Yu Chen
Quick Sort
Quicksort is a divide-and-conquer algorithm:1 Pick an element, called a pivot, from the array.2 Partitioning: reorder the array to a low sub-array (values
smaller than the pivot) and a high sub-array (values largerthan the pivot), equal values can go either way. After thispartitioning, the pivot is in its final position.
Recursively apply the above steps to the sub-arrays.
Figure: Tony Hoare
invent in 1959 in Moscow State UniversitySoviet Union, where he studied machinetranslation under Andrey KolmogorovMost significant work: Quicksort andQuickselect, Hoare logic, CommunicatingSequential Processes (CSP) for concurrentprocesses
24 / 40
![Page 45: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/45.jpg)
©Yu Chen
Quick Sort
Quicksort is a divide-and-conquer algorithm:1 Pick an element, called a pivot, from the array.2 Partitioning: reorder the array to a low sub-array (values
smaller than the pivot) and a high sub-array (values largerthan the pivot), equal values can go either way. After thispartitioning, the pivot is in its final position.
Recursively apply the above steps to the sub-arrays.
Figure: Tony Hoare
invent in 1959 in Moscow State UniversitySoviet Union, where he studied machinetranslation under Andrey KolmogorovMost significant work: Quicksort andQuickselect, Hoare logic, CommunicatingSequential Processes (CSP) for concurrentprocesses
24 / 40
![Page 46: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/46.jpg)
©Yu Chen
One Recursive Round of Quick Sort
input 5 8 1 3 6 2 4 7
1st swap 5 4 1 3 6 2 8 7
2nd swap 5 4 1 3 2 6 8 7
cross happens
partition 2 4 1 3 5 6 8 7
subproblem 2 4 1 3 5 6 8 7
25 / 40
![Page 47: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/47.jpg)
©Yu Chen
One Recursive Round of Quick Sort
input 5 8 1 3 6 2 4 7
1st swap 5 4 1 3 6 2 8 7
2nd swap 5 4 1 3 2 6 8 7
cross happens
partition 2 4 1 3 5 6 8 7
subproblem 2 4 1 3 5 6 8 7
25 / 40
![Page 48: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/48.jpg)
©Yu Chen
One Recursive Round of Quick Sort
input 5 8 1 3 6 2 4 7
1st swap 5 4 1 3 6 2 8 7
2nd swap 5 4 1 3 2 6 8 7
cross happens
partition 2 4 1 3 5 6 8 7
subproblem 2 4 1 3 5 6 8 7
25 / 40
![Page 49: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/49.jpg)
©Yu Chen
One Recursive Round of Quick Sort
input 5 8 1 3 6 2 4 7
1st swap 5 4 1 3 6 2 8 7
2nd swap 5 4 1 3 2 6 8 7
cross happens
partition 2 4 1 3 5 6 8 7
subproblem 2 4 1 3 5 6 8 7
25 / 40
![Page 50: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/50.jpg)
©Yu Chen
One Recursive Round of Quick Sort
input 5 8 1 3 6 2 4 7
1st swap 5 4 1 3 6 2 8 7
2nd swap 5 4 1 3 2 6 8 7
cross happens
partition 2 4 1 3 5 6 8 7
subproblem 2 4 1 3 5 6 8 7
25 / 40
![Page 51: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/51.jpg)
©Yu Chen
Analysis of Quick Sort
Complexity analysis
worst-case: O(n2) comparison and swap (think about when?)already sorted arrays
best-case: O(n logn) comparison and O(1) swapaverage-case: O(n logn) comparison and swap
Advantagesquick: gained widespread adoption: e.g., (1) in Unix as thedefault library sort subroutine; (2) it lent its name to the Cstandard library subroutine qsort; (3) in the referenceimplementation of Java.
Propertynon-stablepivot-choice affect performance
26 / 40
![Page 52: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/52.jpg)
©Yu Chen
Analysis of Quick Sort
Complexity analysisworst-case: O(n2) comparison and swap (think about when?)
already sorted arraysbest-case: O(n logn) comparison and O(1) swapaverage-case: O(n logn) comparison and swap
Advantagesquick: gained widespread adoption: e.g., (1) in Unix as thedefault library sort subroutine; (2) it lent its name to the Cstandard library subroutine qsort; (3) in the referenceimplementation of Java.
Propertynon-stablepivot-choice affect performance
26 / 40
![Page 53: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/53.jpg)
©Yu Chen
Analysis of Quick Sort
Complexity analysisworst-case: O(n2) comparison and swap (think about when?)
already sorted arrays
best-case: O(n logn) comparison and O(1) swapaverage-case: O(n logn) comparison and swap
Advantagesquick: gained widespread adoption: e.g., (1) in Unix as thedefault library sort subroutine; (2) it lent its name to the Cstandard library subroutine qsort; (3) in the referenceimplementation of Java.
Propertynon-stablepivot-choice affect performance
26 / 40
![Page 54: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/54.jpg)
©Yu Chen
Analysis of Quick Sort
Complexity analysisworst-case: O(n2) comparison and swap (think about when?)
already sorted arraysbest-case: O(n logn) comparison and O(1) swapaverage-case: O(n logn) comparison and swap
Advantagesquick: gained widespread adoption: e.g., (1) in Unix as thedefault library sort subroutine; (2) it lent its name to the Cstandard library subroutine qsort; (3) in the referenceimplementation of Java.
Propertynon-stablepivot-choice affect performance
26 / 40
![Page 55: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/55.jpg)
©Yu Chen
Analysis of Quick Sort
Complexity analysisworst-case: O(n2) comparison and swap (think about when?)
already sorted arraysbest-case: O(n logn) comparison and O(1) swapaverage-case: O(n logn) comparison and swap
Advantagesquick: gained widespread adoption: e.g., (1) in Unix as thedefault library sort subroutine; (2) it lent its name to the Cstandard library subroutine qsort; (3) in the referenceimplementation of Java.
Propertynon-stablepivot-choice affect performance
26 / 40
![Page 56: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/56.jpg)
©Yu Chen
Merge Sort
Merge sort is a divide-and-conquer algorithm:divide the unsorted list into n sublists, each containing oneelement (a list of one element is considered sorted).repeatedly merge sublists to produce new sorted sublists untilthere is only one sublist remaining. (this will be the sortedlist.)
canonical case n = 2
Figure: John von Neumann
27 / 40
![Page 57: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/57.jpg)
©Yu Chen
Demo of Merge Sort
input 5 8 1 3 6 2 4 7
split 5 8 1 3 6 2 4 7
recursivesorting 1 3 5 8 2 4 6 7
merge 1 2 3 4 5 6 7 8
28 / 40
![Page 58: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/58.jpg)
©Yu Chen
Demo of Merge Sort
input 5 8 1 3 6 2 4 7
split 5 8 1 3 6 2 4 7
recursivesorting 1 3 5 8 2 4 6 7
merge 1 2 3 4 5 6 7 8
28 / 40
![Page 59: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/59.jpg)
©Yu Chen
Demo of Merge Sort
input 5 8 1 3 6 2 4 7
split 5 8 1 3 6 2 4 7
recursivesorting 1 3 5 8 2 4 6 7
merge 1 2 3 4 5 6 7 8
28 / 40
![Page 60: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/60.jpg)
©Yu Chen
Demo of Merge Sort
input 5 8 1 3 6 2 4 7
split 5 8 1 3 6 2 4 7
recursivesorting 1 3 5 8 2 4 6 7
merge 1 2 3 4 5 6 7 8
28 / 40
![Page 61: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/61.jpg)
©Yu Chen
Analysis of Merge Sort
Complexity analysisworst-case, best-case, average-case: O(n logn) comparisonspace: O(n) total with O(n) auxiliary
Advantagesquick: (1) Linux kernel for linked list; (2) Android platform;(3) python and Java
Propertystable
29 / 40
![Page 62: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/62.jpg)
©Yu Chen
Comparisons Among Sorting Algorithms
Algorithm worst case best case average case stableinsertion sort O(n2) O(n) O(n2) yesbubble sort O(n2) O(n) O(n2) yesquick sort O(n2) O(n logn) O(n logn) nomerge sort O(n logn) O(n logn) O(n logn) yes
30 / 40
![Page 63: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/63.jpg)
©Yu Chen
Complexity Analysis
Which algorithm performs best?How to evaluate it?Can we find better sortingalgorithm?
n2
n logn
?
insertion sortbubble sortquick sort
merge sort
better lower bound
31 / 40
![Page 64: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/64.jpg)
©Yu Chen
1 A Taste of Algorithm DesignSingle Machine Scheduling ProblemReturn on Investment (ROI) Problem
2 A Taste of Algorithm AnalysisSorting Problem
3 Complexity TheoryTravelling Salesman ProblemKnapsack Problem
32 / 40
![Page 65: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/65.jpg)
©Yu Chen
Travelling Salesman Problem (TSP)
Problem. Given n cities and the distances between each pair ofcities, what is the shortest possible route that visits each city andreturns to the origin city?
c3
c1
c2 c4
5
6 310 9
9
33 / 40
![Page 66: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/66.jpg)
©Yu Chen
Formalization
Input. Finite set of cities C = c1, c2, . . . , cn, distanced(ci, cj) = d(cj , ci) ∈ Z+, 1 ≤ i < j ≤ n.Solution. A permutation of 1, 2, . . . , n, a.k.a. k1, k2, . . . , kn suchthat:
min
n−1∑i=1
d(cki , cki+1) + d(ckn , ck1)
Think: can the objective function be simpler?use modular n expression – 0, 1, . . . , n− 1
34 / 40
![Page 67: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/67.jpg)
©Yu Chen
Formalization
Input. Finite set of cities C = c1, c2, . . . , cn, distanced(ci, cj) = d(cj , ci) ∈ Z+, 1 ≤ i < j ≤ n.Solution. A permutation of 1, 2, . . . , n, a.k.a. k1, k2, . . . , kn suchthat:
min
n−1∑i=1
d(cki , cki+1) + d(ckn , ck1)
Think: can the objective function be simpler?use modular n expression – 0, 1, . . . , n− 1
34 / 40
![Page 68: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/68.jpg)
©Yu Chen
About TSP
TSP (first formulated in 1930) is the most intensively studiedproblems NP-hard problem in combinatorial optimization andtheoretical computer science.
TSP is used as a benchmark for many optimization methods.Though TSP is computationally difficult, many heuristics andexact algorithms are known.
some instances with tens of thousands of cities can be solvedcompletelyeven problems with millions of cities can be approximatedwithin a small fraction of 1%.
TSP has several applicationsin its purest formulation: planning, logistics, and themanufacture of microchipsslightly modified: DNA sequencing
35 / 40
![Page 69: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/69.jpg)
©Yu Chen
About TSP
TSP (first formulated in 1930) is the most intensively studiedproblems NP-hard problem in combinatorial optimization andtheoretical computer science.TSP is used as a benchmark for many optimization methods.Though TSP is computationally difficult, many heuristics andexact algorithms are known.
some instances with tens of thousands of cities can be solvedcompletelyeven problems with millions of cities can be approximatedwithin a small fraction of 1%.
TSP has several applicationsin its purest formulation: planning, logistics, and themanufacture of microchipsslightly modified: DNA sequencing
35 / 40
![Page 70: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/70.jpg)
©Yu Chen
About TSP
TSP (first formulated in 1930) is the most intensively studiedproblems NP-hard problem in combinatorial optimization andtheoretical computer science.TSP is used as a benchmark for many optimization methods.Though TSP is computationally difficult, many heuristics andexact algorithms are known.
some instances with tens of thousands of cities can be solvedcompletelyeven problems with millions of cities can be approximatedwithin a small fraction of 1%.
TSP has several applicationsin its purest formulation: planning, logistics, and themanufacture of microchipsslightly modified: DNA sequencing
35 / 40
![Page 71: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/71.jpg)
©Yu Chen
Knapsack Problem
Given n items, each with a weight and a value, determine thenumber of each item to include in a collection so that the totalweight is less than or equal to a given limit B and the total valueis as large as possible.
name: someone who is constrained by a fixed-size knapsackand must fill it with the most valuable items0-1 variant: for each item, include or not
36 / 40
![Page 72: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/72.jpg)
©Yu Chen
Formalization
Solution. vector ⟨x1, x2, . . . , xn⟩ over 0, 1n, xi = 1 iff item i isincluded
objective function: maxn∑
i=1
vixi
constraint:n∑
i=1
wixi ≤ B, xi ∈ 0, 1, i ∈ [n]
37 / 40
![Page 73: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/73.jpg)
©Yu Chen
About Knapsack Problem
Knapsack (since 1897) often arises in resource allocation where thedecision makers have to choose from a set of non-divisible projectsor tasks under a fixed budget or time constraint, respectively. It isNP-complete problem.
Hardness of the knapsack problem depends on the form of theinput.
one theme in research is to identify “hard” instances: identifywhat properties of instances might make them more amenablethan their worst-case NP-complete hardness suggestsapplication in public key cryptography systems, e.g., theMerkle-Hellman knapsack cryptosystem.
The basic problem is a one-dimensional (constraint) knapsackproblem
a multiple constrained problem could consider both the weightand volume of knapsack
38 / 40
![Page 74: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/74.jpg)
©Yu Chen
About Knapsack Problem
Knapsack (since 1897) often arises in resource allocation where thedecision makers have to choose from a set of non-divisible projectsor tasks under a fixed budget or time constraint, respectively. It isNP-complete problem.Hardness of the knapsack problem depends on the form of theinput.
one theme in research is to identify “hard” instances: identifywhat properties of instances might make them more amenablethan their worst-case NP-complete hardness suggestsapplication in public key cryptography systems, e.g., theMerkle-Hellman knapsack cryptosystem.
The basic problem is a one-dimensional (constraint) knapsackproblem
a multiple constrained problem could consider both the weightand volume of knapsack
38 / 40
![Page 75: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/75.jpg)
©Yu Chen
About Knapsack Problem
Knapsack (since 1897) often arises in resource allocation where thedecision makers have to choose from a set of non-divisible projectsor tasks under a fixed budget or time constraint, respectively. It isNP-complete problem.Hardness of the knapsack problem depends on the form of theinput.
one theme in research is to identify “hard” instances: identifywhat properties of instances might make them more amenablethan their worst-case NP-complete hardness suggestsapplication in public key cryptography systems, e.g., theMerkle-Hellman knapsack cryptosystem.
The basic problem is a one-dimensional (constraint) knapsackproblem
a multiple constrained problem could consider both the weightand volume of knapsack
38 / 40
![Page 76: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/76.jpg)
©Yu Chen
NP-hard Problem
NP-hardness (non-deterministic polynomial-time hardness) is aclass of problems that are
informally “at least as hard as the hardest problems in NP”an efficient algorithm for a NP-hard problem implies efficientalgorithms for all NP problem
No “efficient” algorithms found yet:complexity of known algorithm are at least exponentialfunction on input sizeno one can prove the “non-existence” of efficient algorithmsfor those problems
Thousands of NP-hard problems, widely spreads in all areas.
39 / 40
![Page 77: Design and Analysis of Algorithms · Design and Analysis of Algorithms Introduction to Algorithms 1 A Taste of Algorithm Design Single Machine Scheduling Problem Return on Investment](https://reader030.fdocuments.net/reader030/viewer/2022040113/5ed82d570fa3e705ec0dfaaf/html5/thumbnails/77.jpg)
©Yu Chen
Summary
The significance of algorithmAlgorithm evaluation criteria
Efficient: low time complexity & space complexityCorrect: yield optimal solution for all instance
The Scope of AlgorithmDesign technique (exemplified by SMSP and ROIP)
modeling ; find an algorithmproof ; prove the correctness
Complexity analysis (exemplified by sorting problem)calculate the number of basic operations
Complexity theory (TSP and Knapsack)measuring of complexitycomplexity classificationlower bounds
40 / 40