14 algorithm efficiency - GitHub Pages
Transcript of 14 algorithm efficiency - GitHub Pages
![Page 1: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/1.jpg)
Algorithm Efficiency (More formally)
Tiziana Ligorio!1
![Page 2: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/2.jpg)
Today’s Plan
Algorithm Efficiency
!2
![Page 3: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/3.jpg)
What is CSCI 235?Programming => Software Analysis and DesignExpected professional and responsible attitude
Think like a Computer Scientist:
Design and maintain complex programs
Software Engineering, Abstraction, OOP
Design and represent data and its management
Abstract Data Types
Implement data representation and operations
Data Structures
Algorithms
Analyze Algorithms and their Efficiency
!3
![Page 4: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/4.jpg)
Algorithm Efficiency
!4
![Page 5: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/5.jpg)
Scenario 1
You are using an application and suddenly it stalls… whatever it is doing it’s taking way too long…
!5
![Page 6: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/6.jpg)
Scenario 1
You are using an application and suddenly it stalls… whatever it is doing it’s taking way too long…
how “long” does that have to be for you to become ridiculously frustrated?
!6
![Page 7: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/7.jpg)
Scenario 1
You are using an application and suddenly it stalls… whatever it is doing it’s taking way too long…
how “long” does that have to be for you to become ridiculously frustrated?
… probably not that long
!7
![Page 8: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/8.jpg)
!8
Scenario 2At your next super high-end job with the company/research-center of your dreams you are given a very difficult problem to solve.
You work hard on it, find a solution, code it up and it works!!!! Proudly you present it the next day but…
Given some new (large) input it keeps stalling…
![Page 9: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/9.jpg)
Scenario 2At your next super high-end job with the company/research-center of your dreams you are given a very difficult problem to solve.
You work hard on it, find a solution, code it up and it works!!!! Proudly you present it the next day but…
Given some new (large) input it keeps stalling…
Well… sorry but your solution is no good!!!
!9
![Page 10: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/10.jpg)
You need to have a means to estimate/predict the efficiency of your algorithms on unknown input.
!10
![Page 11: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/11.jpg)
What is a good solution?
How can we compare solutions to a problem? (Algorithms)
!11
![Page 12: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/12.jpg)
What is a good solution?
!12
Correct
If it’s not correct it is not a solution at all
![Page 13: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/13.jpg)
What is a good solution?
!13
Correct Efficient
Time Space
![Page 14: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/14.jpg)
What is a good solution?
!14
Correct Efficient
Time SpaceWe are going to focus on time
![Page 15: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/15.jpg)
How can we measure time efficiency?
!15
![Page 16: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/16.jpg)
How can we measure time efficiency?
!16
Runtime?
![Page 17: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/17.jpg)
Problems with actual runtime for comparison
What computer are you using? Runtime is highly sensitive to hardware
!17
![Page 18: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/18.jpg)
Problems with actual runtime for comparison
What computer are you using? Runtime is highly sensitive to hardware
What implementation are you using? Implementation details may affect runtime but are not reflective of algorithm efficiency
!18
![Page 19: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/19.jpg)
How should we measure execution time?
!19
![Page 20: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/20.jpg)
How should we measure execution time?
!20
Number of “steps” or “operations” as a function of the size of the input
Constant
Variable
![Page 21: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/21.jpg)
!21
template<typename ItemType>void List<ItemType>::traverse(){ for(Node<ItemType>* ptr = first; ptr != nullptr; ptr = ptr->getNext()) { cout << ptr->getItem() << endl; }}
![Page 22: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/22.jpg)
template<typename ItemType>void List<ItemType>::traverse(){ for(Node<ItemType>* ptr = first; ptr != nullptr; ptr = ptr->getNext()) { cout << ptr->getItem() << endl; }}
!22
What are the operations?Let n be the number of nodes
![Page 23: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/23.jpg)
!23
1 node instantiation and assignment upon entering the loop
What are the operations?Let n be the number of nodes
template<typename ItemType>void List<ItemType>::traverse(){ for(Node<ItemType>* ptr = first; ptr != nullptr; ptr = ptr->getNext()) { cout << ptr->getItem() << endl; }}
![Page 24: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/24.jpg)
!24
1 node instantiation and assignment upon entering the loop
pointer comparison
What are the operations?Let n be the number of nodes
template<typename ItemType>void List<ItemType>::traverse(){ for(Node<ItemType>* ptr = first; ptr != nullptr; ptr = ptr->getNext()) { cout << ptr->getItem() << endl; }}
![Page 25: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/25.jpg)
!25
1 node instantiation and assignment upon entering the loop
pointer comparison
call to getNext()
What are the operations?Let n be the number of nodes
template<typename ItemType>void List<ItemType>::traverse(){ for(Node<ItemType>* ptr = first; ptr != nullptr; ptr = ptr->getNext()) { cout << ptr->getItem() << endl; }}
![Page 26: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/26.jpg)
!26
1 node instantiation and assignment upon entering the loop
pointer comparison
call to getNext()
pointer assignment
What are the operations?Let n be the number of nodes
template<typename ItemType>void List<ItemType>::traverse(){ for(Node<ItemType>* ptr = first; ptr != nullptr; ptr = ptr->getNext()) { cout << ptr->getItem() << endl; }}
![Page 27: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/27.jpg)
!27
1 node instantiation and assignment upon entering the loop
pointer comparison
call to getNext()
pointer assignment
call to getItem()
write to the console
What are the operations?Let n be the number of nodes
template<typename ItemType>void List<ItemType>::traverse(){ for(Node<ItemType>* ptr = first; ptr != nullptr; ptr = ptr->getNext()) { cout << ptr->getItem() << endl; }}
![Page 28: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/28.jpg)
!28
What are the operations?Let n be the number of nodes
K0
K1K2
K3K4
1 node instantiation and assignment upon entering the loop
pointer comparison
call to getNext()
pointer assignment
call to getItem()
write to the consoleK5
template<typename ItemType>void List<ItemType>::traverse(){ for(Node<ItemType>* ptr = first; ptr != nullptr; ptr = ptr->getNext()) { cout << ptr->getItem() << endl; }}
![Page 29: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/29.jpg)
!29
What are the operations?Let n be the number of nodes
Operations = K0 + n(K1+K2+K3+K4+K5)
1 node instantiation and assignment upon entering the loop
pointer comparison
call to getNext()
pointer assignment
call to getItem()
write to the console
K0
K1K2
K3K4
K5
template<typename ItemType>void List<ItemType>::traverse(){ for(Node<ItemType>* ptr = first; ptr != nullptr; ptr = ptr->getNext()) { cout << ptr->getItem() << endl; }}
![Page 30: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/30.jpg)
!30
What are the operations?Let n be the number of nodes
Operations = K0 + nK6
1 node instantiation and assignment upon entering the loop
pointer comparison
call to getNext()
pointer assignment
call to getItem()
write to the console
K0
K1K2
K3K4
K5
template<typename ItemType>void List<ItemType>::traverse(){ for(Node<ItemType>* ptr = first; ptr != nullptr; ptr = ptr->getNext()) { cout << ptr->getItem() << endl; }}
![Page 31: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/31.jpg)
Lecture Activity
bool linearSearch(const string& str, char ch){
for (int i = 0; i < str.length(); i++) { if (str[i] == ch) { return true; } } return false;}
!31
Identify the steps and write down an expression for execution time
![Page 32: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/32.jpg)
Lecture Activity
bool linearSearch(const string& str, char ch){
for (int i = 0; i < str.length(); i++) { if (str[i] == ch) { return true; } } return false;}
!32
Identify the steps and write down an expression for execution time
Was this tricky?
![Page 33: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/33.jpg)
bool linearSearch(const string& str, char ch){
// 1 int assignment upon entering loop for (int i = 0; i < str.length(); i++) { // call to length() and increment if (str[i] == ch) { // Comparisons return true; //return operation, maybe } } return false; //return operation, maybe}
!33
n here is the length of the string
![Page 34: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/34.jpg)
bool linearSearch(const string& str, char ch){
// 1 int assignment upon entering loop for (int i = 0; i < str.length(); i++) { // call to length() and increment if (str[i] == ch) { // Comparisons return true; //return operation, maybe } } return false; //return operation, maybe}
!34
n here is the length of the string
Maybe stop in the middle
Maybe stop at end of loop
![Page 35: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/35.jpg)
bool linearSearch(const string& str, char ch){ for (int i = 0; i < str.length(); i++) { if (str[i] == ch) { return true; } } return false;}
!35
Execution completes in at most: k0n+k1 operations
In the WORST CASE
n here is the length of the string
![Page 36: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/36.jpg)
Types of Analysis Best case analysis: running time under best input (e.g., in linear search item we are looking for is the first ) - not reflective of overall performance)
Average case analysis: assumes equal probability of input (usually not the case)
Expected case analysis: assumes probability of occurrence of input is known or can be estimated, and if it were possible may be too expensive
Worst case analysis: running time under worst input, gives upper bound, it can’t get worse, good for sleeping well at night!
!36
![Page 37: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/37.jpg)
bool linearSearch(const string& str, char ch){ for (int i = 0; i < str.length(); i++) { if (str[i] == ch) { return true; } } return false;}
!37
Execution completes in at most: k0n+k1 operations
Some constant number of operations repeated
inside the loop
Some constant number of operations performed
outside the loop
n here is the length of the string
![Page 38: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/38.jpg)
bool linearSearch(const string& str, char ch){ for (int i = 0; i < str.length(); i++) { if (str[i] == ch) { return true; } } return false;}
!38
Execution completes in at most: k0n+k1 operations
Some constant number of operations repeated
inside the loop
Some constant number of operations performed
outside the loop
The number of times the loop is repeated, i.e. the size of str
n here is the length of the string
![Page 39: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/39.jpg)
Observation
Don’t need to explicitly compute the constants ki
4n + 1000
n + 137
Dominant term is sufficient to explain overall behavior (in this case linear)
!39
![Page 40: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/40.jpg)
Big-O Notation
Ignores everything except dominant term
Examples: T(n) = 4n + 4 = O(n) T(n) = 164n + 35 = O(n) T(n) = n2 + 35n + 5 = O(n2) T(n) = 2n3 + 98n2 + 210 = O(n3) T(n) = 2n + 5 = O(2n)
!40
Notation: describes the overall behavior
![Page 41: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/41.jpg)
Big-O Notation
Ignores everything except dominant term
Examples: T(n) = 4n + 4 = O(n) T(n) = 164n + 35 = O(n) T(n) = n2 + 35n + 5 = O(n2) T(n) = 2n3 + 98n2 + 210 = O(n3) T(n) = 2n + 5 = O(2n)
!41
T(n) is the running time
n is the size of the input
Notation: describes the overall behavior
![Page 42: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/42.jpg)
Big-O Notation
Ignores everything except dominant term
Examples: T(n) = 4n + 4 = O(n) T(n) = 164n + 35 = O(n) T(n) = n2 + 35n + 5 = O(n2) T(n) = 2n3 + 98n2 + 210 = O(n3) T(n) = 2n + 5 = O(2n)
!42
Big-O describes the overall behavior
Let T(n) be the running time of an algorithm measured as number of
operations given input of size n. T(n) is O(f(n))
if it grows no faster than f(n)
![Page 43: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/43.jpg)
Big-O Notation
Ignores everything except dominant term
Examples: T(n) = 4n + 4 = O(n) T(n) = 164n + 35 = O(n) T(n) = n2 + 35n + 5 = O(n2) T(n) = 2n3 + 98n2 + 210 = O(n3) T(n) = 2n + 5 = O(2n)
!43
Big-O describes the overall behavior
But 164n+35 > n
Let T(n) be the running time of an algorithm measured as number of
operations given input of size n. T(n) is O(f(n))
if it grows no faster than f(n)
![Page 44: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/44.jpg)
Big-O Notation
Ignores everything except dominant term
Examples: T(n) = 4n + 4 = O(n) T(n) = 164n + 35 = O(n) T(n) = n2 + 35n + 5 = O(n2) T(n) = 2n3 + 98n2 + 210 = O(n3) T(n) = 2n + 5 = O(2n)
!44
Notation: describes the overall behavior
More formally:T(n) is O(f(n))
if there exist constants k and n0 such that for all n ≥ n0
T(n) ≤ kf(n)
![Page 45: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/45.jpg)
!45
More formally:T(n) is O(f(n))
if there exist constants k and n0 such that for all n ≥ n0,
T(n) ≤ kf(n)
n0
k = 3
T(n) = n2 - 3n + 10 T(n) is O(n2)
For k=3 and n≥1.5
![Page 46: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/46.jpg)
!46
More formally:T(n) is O(f(n))
if there exist constants k and n0 such that for all n ≥ n0,
T(n) ≤ kf(n)
n0
k = 3
T(n) = n2 - 3n + 10 T(n) is O(n2)
For k=3 and n≥1.5
This is why we can look at dominant
term only to explain behavior
![Page 47: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/47.jpg)
Big-O describes the overall growth rate of an algorithms for large n
!47
![Page 48: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/48.jpg)
Proving Big-O Relationship
Apply definition of Big-O to prove that T(n) is O(f(n)) for particular functions T and f
Do so by choosing k and n0 s.t. for all n ≥ n0, T(n) ≤ kf(n)
!48
![Page 49: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/49.jpg)
Example: Suppose T(n) = (n+1)2 We can say that T(n) is O(n2)
To prove it must find k and n0 s.t. for all n ≥ n0 , (n+1)2 ≤ kn2
Proving Big-O Relationship
!49
![Page 50: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/50.jpg)
Example: Suppose T(n) = (n+1)2 We can say that T(n) is O(n2)
To prove it must find k and n0 s.t. for all n ≥ n0, (n+1)2 ≤ kn2
Expand (n+1)2 = n2 + 2n + 1Observe that, as long as n ≥1, n ≤ n2 and 1 ≤ n2
Proving Big-O Relationship
!50
![Page 51: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/51.jpg)
f(n)
Proving Big-O Relationship
Example: Suppose T(n) = (n+1)2 We can say that T(n) is O(n2)
To prove it must find k and n0 s.t. for all n ≥ n0, (n+1)2 ≤ kn2
Expand (n+1)2 = n2 + 2n + 1Observe that, as long as n ≥1, n ≤ n2 and 1 ≤ n2
Thus if we choose n0 =1 and k = 4 we haven2 + 2n + 1 ≤ n2 + 2n2 + n2 = 4n2
!51T(n) k
![Page 52: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/52.jpg)
Proving Big-O RelationshipExample: Suppose T(n) = (n+1)2 We can say that T(n) is O(n2)
To prove it must find k and n0 s.t. for all n ≥ n0, (n+1)2 ≤ kn2
Expand (n+1)2 = n2 + 2n + 1 Observe that, as long as n ≥1, n ≤ n2 and 1 ≤ n2
Thus if we choose n0 =1 and k = 4 we haven2 + 2n + 1 ≤ n2 + 2n2 + n2 = 4n2
!52
![Page 53: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/53.jpg)
Proving Big-O Relationship
Not Unique: Could also choose n0 =3 and k = 2 because(n+1)2 ≤ 2n2 for all n ≥ 3
For proof one is enough
!53
![Page 54: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/54.jpg)
A visual comparison of growth rates
!54
![Page 55: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/55.jpg)
!55
0
2
4
6
8
10
12
14
16
Growth Rates, Part One
O(1)
O(log n)
O(n)
![Page 56: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/56.jpg)
!56
0
50
100
150
200
250
Growth Rates, Part Two
O(n)
O(n log n)
O(n²)
![Page 57: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/57.jpg)
!57
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
Growth Rates, Part Three
O(n²)
O(n³)
O(2ⁿ)
![Page 58: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/58.jpg)
!58
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
To Give You A Better Sense...
O(1)
O(log n)
O(n)
O(n log n)
O(n²)
O(n³)
O(2ⁿ)
![Page 59: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/59.jpg)
Tight is more meaningful
If T(n) is O(n)It is also true that T(n) is O(n3)And it is also true that T(n) is O(2n) But what does it mean???
The closest Big-O is the most descriptive of the overall worst-case behavior
!59
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
To Give You A Better Sense...
O(1)
O(log n)
O(n)
O(n log n)
O(n²)
O(n³)
O(2ⁿ)
![Page 60: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/60.jpg)
Big-O: upper bound T(n) is O(f(n)) if there exist constants k and n0 such that for all n ≥ n0 T(n) ≤ k f(n) Grows no faster than f(n)
Tightening the bounds
!60
![Page 61: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/61.jpg)
Tightening the boundsBig-O: upper bound T(n) is O(f(n)) if there exist constants k and n0 such that for all n ≥ n0 T(n) ≤ k f(n) Grows no faster than f(n)
Omega: lower bound T(n) is Ω(f(n)) if there exist constants k and n0 such that for all n ≥ n0 T(n) ≥ k f(n) Grows at least as fast as f(n)
!61
![Page 62: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/62.jpg)
Tightening the boundsTheta: tight bound T(n) is Θ(f(n)) Grows at the same rate as f(n) : iff both T(n) is O(f(n)) and Ω(f(n))
!62
![Page 63: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/63.jpg)
A numerical comparison of growth rates
!63
![Page 64: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/64.jpg)
!64
10 100 1,000 10,000 100,000 1,000,000
1 1 1 1 1 1 1
log2n 3 6 9 13 16 19
n 10 102 103 104 105 106
n * log2n 30 664 9,965 105 106 107
n2 102 104 106 108 1010 1012
n3 103 106 109 1012 1015 1018
2n 103 1030 10301 103,010 1030,103 10301,030
nf(n)
![Page 65: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/65.jpg)
What does Big-O describe?
“Long term” behavior of a function If algorithm A has runtime O(n) and algorithm B has runtime O(n2), for large inputs A will always be faster. If algorithm A has runtime O(n), doubling the size of the input will double the runtime
!65
Compare behavior of 2 algorithms
Analyze algorithm behavior with growing input
![Page 66: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/66.jpg)
What can’t Big-O describe?
The actual runtime of an algorithm 10100n = O(n) 10-100n = O(n)
How an algorithm behaves on small input n3 = O(n3) 106 = O(1)
!66
![Page 67: 14 algorithm efficiency - GitHub Pages](https://reader031.fdocuments.net/reader031/viewer/2022012018/61dade2c0f302c27dd442bdb/html5/thumbnails/67.jpg)
To summarize Big-O
It is a means of describing the growth rate of a function
It ignores all but the dominant term
It ignores constants
Allows for quantitative ranking of algorithms
Allows for quantitative reasoning about algorithms
!67