Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when...
-
Upload
ross-patrick -
Category
Documents
-
view
234 -
download
6
Transcript of Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when...
![Page 1: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/1.jpg)
Algorithm Analysis
![Page 2: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/2.jpg)
Algorithm
An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipesdirections for putting together a bicycleprograms
![Page 3: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/3.jpg)
Algorithm Analysis
The process by which we determine the amount of resources used by the algorithm
Resources usually measured timememory space
We will concentrate on run time of algorithms
![Page 4: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/4.jpg)
Common functions encountered
Some commonly encountered functions are: ,linear ,quadratic ,cubic
The next slide shows the graphs of these functions
)log(NN2N
3N
N
![Page 5: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/5.jpg)
Growth Rates
Growth rates
-20
0
20
40
60
80
100
120
0 20 40 60 80 100 120
N
Series1
Series2
Series3
Series4
![Page 6: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/6.jpg)
Running Time T(N)Example 1
Problem: Given an array of N elements, find the smallest.
Solution:1. key = Integer.Mininum_Value
2. for( int i=0 ; i< inarray.length ; i++ )
3. if ( inarray[i] < key ) key = inarray[i];
![Page 7: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/7.jpg)
Running Time T(N)Example 2
Suppose we wish to download a file 2 seconds for setting up the connectiondownloading proceeds at 2.8 K/sec
If the downloaded file is N kilobytes, the time to download is T(N) = N/2.8 + 280K file takes about 31 seconds160K file takes about 59 seconds
This is a linear algorithm
![Page 8: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/8.jpg)
Functions in Increasing Order
Function Name
c
log N
log2 N
N
N log N
N2
N3
2N
Constant
Logarithmic
Log-squared
Linear
N log N
Quadratic
Cubic
Exponential
![Page 9: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/9.jpg)
Measurement of T(N)
Place your timer before/after the algorithm Run the algorithm multiple times Take the average of the total time
1. time = System.currentTimeMillis();
2. for (int i = 0; i<num_iteration ; i++)
3. // Your algorithm comes here
4. newtime = System.currentTimeMillis();
5. output (newTime-time)/num_iteration
![Page 10: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/10.jpg)
Big-O notation
We use big-O to specify growth rates of algorithms linear functions are specified as O(N)quadratic functions as O(N2)
![Page 11: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/11.jpg)
Formal Definition of Big-O
Definition: T(N) is O(F(N)) if there are positive constants c and N0 such that T(N)cF(N) whenever N N0 .
Usually T(N) is a complicated function involving N.
We choose F(N) as simple as possible.
![Page 12: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/12.jpg)
Examples of Big-O
Find equivalent Big-Oh functions for the following runtime T(N).
T1(N)=N2
T2(N)=20*N
T3(N)=N+1000*N2
T4(N)=N+60 N3 -1000*N2
T5(N)=N*log(N)+N2
![Page 13: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/13.jpg)
T6(N)=50*N*log(N) + N
T7(N)=N + 60 N3 - N*log(N)
T8(N)= 20*N +100*N2
T9(N)=90*N2 + 60 N3
Examples of Big-O
![Page 14: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/14.jpg)
Algorithm Analysis
A1: Minimum element in an arrayGiven an array of N elements, find the
smallest.
A2: Closest points in the planeGiven N points in the plane, find the pair of
points that are closest together.
A3: Collinear points in the planeGiven N points in the plane, determine if
any three are on the same line.
![Page 15: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/15.jpg)
A1: Minimal Element in an Array
Algorithm Initialize min as the first element Scan thru the other N-1 elements, updating min as
appropriate
There is a fixed amount of work for each element of the array
The running time is O(N)
![Page 16: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/16.jpg)
A2: Closest Point in the Plane
AlgorithmCalculate the distance between each pair of
points [there are N(N-1)/2 pairs of points] find the minimum of these distances
Finding the minimum of N(N-1)/2 distances by this algorithm is O(N2)
There is a O(N log N) algorithm to solve this problem, and one thought to be O(N).
![Page 17: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/17.jpg)
A3: Collinear Points in the Plane
AlgorithmEnumerate all triples of the N points. There
are N(N-1)(N-2)/6 of these triples.check to see if the three points are on the
same lineThis is a O(N3) algorithmThere is a clever quadratic algorithm for
solving this problem
![Page 18: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/18.jpg)
Maximum Contiguous Subsequence Sum Problem
Given (possibly negative) integers A1, A2,…, AN , find (and identify the subsequence corresponding to) the maximum value Ai + Ai+1 +…+ Ak .
The maximum contiguous subsequence sum is zero if all the integers are negative
![Page 19: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/19.jpg)
Examples
With input of [-2, 11, -4, 13, -5, 2], then the maximum sum is 20 using items at positions 2 through 4.
With input of [ 1, -3, 4, -2, -1, 6], then the maximum sum is 7 using the last four items.
![Page 20: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/20.jpg)
Obvious O(N3) Algorithm
maxSum = 0;for (i=0; i < N; i++)
for (j=i; j < N; j++){thisSum = 0;for (k=i;k <= j; k++)
thisSum += a[k];if (thisSum > maxSum){
maxSum = thisSum;seqStart = i;seqStop = j;
}}
return maxSum;
![Page 21: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/21.jpg)
Improving the Algorithm
If we can eliminate one of the loops, we will be able to reduce this algorithm to a O(N2) algorithm.
Observe that Ai + Ai+1 +…+ Ak = (Ai + Ai+1 +…+ Ak-1 ) + Ak . The inner loop is used to calculate Ai + Ai+1 +…+ Ak-1 , then this result is thrown away and then Ai + Ai+1 +…+ Ak is calculated (duplicating work)
![Page 22: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/22.jpg)
Improved O(N2) Algorithm
maxSum = 0;for (i=0; i < N; i++){
thisSum = 0;for (j=i; j < N; j++){
thisSum += a[j];if (thisSum > maxSum){
maxSum = thisSum;seqStart = i;seqStop = j;
}}
}return maxSum;
![Page 23: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/23.jpg)
Observations
no maximum contiguous subsequence sum begins with a subsequence which has a negative sum
all subsequences which border on the maximum contiguous subsequence must have negative or zero sums
![Page 24: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/24.jpg)
Improving again
Using these observations, any time the current subsequence sum (thisSum) becomes negative, we can begin a new subsequence sum at the next position since the subsequence we are searching for cannot begin with a subsequence with negative sum.
![Page 25: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/25.jpg)
Linear Algorithm
maxSum = thisSum = 0;for (i=0, j=0; j < N; j++){
thisSum += a[j];if (thisSum > maxSum){
maxSum = thisSum;seqStart = i;seqStop = j;
}else if (thisSum < 0){
i = j+1;thisSum = 0;
}}return maxSum;
![Page 26: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/26.jpg)
Big-O
Definition: T(N) is O(F(N)) if there are positive constants c and N0 such that T(N)cF(N) whenever N N0 .
This means, that for sufficiently large N, T(N) is bounded by some multiple of F(N).
This means that the growth rate of T is no worse than the growth rate of F.
![Page 27: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/27.jpg)
Big- (omega)
Definition: T(N) is (F(N)) if there are positive constants c and N0 such that T(N) cF(N) whenever N N0 .
This means, that for sufficiently large N, T(N) is bounded below by some multiple of F(N).
This means that the growth rate of T is no better than the growth rate of F.
![Page 28: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/28.jpg)
Big- (theta)
Definition: T(N) is (F(N)) if and only if T(N) is O(F(N)) and T(N) is (F(N)).
This means that the growth rate of T is equal to the growth rate of F.
![Page 29: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/29.jpg)
Little-o
Definition: T(N) is o(F(N)) if and only if T(N) is O(F(N)) and T(N) is not (F(N)).
This means that the growth rate of T is strictly better than the growth rate of F
![Page 30: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/30.jpg)
Summary
Mathematical Expression Relative Rates of Growth
T(N) = O(F(N)) Growth of T(N) growth of F(N)
T(N) = (F(N)) Growth of T(N) growth of F(N)
T(N) = (F(N)) Growth of T(N) = growth of F(N)
T(N) = o(F(N)) Growth of T(N) < growth of F(N)
![Page 31: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/31.jpg)
The Logarithm
Definition: For any B, N > 0, logBN = K if and only if BK = N
In Computer Science, base 2 logarithms are used almost exclusively.
The use of the symbol log indicates log2 unless otherwise indicated
![Page 32: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/32.jpg)
Use of logarithms
Bits in a binary numberThe number of bits needed to represent N
consecutive integers is log NRepeated Doubling
Starting with X=1, the number of times needed to double X to reach at least N is log N
![Page 33: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/33.jpg)
Searching
Sequential Search the average number of comparisons made
to find an item in an unordered list of N items is N/2 = O(N)
An unsuccessful search requires testing every item in the list, so is also O(N)
![Page 34: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/34.jpg)
Binary Search
If searching a sorted list of N items, we can do better
low = 1; high = N; while (low < high)
{mid = (low+high)/2;if (A[mid]< x)
low = mid +1;else high = mid;
} if (X == A[low])
return low; return -1;
![Page 35: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/35.jpg)
Limitations of Big-O
In the following definition of O:
T(N) is O(F(N)) if there are positive constants c and N0 such that T(N)cF(N) whenever N N0 .
N0 can be a very large number.
The positive constants c can be a large number.
Example : Compare the running time two algorithms
Algorithm1 = 10000N
Algorithm2 = 2N log(N)
![Page 36: Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649e755503460f94b75893/html5/thumbnails/36.jpg)
Other Issues of Algorithm Analysis
Memory accessDisk access Average-case v.s. worst-case running
time.Implementation Difficulties