Chapter Two

Algorithm Analysis Empirical vs. theoretical Space vs. time Worst case vs. Average case Upper, lower, or tight bound Determining the runtime of programs What about recursive programs?

What’s the runtime?int n;

cin >> n;

for (int i=0; i<n; i++)

for (int j=0; j<n; j++)

for (int k=0; k<n; k++)

cout << “Hello world!\n”;

What if the last line is replaced by:

string *s=new string(“Hello world!\n”);

O(n3) runtime

O(n3) time and space

2n3+n2+n+2?

Resource Analysis Runtime: we’d like to count the steps –

but that would be machine dependent

Space: we may also be interested in space usage

ignore constant factors, use O() notation count steps equivalent to machine language

instructions

count the bytes used

Asymptotic notation g(n) is said to be O(f(n)) if there exist constants c and n0

such that g(n) < c f(n) for all n > n0

g(n) is said to be (f(n)) if there exist positive constants c and n0 such that 0 <= c f(n) < g(n) for all n > n0

g(n) is said to be (f(n)) if g(n) = O(f(n)) and g(n) = (f(n))

O: like <= for functions (asymptotically speaking) : like >= : like =

for all n > n0

ignore constant factors, lower order terms

Asymptotic notation: examples Asymptotic runtime, in terms of O, ,

? Suppose the runtime for a function is

n2 + 2n log n + 40 0.0000001 n2+ 1000000n1.999

n3 + n2 log n n2.0001 + n2 log n 2n+ 100 n2

1.00001n+ 100 n97

Asymptotic comparisons 0.0000001 n2 = O(1000000n1.999 )?

n1.000001 = O(n log n)?

1.0001n = O(n943)?

lg n = (ln n)?

(Compare the limit of the quotient of the functions)

No – a polynomial with a higher power dominates one with a lower power

No – all polynomials (n.000001) dominate any polylog (log n)

No – all exponentials dominate any polynomial

Yes – different bases are just a constant factor difference

What’s the runtime?int n;

cin >> n;

for (int i=0; i<n; i++)

for (int j=0; j<n; j++)

for (int k=0; k<n; k++)

cout << “Hello world!\n”;

for (int i=0; i<n; i++)

for (int j=0; j<n; j++)

for (int k=0; k<n; k++)

cout << “Hello world!\n”;

(n3) + (n3) = (n3)Statements or blocks in sequence: add

What’s the runtime?int n;

cin >> n;

for (int i=0; i<n; i++)

for (int j=n; j>1; j/=2)

cout << “Hello world!\n”;

Loops: add up cost of each iteration(multiply loop cost by number of iterations

if they all take the same time)

log n iterations of n steps (n log n)

What’s the runtime?int n;

cin >> n;

for (int i=0; i<n; i++)

for (int j=0; j<i; j++)

cout << “Hello world!\n”;

Loops: add up cost of each iteration

1 + 2 + 3 + … + n = n(n+1)/2 = O(n2)

What’s the runtime?template <class Item>

void insert(Item a[], int l, int r)

{ int i;

for (i=r; i>l; i--) compexch(a[i-1],a[i]);

for (i=l+2; i<=r; i++)

{ int j=i; Item v=a[i];

while (v<a[j-1])

{ a[j] = a[j-1]; j--; }

a[j] = v;

}

}

What’s the runtime?void myst(int n)

{ if (n<100)

for (int i=0; i<n; i++)

for (int j=0; j<n; j++)

for (int k=0; k<n; k++)

cout << “Hello world!\n”;

else

for (int i=0; i<n; i++)

for (int j=0; j<n; j++

cout << “Hello world!\n”;

}

Estimate the runtime Suppose an algorithm has runtime (n3)

suppose solving a problem of size 1000 takes 10 seconds. How long to solve a problem of size 10000?

Suppose an algorithm has runtime (n log n)

suppose solving a problem of size 1000 takes 10 seconds. How long to solve a problem of size 10000?

runtime 10-8 n3; if n=10000, runtime 10000s = 2.7hr

runtime 10-3 n lg n; if n=10000, runtime 133 secs

Worst vs. average case You might be interested in worst, best, or

average case analysis of an algorithm You can have upper, lower, or tight bounds on

each of those functions. Eg. For each n, some problem instances of

size n have runtime n and some have runtime n2.

Worst case: Best case: Average case:

(n2), (n), (log n), O(n2), O(n3)

(n), (log n), O(n2), (n)

(n), (log n), O(n2), O(n3)

Average case: need to know distribution of inputs

The Taxpayer Problem Tax time is coming up. The IRS needs to

process tax forms. How to access and update each taxpayer’s info?

ADT? ADT Dictionary: find(x), insert(x),

delete(x) Implementation?

Array Implementation Insert(x):

Find(k):

Delete(I):

Records[numRecs++] = x;Runtime: O(1)

For (I=0; I<numRecs; I++) if (records[I].key == k)

return I;Runtime: O(n)

records[I]=records[--numRecs];Runtime: O(1)

Time for nOperations?

O(n2)

Sorted Array Implementation Find(x):

Runtime?

int bot=1, top=numRecs-1, mid;

while (bot <= top) {

mid = (bot + top)/2;

if (data[mid]==x) return mid;

if (data[mid]<x) top=mid-1; else bot=mid+1;

}

return –1;

Analysis of Binary Search How many steps to search among n

items? Number of items eliminated at each step? Definition of lg(x)? Runtime?

O(log n)

Sorted Array, cont. Insert(x)?

Delete(x)?

Time for n insert, delete, and find ops?

O(n)

O(n)

O(n2)

Which implementation is better?

find(x) insert(x) delete(x)ArrayS. Array

Worst case for n operations? Array: Sorted Array:

O(n2)

What if some operations are more frequent than others?

O(n2)

O(log n) O(n) O(n)

O(n) O(1) O(1)

Molecule viewer example Java demos: molecule viewer

Example1 Example2 Example3

Molecule Viewer Source Snippet/* * I use a bubble sort since from one iteration to the next, the sort * order is pretty stable, so I just use what I had last time as a * "guess" of the sorted order. With luck, this reduces O(N log N) * to O(N) */

for (int i = nvert - 1; --i >= 0;) { boolean flipped = false; for (int j = 0; j <= i; j++) {

int a = zs[j];int b = zs[j + 1];if (v[a + 2] > v[b + 2]) { zs[j + 1] = a; zs[j] = b; flipped = true; }

} if (!flipped) break;}

Merge sort runtime?void mergesort(first, last) {

if (last-first >= 1) {

mid=(last-first)/2 + first;

mergesort(first, mid);

mergesort(mid+1,last);

merge(first, mid, last);

}

}

T(n) = 2T(n/2) + c nT(1) = b;

Called a recurrence relation

Recurrence relationsIn Discrete Math: you’ll learn how to solve these.

In this class: we’ll say “Look it up.”

But you will be responsible for knowing how to write down a recurrence relation for the runtime of a program.

Divide-and-conquer algorithms like merge sort that divide problem size by 2 and use O(n) time to conquer

T(n) = 2T(n/2) + c n

have runtime O(n log n)

Hanoi runtime?void hanoi(n, from, to, spare {

if (n > 0) {

hanoi(n-1,from,spare,to);

cout << from << “ – “ << to << endl;

hanoi(n-1,spare,to,from);

}

}

T(n) = 2T(n-1) + cT(0) = b

Look it up: T(n) = O(2n)

Hanoi recurrence solutionT(n)=2T(n-1)+cT(n-1) = 2T(n-2) + cT(n-2) = 2T(n-3) + c_______T(n) = 2T(n-1) + c = 2 [ 2T(n-2) + c ] + c = 22 T(n-2) + 2 c + c = 23 T(n-3) + 22 c + 21 c + 20 c … = 2k T(n-k) + 2k-1 c + 2k-2 c + … + 21 c + 20 c = 2k T(n-k) + c(2k – 1)Done when n-k=0 since we know T(0). T(n) = 2n b + c 2n - c = (2n)

Binary Search recurrence?Recurrence relation?

T(n)=T(n/2)+c; T(1) = b

Look it up: (log n)