COP 3530 Spring2012Data Structures & Algorithms

Discussion Session Week 5

Outline

• Growth of functions– Big Oh– Omega– Theta– Little Oh

Growth of Functions

Gives a simple view of the algorithm’s efficiency.

Allows us to compare the relative performance of alternative algorithms. f1(n) is O(n)f2(n) is O(n^2)

Growth of Functions

Exact running time of an algorithm is usually hard to compute, and it’s unnecessary.

For large enough inputs, the lower-order terms of an exact running time are dominated by high-order terms. f(n) = n^2 + 5n + 234 n^2 >> 5n + 234, when n is large enough

Asymptotic Notation: Big Oh (O)

f(n)= O(g(n)) iff there exist positive constants c and n0 such that f(n) ≤ cg(n) for all n ≥ n0

O-notation to give an upper bound on a function

Asymptotic Notation: Big Oh (O)

Example 1[linear function] f(n) = 3n+2 For n >= 2, 3n+2 <= 3n+n <= 4n. So f(n) = O(n).

We can arrive the same conclusion in other ways. For example, 3n + 2 <= 10n for n > 1.

The specific values of c and n0 used to satisfy the definition of big oh are not important, we only say f(n) is big oh of g(n). c/n0 do not matter.

Asymptotic Notation: Big Oh (O)

Example 2[quadric function] f(n) = 10n^2+4n+2 For n >= 2, f(n) <= 10n^2+5n. For n>= 5, 5n < n^2. Hence for n>= n0 = 5, f(n) <= 10n^2+n^2 = 11n^2. Therefore, f(n) = O(n^2).

Only the highest order term matters !!!

Asymptotic Notation: Big Oh (O)

Example 3[exponential function] f(n) = 6*2^n + n^2 For n>=4, n^2 <= 2^n. f(n) <= 6*2^n + 2^n = 7*2^n for n>=4Therefore, 6*2^n + n^2 = O(2^n)

Example 4[constant function] f(n) = 3 For any n, f(n) <= 4*1Therefore, f(n) = O(1)

Asymptotic Notation: Big Oh (O)

Example 5[loose bounds] f(n) = 3n+3 For n >= 10, 3n+3 <= 3n^2. Therefore, f(n) = O(n^2).

Usually, we mean tight upper bound when using big oh notation.

Example 6[Incorrect bounds] 3n+2 != O(1) since we cannot find n0/c such that 3n + 2 <= c, when n>=c0 (n can be infinity).

Similarly, 10n^2 + 6n + 2 != O(n).

Asymptotic Notation: Big Oh (O)

The specific values of c and n0 used to satisfy the definition of big oh are not important, we only say f(n) is big oh of g(n).

c/n0 do NOT matter, WHY?

f(n)=n. It’s close to 10n when comparing with n^2, n^3

f(n) is relatively small when n<n0

Asymptotic Notation: Omega Notation

Big oh provides an asymptotic upper bound on a function.Omega provides an asymptotic lower bound on a function.

Asymptotic Notation: Omega Notation

Example 7 f(n) = 3n+3 > 3n for all n. So f(n) = Omega(n)

Example 8[loose bounds] f(n) = 3n+3 > 1 for all n, so f(n) = Omega(1)

Asymptotic Notation: Theta Notation

Theta notation is used when function f can be bounded both from above and below by the same function g

Asymptotic Notation: Theta Notation

Example 9: f(n) = 3n+3 is Theta(n), since n <= 3n+3 <= 4n, when n >= 3.

Similarly, f(n) = 3n+2 is Theta(n) f(n) = 5n^2 - 10n + 9 is Theta(n^2)

f(n) is Theta(g(n)) iff f(n) is Omega(g(n)) and O(g(n))

Asymptotic Notation: Little oh (o)

The asymptotic upper bound provided by O-notation may or may not be asymptotically tight. 2n = O(n) is tight, 2n = O(n^2) is not tight.

We use o-notation to denote an upper bound that is NOT asymptotically tight.

Asymptotic Notation: Little oh (o)

f(n) = o(g(n)) iff f(n) = O(g(n)) and f(n) != Omega(g(n))

Example 10 3n+2 = o(n^2) as 3n+2 = O(n^2) and 3n+2 != Omega(n^2)

Example 11 3n+2 != o(n) as 3n+2 = Omega(n)

Review

Big oh: upper bound on a function.

Omega: lower bound.

Theta: lower and upper bound. - f(n) is Theta(g(n)) iff f(n) is O(g(n)) and Omega(g(n))

Little oh: loose upper bound. - f(n) = o(g(n)) iff f(n) = O(g(n)) and f(n) != Omega(g(n))

Office Hour This Week:

Thursday 9th period at E309