growth-of-functions.pptx
Transcript of growth-of-functions.pptx
Growth of functions
Growth of Functions 2
Function Growth
• The running time of an algorithm as input size approaches infinity is called the asymptotic running time
• We study different notations for asymptotic efficiency.• In particular, we study tight bounds, upper bounds and lower bounds.
Growth of Functions 3
The “sets” and their use – big Oh• Big “oh” - asymptotic upper bound on the growth of
an algorithm• When do we use Big Oh?1. Theory of NP-completeness2. To provide information on the maximum number of
operations that an algorithm performs• Insertion sort is O(n2) in the worst case
• This means that in the worst case it performs at most cn2 operations
• Insertion sort is also O(n6) in the worst case since it also performs at most dn6 operations
Growth of Functions 4
The “sets” and their use – OmegaOmega - asymptotic lower bound on the growth of an
algorithm or a problem*
When do we use Omega?1. To provide information on the minimum number of
operations that an algorithm performs• Insertion sort is (n) in the best case
• This means that in the best case its instruction count is at least cn, • It is (n2) in the worst case
• This means that in the worst case its instruction count is at least cn2
Growth of Functions 5
The “sets” and their use – Omega cont.2. To provide information on a class of algorithms that solve a problem
• Sort algorithms based on comparison of keys are (nlgn) in the worst case• This means that all sort algorithms based only on comparison of keys have to do at least
cnlgn operations• Any algorithm based only on comparison of keys to find the maximum of n
elements is (n) in every case• This means that all algorithms based only on comparison of keys to find maximum have
to do at least cn operations
Growth of Functions 6
The “sets” and their use - Theta• Theta - asymptotic tight bound on the growth rate of an algorithm
• Insertion sort is (n2) in the worst and average cases• The means that in the worst case and average cases insertion
sort performs cn2 operations• Binary search is (lg n) in the worst and average cases
• The means that in the worst case and average cases binary search performs clgn operations
• Note: We want to classify an algorithm using Theta. • In Data Structures used Oh
• Little “oh” - used to denote an upper bound that is not asymptotically tight. n is in o(n3). n is not in o(n)
Growth of Functions 7
The functions
• Let f(n) and g(n) be asymptotically nonnegative functions whose domains are the set of natural numbers N={0,1,2,…}.
• A function g(n) is asymptotically nonnegative, if g(n)³0 for all n³n0 where n0ÎN
Growth of Functions 8
Asymptotic Upper Bound: big O
f (n)
c g (n)
f (n) = O ( g ( n ))N
Why only for n N ?What is the purpose of multiplying by c > 0?
Graph shows thatfor all n N,f(n) c*g(n)
Growth of Functions 9
Asymptotic Upper Bound: O
Definition: Let f (n) and g(n) be asymptotically non-negative functions. We say f (n) is in O ( g ( n )) if there is a real positive constant c and a positive Integer N such that for every n ³ N 0 £ f (n) £ c g (n ).
Or using more mathematical notationO ( g (n) ) =
{ f (n )| there exist positive constant c and a positive integer N such that
0 £ f( n) £ c g (n ) for all n ³ N }
Growth of Functions 10
n2 + 10 n O(n2) Why?
0200400600800
100012001400
0 10 20 30
n2 + 10n
2 n2
take c = 2N = 10
2n2 > n2 + 10 n for all n>=10
Growth of Functions 11
Does 5n+2 ÎO(n)?Proof: From the definition of Big Oh, there must exist c>0
and integer N>0 such that 0 £ 5n+2£cn for all n³N.Dividing both sides of the inequality by n>0 we get:
0 £ 5+2/n£c.2/n £ 2, 2/n>0 becomes smaller when n increasesThere are many choices here for c and N. If we choose N=1 then c ³ 5+2/1= 7.If we choose c=6, then 0 £ 5+2/n£6. So N ³ 2.In either case (we only need one!) we have a c>o and N>0
such that 0 £ 5n+2£cn for all n ³ N. So the definition is satisfied and 5n+2 ÎO(n)
Growth of Functions 12
Does n2Î O(n)? No.
We will prove by contradiction that the definition cannot besatisfied. Assume that n2Î O(n).From the definition of Big Oh, there must exist c>0 and
integer N>0 such that 0 £ n2£cn for all n³N.
Dividing the inequality by n>0 we get 0 £ n £ c for all n³N.
n £ c cannot be true for any n >max{c,N }, contradicting our assumption
So there is no constant c>0 such that n£c is satisfied for all n³N, and n2 O(n)
Growth of Functions 13
O ( g (n) ) = { f (n )| there exist positive constant c
and positive integer N such that 0 £ f( n) £ c g (n ) for all n ³ N }
• 1,000,000 n2 O(n2) why/why not?
• (n - 1)n / 2 O(n2) why /why not?
• n / 2 O(n2) why /why not?
• lg (n2) O( lg n ) why /why not?
• n2 O(n) why /why not?
Growth of Functions 14
Asymptotic Lower Bound, Omega: Wf (n)
c * g (n)
f(n) = ( g ( n ))N
Growth of Functions 15
Asymptotic Lower Bound: W
Definition: Let f (n) and g(n) be asymptotically non-negative functions. We say f (n) is W ( g ( n )) if there is a positive real constant c and a positive integer N such that for every n ³ N 0 £ c * g (n ) £ f ( n).
Or using more mathematical notationW ( g ( n )) =
{ f (n) | there exist positive constant c and a positive integer N such that
0 £ c * g (n ) £ f ( n) for all n ³ N }
Growth of Functions 16
Is 5n-20Î W (n)?Proof: From the definition of Omega, there must exist c>0 and integer N>0
such that 0 £ cn £ 5n-20 for all n³N
Dividing the inequality by n>0 we get: 0 £ c £ 5-20/n for all n³N.
20/n £ 20, and 20/n becomes smaller as n grows.
There are many choices here for c and N.
Since c > 0, 5 – 20/n >0 and N >4For example, if we choose c=4, then 5 – 20/n ³ 4 and N ³ 20
In this case we have a c>o and N>0 such that 0 £ cn £ 5n-20 for all n ³ N. So the definition is satisfied and 5n-20 Î W (n)
Growth of Functions 17
W ( g ( n )) = { f (n) | there exist positive constant c and
a positive integer N such that 0 £ c * g (n ) £ f ( n) for all n ³ N }
• 1,000,000 n2 W (n2) why /why not?
• (n - 1)n / 2 W (n2) why /why not?
• n / 2 W (n2) why /why not?
• lg (n2) W ( lg n ) why /why not?
• n2 W (n) why /why not?
Growth of Functions 18
Asymptotic Tight Bound: Q
f (n)
d g (n)
f (n) = Q ( g ( n ))
N
c g (n)
Growth of Functions 19
Asymptotic Bound Theta: Q
Definition: Let f (n) and g(n) be asymptotically non-negative functions. We say f (n) is Q( g ( n )) if there are positive constants c, d and a positive integer N such that for every n ³ N 0 £ c g (n ) £ f ( n) £ d g ( n ).
Or using more mathematical notationQ ( g ( n )) =
{ f (n) | there exist positive constants c, d and a positive integer N such that
0 £ c g (n ) £ f ( n) £ d g ( n ). for all n ³ N }
Growth of Functions 20
More on Q• We will use this definition:
Q (g (n)) = O( g (n) ) Ç W ( g (n) )
Growth of Functions 21
• We show:
1.
2.
?)(32
1 Does 22 nnn
)(32
1 22 nOnn
)(32
1 22 nnn
Growth of Functions 22
?)(32
1 Does 22 nOnn
12 and ,
. all for
:get weby inequality the Dividing
. all for
that such and, exist must there definition the From
Choose
1/2. Since
Nn
Nncn
n
Nncnnn
Nc
c
cnn
4
1321
3210
02
232210
00
So
.4/1
, finitefor 0/3
Growth of Functions 23
?)(32
1 Does 22 nnn
. and So
. all for
1/4.c Choose finite for 0 Since
6. Since
.
that such and exist must There
1241
12321
41
.2/1 ,3
and 32100
3210
get we02by Dividing
allfor 322120
00
N/c
nn
cn/n
N, c
nc
n
Nnnncn
Nc
N
Growth of Functions 24
More Q• 1,000,000 n2 Q(n2) why /why not?
• (n - 1)n / 2 Q(n2) why /why not?
• n / 2 Q(n2) why /why not?
• lg (n2) Q ( lg n ) why /why not?
• n2 Q(n) why /why not?
Qualified Notation
Ω Omega Notation • best case Ω - describes a lower bound for all input (it can't get any
better than this). Example: the array is already correctly sorted.
• worst case Ω - describes a lower bound for worst case input, possibly greater than best case. Example: the array is sorted in reverse order.
• just Ω - same as best case Ω
Qualified Notation
O Big-O Notation • best case O - describes an upper bound for best case input, possibly
lower than worst case. Example: the array is already correctly sorted.
• worst case O - describes an upper bound for all input (it can't get any worse than this). Example: the array is sorted in reverse order.
• just O - same as worst case O
Qualified Notation
Θ Theta Notation • best case Θ - not used
• worst case Θ - describes asymptotic bounds for worst case input
• just Θ - same as worst case Θ
Comparing functions
Similarly forReflexivitySymmetryTranspose symmetry