14.1 CompSci 102© Michael Frank Today’s topics Recurrence relationsRecurrence relations...

CompSci 102 © Michael Frank 14.1

Today’s topics

• Recurrence relationsRecurrence relations– Stating recurrencesStating recurrences– LiHoReCoCoLiHoReCoCo– Divide & conquerDivide & conquer– Master’s methodMaster’s method

• Reading: Sections 6.1-6.3Reading: Sections 6.1-6.3• UpcomingUpcoming

– Applications of inclusion-exclusionApplications of inclusion-exclusion


§6.1: Recurrence Relations

• A A recurrence relationrecurrence relation (R.R., or just (R.R., or just recurrencerecurrence) for a ) for a sequence sequence {{aann}} is an equation that expresses is an equation that expresses aann in terms of in terms of

one or more previous elements one or more previous elements aa00, …, , …, aann−1−1 of the sequence, of the sequence, for all for all nn≥≥nn00..– I.e.I.e., just a recursive definition, without the base cases., just a recursive definition, without the base cases.

• A particular sequence (described non-recursively) is said A particular sequence (described non-recursively) is said to to solvesolve the given recurrence relation if it is consistent with the given recurrence relation if it is consistent with the definition of the recurrence.the definition of the recurrence.– A given recurrence relation may have many solutions.A given recurrence relation may have many solutions.


Recurrence Relation Example

• Consider the recurrence relationConsider the recurrence relation

aann = 2 = 2aann−1−1 − − aann−2−2 ( (nn≥2)≥2)..

• Which of the following are solutions?Which of the following are solutions?aann = 3 = 3nn

aann = 2 = 2nn

aann = 5 = 5

Yes

Yes

No


Example Applications

• Recurrence relation for growth of a bank Recurrence relation for growth of a bank account with account with PP% interest per given period:% interest per given period:

MMnn = = MMnn−1−1 + ( + (PP/100)/100)MMnn−1−1

• Growth of a population in which each Growth of a population in which each organism yields 1 new one every period organism yields 1 new one every period starting 2 time periods after its birth.starting 2 time periods after its birth.

PPnn = = PPnn−1−1 + + PPnn−2−2 (Fibonacci relation) (Fibonacci relation)


Solving Compound Interest RR

• MMnn = = MMnn−1−1 + ( + (PP/100)/100)MMnn−1−1

= (1 + = (1 + PP/100) /100) MMnn−1−1

= = rr MMnn−1−1 (let (let rr = = 1 + 1 + PP/100/100))

= = rr ( (rr MMnn−2−2))

= = rr··rr·(·(rr MMnn−3−3)) …and so on to……and so on to…

= = rrnn MM00


Tower of Hanoi Example

• Problem: Get all disks from peg 1 to peg 2.Problem: Get all disks from peg 1 to peg 2.– Rules: (a) Only move 1 disk at a time.Rules: (a) Only move 1 disk at a time.– (b) Never set a larger disk on a smaller one.(b) Never set a larger disk on a smaller one.

Peg #1 Peg #2 Peg #3


Hanoi Recurrence Relation

• Let Let HHnn = # moves for a stack of = # moves for a stack of nn disks. disks.

• Here is the optimal strategy:Here is the optimal strategy:– Move top Move top nn−1−1 disks to spare peg. ( disks to spare peg. (HHnn−1−1 moves) moves)

– Move bottom disk. (1 move)Move bottom disk. (1 move)

– Move top Move top nn−1−1 to bottom disk. ( to bottom disk. (HHnn−1−1 moves) moves)

• Note that:Note that: HHnn = 2 = 2HHnn−1−1 + 1 + 1

– The # of moves is described by a Rec. Rel.The # of moves is described by a Rec. Rel.


Solving Tower of Hanoi RR

HHnn = 2 = 2 HHnn−1−1 + 1 + 1

= = 2 (2 2 (2 HHnn−2−2 + 1) + 1 + 1) + 1 = 2= 22 2 HHnn−2−2 + 2 + 1 + 2 + 1

= 2= 222(2 (2 HHnn−3−3 + 1) + 2 + 1 + 1) + 2 + 1 = 2= 233 HHnn−3−3 + 2 + 222 + 2 + 1 + 2 + 1

… …

= 2= 2nn−1−1 HH11 + 2 + 2nn−2−2 + … + 2 + 1 + … + 2 + 1

= 2= 2nn−1−1 + 2 + 2nn−2−2 + … + 2 + 1 + … + 2 + 1 (since (since HH11 = 1) = 1)

= =

= 2= 2nn − 1 − 1

∑−

=

1

0

2n

i

i


Another R.R. Example

• Find a R.R. & initial conditions for the number of Find a R.R. & initial conditions for the number of bit strings of length bit strings of length nn without two consecutive 0s. without two consecutive 0s.

• We can solve this by breaking down the strings to We can solve this by breaking down the strings to be counted into cases that end in 0 and in 1.be counted into cases that end in 0 and in 1.– For each ending in 0, the previous bit must be 1, and For each ending in 0, the previous bit must be 1, and

before that comes any qualifying string of length before that comes any qualifying string of length nn−2−2..– For each string ending in 1, it starts with a qualifying For each string ending in 1, it starts with a qualifying

string of length string of length nn−1−1..

• Thus, Thus, aann = = aann−1 −1 + + aann−2−2. (Fibonacci recurrence.). (Fibonacci recurrence.)– The initial conditions are: The initial conditions are: aa00 = 1 = 1 ( (εε), ), aa11 = 2 = 2 (0 and 1). (0 and 1).

1 0(n−2 bits) 1(n−1 bits)


Yet another R.R. example…

• Give a recurrence (and base cases) for the Give a recurrence (and base cases) for the number of number of nn-digit decimal strings containing -digit decimal strings containing an an eveneven number of 0 digits. number of 0 digits.

• Can break down into the following cases:Can break down into the following cases:– Any valid string of length Any valid string of length nn−1−1 digits, with any digits, with any

digit 1-9 appended.digit 1-9 appended.– Any Any invalidinvalid string of length string of length nn−1−1 digits, + a 0. digits, + a 0.

• aann = 9 = 9aann−1−1 + (10 + (10nn−1−1 − − aann−1−1) = 8) = 8aann−1−1 + 10 + 10nn−1−1..– Base cases: Base cases: aa00 = 1 = 1 ( (εε), ), aa11 = 9 = 9 (1-9).(1-9).


§6.2: Solving Recurrences

• A A lilinear near hohomogeneous mogeneous rerecurrence of degree currence of degree kk with with coconstant nstant cocoefficientsefficients (“(“kk--LiHoReCoCo”) is a rec. rel. of the formLiHoReCoCo”) is a rec. rel. of the form

aann = = cc11aann−1−1 + … + + … + cckkaann−−kk,,

where the where the ccii are all real, and are all real, and cckk ≠ 0≠ 0..

• The solution is uniquely determined if The solution is uniquely determined if kk initial conditions initial conditions aa00……aakk−1−1 are provided. are provided.

General Solution Schemas


Solving LiHoReCoCos

• Basic idea: Look for solutions of the form Basic idea: Look for solutions of the form aann = = rrnn, where , where rr

is a constant.is a constant.

• This requires solving the This requires solving the characteristic equationcharacteristic equation::rrnn = = cc11rrnn−1−1 + … + + … + cckkrrnn−−kk, , i.e.i.e., ,

rrkk − − cc11rrkk−1−1 − … − − … − cckk = 0 = 0

• The solutions The solutions rr to this equation are called the to this equation are called the characteristic rootscharacteristic roots of the LiHoReCoCo. of the LiHoReCoCo.– They can yield an explicit formula for the sequence.They can yield an explicit formula for the sequence.

(rearrange& × by rk−n)


Solving 2-LiHoReCoCos

• Consider an arbitrary 2-LiHoReCoCo:Consider an arbitrary 2-LiHoReCoCo:aann = = cc11aann−1−1 + + cc22aann−2−2

• It has the characteristic equation (C.E.): It has the characteristic equation (C.E.): rr22 − − cc11r r − − cc22 = 0 = 0

• Theorem 1:Theorem 1: If the CE has 2 roots If the CE has 2 roots rr11≠≠rr22, , then the solutions to the RR are given by:then the solutions to the RR are given by:

aann = = αα11rr11nn + + αα22rr22

nn for for nn≥0≥0for any and all constants for any and all constants αα11, , αα22..Special case: an = r1

n and an = r2n are, of course, solutions.


Example

• Solve the recurrence Solve the recurrence aann = = aann−1−1 + 2 + 2aann−2−2 given the given the initial conditions initial conditions aa00 = 2, = 2, aa11 = 7 = 7..

• Solution: Use theorem 1:Solution: Use theorem 1:– We have We have cc11 = 1, = 1, cc22 = 2 = 2– The characteristic equation is: The characteristic equation is: rr22 − − rr − 2 = 0 − 2 = 0– Solve it:Solve it:

– so, so, rr = 2 = 2 or or rr = −1 = −1..– So, So, aann = = αα11 2 2nn + + αα22 (−1) (−1)nn. .

(Using thequadraticformula here.)

a

acbbx

cbxax

2

4

0

2

2

−±−=

⇔=++

.1or 22

31

2

91

)1(2

)2)(1(4)1()1( 2

−=±

=±

=−−−±−−

=r


Example Continued…

• To find To find αα11 and and αα22, just solve the equations for the initial , just solve the equations for the initial conditions conditions aa00 and and aa11: :

aa00 = 2 = = 2 = αα112200 + + αα22 (−1) (−1)00

aa11 = 7 = = 7 = αα112211 + + αα22 (−1) (−1)11

Simplifying, we have the pair of equations:Simplifying, we have the pair of equations:2 = 2 = αα11 + + αα22

7 = 27 = 2αα11 − − αα22which we can solve easily by substitution:which we can solve easily by substitution:

αα22 = 2− = 2−αα11; ; 7 = 27 = 2αα11 − (2− − (2−αα11) = 3) = 3αα11 − 2 − 2; ;

9 = 39 = 3αα11; ; αα11 = 3 = 3; ; αα22 = −1 = −1..• Using Using αα11 and and αα22, our final answer is: , our final answer is: aann = = 3·23·2nn − (−1) − (−1)nn

Check: {an≥0} = 2, 7, 11, 25, 47, 97 …


Proof of Theorem 1

• Proof that Proof that aann = = αα11rr11nn++αα22rr22

nn is always a solution: is always a solution:

– We know We know rr1122 = = cc11rr1 1 + + cc22 and and rr22

22 = = cc11rr22 + + cc22..

– Now we can show the proposed sequence satisfies the Now we can show the proposed sequence satisfies the recurrence recurrence aann = = cc11aann−1−1 + + cc22aann−2−2::

cc11aann−1−1 + + cc22aann−2 −2 = = cc11((αα11rr11nn−1−1++αα22rr22

nn−1−1) + ) + cc22((αα11rr11nn−2−2++αα22rr22

nn−2−2))

= = αα11rr11nn−2−2((cc11rr11++cc22) + ) + αα22rr22

nn−2−2((cc11rr22++cc22))

= = αα11rr11nn−2−2rr11

22 + + αα22rr22nn−2−2rr22

22 = = αα11rr11nn + + αα22rr22

nn = a = ann. □ . □

• Can complete the proof by showing that for any Can complete the proof by showing that for any initial conditions, we can find corresponding initial conditions, we can find corresponding αα’s. ’s. – But it turns out this goes through only if But it turns out this goes through only if rr11≠r≠r22..


The Case of Degenerate Roots

• Now, what if the C.E. Now, what if the C.E. rr22 − − cc11r r − − cc22 = 0 = 0 has has

only 1 root only 1 root rr00??

• Theorem 2:Theorem 2: Then, Then,aann = = αα11rr00

nn + + αα22nrnr00nn, for all , for all nn≥0≥0,,

for some constants for some constants αα11, , αα22..


Degenerate Root Example

• Solve Solve aann = 6 = 6aann−1−1−9−9aann−2−2 with with aa00=1, =1, aa11=6=6..

• The C.E. is: The C.E. is: rr22−6−6rr+9=0+9=0..– Note that Note that bb22−4−4acac = (−6) = (−6)22−4·1·9 = 36−36 = 0−4·1·9 = 36−36 = 0..

• Therefore, there is only one root, namelyTherefore, there is only one root, namely−−bb/2/2aa = −(−6)/2 = 3 = −(−6)/2 = 3..


k-LiHoReCoCos

• Consider a Consider a kk-LiHoReCoCo:-LiHoReCoCo:• It’s C.E. is:It’s C.E. is:

• Theorem 3:Theorem 3: If this has If this has kk distinct roots distinct roots rrii, , then the then the

solutions to the recurrence are of the form:solutions to the recurrence are of the form:

for all for all nn≥0≥0, where the , where the ααii are constants. are constants.

∑=

−=k

iinin aca

101

=−∑=

−k

i

iki

k rcr

∑=

=k

i

niin ra

1

α


Degenerate k-LiHoReCoCos

• Suppose there are Suppose there are tt roots roots rr11,…,,…,rrtt with with

multiplicities multiplicities mm11,…,,…,mmtt. Then:. Then:

for all for all nn≥0≥0, where all the , where all the αα are constants. are constants.

∑ ∑=

−

=⎟⎟⎠

⎞⎜⎜⎝

⎛=

t

i

ni

m

j

jjin rna

i

1

1

0,α


LiNoReCoCos

• Linear Linear nononhomogeneousnhomogeneous RRs with constant RRs with constant coefficients may (unlike Licoefficients may (unlike LiHoHoReCoCos) ReCoCos) contain some terms contain some terms FF((nn)) that depend that depend onlyonly on on nn (and (and notnot on any on any aaii’s). General form:’s). General form:

aann = = cc11aann−1−1 + … + + … + cckkaann−−kk + + FF((nn))

The associated homogeneous recurrence relation(associated LiHoReCoCo).


Solutions of LiNoReCoCos

• A useful theorem about LiNoReCoCos:A useful theorem about LiNoReCoCos:– If If aann = = pp((nn)) is any is any particularparticular solution to the solution to the

LiNoReCoCo:LiNoReCoCo:

– Then Then allall of its solutions are of the form: of its solutions are of the form:aann = = pp((nn) + ) + hh((nn)),,

where where aann = = hh((nn)) is is anyany solution to the solution to the associated associated

homogeneous RRhomogeneous RR

)(1

nFacak

iinin +⎟⎠

⎞⎜⎝

⎛= ∑

=−

⎟⎠

⎞⎜⎝

⎛= ∑

=−

k

iinin aca

1


LiNoReCoCo Example

• Find all solutions to Find all solutions to aann = 3 = 3aann−1−1+2+2nn. Which . Which

solution has solution has aa11 = 3 = 3??

– Notice this is a 1-LiNotice this is a 1-LiNoNoReCoCo. Its associated ReCoCo. Its associated 1-Li1-LiHoHoReCoCo is ReCoCo is aann = 3 = 3aann−1−1, whose solutions , whose solutions

are all of the form are all of the form aann = = αα33nn. Thus the solutions . Thus the solutions

to the original problem are all of the form to the original problem are all of the form aann = = pp((nn) + ) + αα33nn.. So, all we need to do is find So, all we need to do is find

one one pp((nn)) that works. that works.


Trial Solutions

• If the extra terms If the extra terms FF((nn)) are a degree- are a degree-tt polynomial in polynomial in nn, you should try a general degree-, you should try a general degree-tt polynomial as polynomial as the particular solution the particular solution pp((nn))..

• This case: This case: FF((nn)) is linear so try is linear so try aann = = cncn + + dd..

cn+dcn+d = 3( = 3(cc((nn−1)+−1)+dd) + 2) + 2nn (for all (for all nn))(2(2cc+2)+2)nn + (2 + (2dd−3−3cc) = 0) = 0 (collect terms) (collect terms)So So cc = −1 = −1 and and dd = −3/2 = −3/2..

So So aann = − = −nn − 3/2 − 3/2 is a solution. is a solution.• Check:Check: aann≥1≥1 = {−5/2, −7/2, −9/2, … } = {−5/2, −7/2, −9/2, … }


Finding a Desired Solution

• From the previous, we know that all general From the previous, we know that all general solutions to our example are of the form:solutions to our example are of the form:

aann = = −−nn − 3/2 + − 3/2 + αα33nn..

Solve this for Solve this for αα for the given case, for the given case, aa11 = 3 = 3::

3 = −1 − 3/2 + 3 = −1 − 3/2 + αα3311

αα = 11/6 = 11/6• The answer is The answer is aann = = −−nn − 3/2 + (11/6)3 − 3/2 + (11/6)3nn..


Double Check Your Answer!

• Check the base case, Check the base case, aa11=3:=3:

aann = = −−nn − 3/2 + (11/6)3 − 3/2 + (11/6)3nn

aa11 = = −1 − 3/2 + (11/6)3−1 − 3/2 + (11/6)311

= −2/2 − 3/2 + 11/2 = −5/2 + 11/2 = 6/2 = 3 = −2/2 − 3/2 + 11/2 = −5/2 + 11/2 = 6/2 = 3

• Check the recurrence, Check the recurrence, aann = 3 = 3aann−1−1+2+2nn::−−nn − 3/2 + (11/6)3 − 3/2 + (11/6)3nn = 3[−( = 3[−(nn−1) − 3/2 + (11/6)3−1) − 3/2 + (11/6)3nn−1−1]+2]+2nn

= = 3[−3[−nn − 1/2 + (11/6)3 − 1/2 + (11/6)3nn−1−1] + 2] + 2nn= −3= −3nn − 3/2 + (11/6)3 − 3/2 + (11/6)3nn + + 22nn = − = −nn − 3/2 + (11/6)3 − 3/2 + (11/6)3nn ■ ■


§6.3: Divide & Conquer R.R.s

• Many types of problems are solvable by Many types of problems are solvable by reducing a problem of size reducing a problem of size nn into some into some number number aa of independent subproblems, each of independent subproblems, each of size of size nn//bb, where , where aa11 and and bb>1>1..

• The time complexity to solve such The time complexity to solve such problems is given by a recurrence relation:problems is given by a recurrence relation:– TT((nn) = ) = aa··TT((nn//bb) + ) + gg((nn)) Time to break problem

up into subproblemsTime for each subproblem


Divide+Conquer Examples

• Binary search:Binary search: Reduce problem to 1 sub- Reduce problem to 1 sub-problem (namely, searching half of the list) problem (namely, searching half of the list) (thus, (thus, aa=1=1) of size ) of size nn/2/2 (thus, (thus, bb=2=2).).– So So TT((nn) = ) = TT((nn/2/2)+)+cc ( (gg((nn)=)=cc, a constant), a constant)

• Merge sort:Merge sort: Break list of length Break list of length nn into 2 into 2 sublists (sublists (aa=2=2), each of size ), each of size nn/2/2 (so (so bb=2=2), ), sort them, then merge, in sort them, then merge, in gg((nn) = ) = ΘΘ((nn)) time. time.– So So TT((nn) = ) = TT((nn/2/2) + ) + cncn (roughly, for some (roughly, for some cc))


Fast Multiplication Example

• The ordinary grade-school algorithm takes The ordinary grade-school algorithm takes ΘΘ((nn22)) steps to multiply two steps to multiply two nn-digit numbers.-digit numbers.– Seems like too much work!Seems like too much work!

• Let’s find an asymptotically Let’s find an asymptotically faster faster algorithm!algorithm!• To find the product To find the product cdcd of two of two 22nn-digit base--digit base-bb

numbers, numbers, cc=(=(cc22nn-1-1cc22nn-2-2……cc00))bb and and dd=(=(dd22nn-1-1dd22nn-2-2……dd00))bb: first, we break : first, we break cc and and dd in half: in half:

cc==bbnnCC11++CC00, , dd==bbnnDD11++DD00, , and then... and then... (see next slide)(see next slide)


))((

)1()(

)(

)1()(

))()((

)(

))((

1001

00112

10001101

00112

000011111001

00112

001001112

0101

DDCCb

DCbDCbb

DCDCDCDCb

DCbDCbb

DCDCDCDCDCDCb

DCDCb

DCDCDCbDCb

DDbCCbcd

n

nnn

n

nnn

n

n

nn

nn

−−

++++=

+−−

++++=

−+−++

++=

+++=

++=

Derivation of Fast Multiplication

Zero

(Multiply outpolynomials)

(Factor last term)

Three multiplications, each with n-digit numbers


Recurrence Rel. for Fast Mult.

Notice that the time complexity Notice that the time complexity TT((nn)) of the of the fast multiplication algorithm obeys the fast multiplication algorithm obeys the recurrence:recurrence:

• TT(2(2nn)=3)=3TT((nn)+)+((nn))i.e.i.e.,,

• TT((nn)=3)=3TT((nn/2)+/2)+((nn))

So So aa=3, =3, bb=2=2..

Time to do the needed adds & subtracts of n-digit and 2n-digitnumbers


The Master Theorem

Consider a function Consider a function ff((nn)) that, for all that, for all nn==bbkk for for all all kkZZ++,,,,satisfies the recurrence relation:satisfies the recurrence relation:

ff((nn) = ) = afaf((nn//bb) + ) + cncndd

with with aa≥1≥1, integer , integer bb>1>1, real , real cc>0, >0, dd≥0≥0. Then:. Then:

⎪⎩

⎪⎨

⎧

>=<

∈da

dd

dd

ban

bann

ban

nfb if)(O

if)log(O

if)(O

)(log


Master Theorem Example

• Recall that complexity of fast multiply was:Recall that complexity of fast multiply was:

TT((nn) = 3) = 3TT((nn/2) + /2) + ((nn))

• Thus, Thus, aa=3, =3, bb=2, =2, dd=1=1. So . So aa > > bbdd, so case 3 , so case 3 of the master theorem applies, so:of the master theorem applies, so:

which is which is O(O(nn1.58…1.58…)), so the new algorithm is , so the new algorithm is strictly faster than ordinary strictly faster than ordinary ΘΘ((nn22)) multiply! multiply!

)(O)(O)( 3loglog 2nnnT ab ==

14.1 CompSci 102© Michael Frank Today’s topics Recurrence relationsRecurrence relations...

Documents

Transcript of 14.1 CompSci 102© Michael Frank Today’s topics Recurrence relationsRecurrence relations...