CS 312: Algorithm Analysis
description
Transcript of CS 312: Algorithm Analysis
![Page 1: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/1.jpg)
CS 312: Algorithm Analysis
Lecture #22: Intro. to Dynamic Programming
Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick
This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.
![Page 2: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/2.jpg)
Objectives
Motivate Dynamic Programming (DP) Understand the tabular approach to DP Use DP to solve some example problems Extract the composition of a solution from
a DP table
![Page 3: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/3.jpg)
Families of Algorithms Introductory
Importance of algorithm analysis, seen through RSA Elegant Design Principles
Divide-and-Conquer Graph Exploration Greedy
Drawback: only usable on very specific types of problems
Need: “Sledgehammers of thealgorithms craft” Dynamic Programming Linear Programming
![Page 4: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/4.jpg)
Divide & Conquer
A
E F G E G H
B C B
E F G
![Page 5: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/5.jpg)
Dynamic Programming
start solving sub-problems at the bottom.
A
B C
E F G E G H
B
E F G
![Page 6: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/6.jpg)
Dynamic Programming
Build a table of results as you go
A
B C
E F G E G H
B
E F G
E solutionE
F solutionF
G solutionG
![Page 7: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/7.jpg)
Dynamic Programming
Avoid recomputing intermediate results;Consult the table
A
B C
E F G E G H
B
E F G
E solutionE
F solutionF
G solutionG
B solutionB
![Page 8: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/8.jpg)
Dynamic Programming
A
B C
E F G E G H
B
E F G
E solutionE
F solutionF
G solutionG
B solutionB
H solutionH
Avoid recomputing intermediate results;Consult the table
![Page 9: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/9.jpg)
Dynamic Programming
A
B C
E F G H
A
B C
E F G E G H
B
E F G
![Page 10: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/10.jpg)
DP Key Idea #1
Not every sub-problem is new.
Save time: retain prior results.
![Page 11: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/11.jpg)
DP Key Idea #2
Devise a minimal description for any problem instance and sub-problem
Use this description as key to a table
![Page 12: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/12.jpg)
Dynamic Programming Identify sub-problems Define their minimal description
Perspective #1: Divide & Conquer with Memory Table Solve the sub-problems one by one, smallest first Store the solutions to sub-problems in a table and reuse the solutions to
solve larger sub-problems Until the top (original) problem instance is solved
Perspective #2: DAG Nodes are sub-problems; edges are dependencies between the sub-
problems Linearize! Solve the sub-problems one by one in the linearized order
![Page 13: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/13.jpg)
Example: Binomial Coefficients
!( , )!( )!n k
n nC n k Ck k n k
How many ways are there to choose k items from a set of n items?
1 if 0 or
if 0
0 otherw e
1
s
11
i
n nk k
k k n
k n
![Page 14: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/14.jpg)
1 if 0 or 1 1
( , ) if 01
0 otherwise
k k nn n n
C n k k nk k k
Example: C(5,3)
C(5,3)
C(4,2) C(4,3)
C(3,1) C(3,2) C(3,2) C(3,3)
C(2,0) C(2,1)C(1,0) C(1,1)
C(2,1) C(2,2) C(2,1) C(2,2)C(1,0) C(1,1) C(1,0) C(1,1)
(C(n,k))1 1 1 1 1 1 1
1
1
1
![Page 15: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/15.jpg)
C(5,3): Be Wise
C(5,3)
C(4,2) C(4,3)
C(3,1) C(3,2) C(3,3)
C(2,0) C(2,1)
C(1,0) C(1,1)
C(2,2)
![Page 16: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/16.jpg)
Embed in a table; evaluate in the same order as before
From DAG to Table What is the minimal description? Can you embed this DAG in a table?
C(5,3)
C(4,2) C(4,3)
C(3,1) C(3,2) C(3,3)
C(2,0) C(2,1)
C(1,0)C(1,1)
C(2,2)
C(5,3)C(4,2) C(4,3)
C(3,1) C(3,2) C(3,3)
C(2,0) C(2,1) C(2,2)
Linearize our DAG
C(2,0) C(2,1) C(2,2) C(3,1) C(3,2) … C(5,3)
→𝑘→
![Page 17: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/17.jpg)
5
4
3
2
1
0
0 1 2 3
1 if 0 or 1 1
( , ) if 01
0 otherwise
k k nn n n
C n k k nk k k
Does this look familiar?
C(5, 3)
n:
k:
![Page 18: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/18.jpg)
Pascal’s Triangle
Blaise Pascal (1623-1662)Second person to invent the calculatorReligious philosopherMathematician and physicist
![Page 19: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/19.jpg)
“Dynamic Programming” Very little to do with writing code Developed in the area of Operations Research
Conceived to optimally plan multi-stage processes Then, “programming” = “planning” Solves an optimization problem
Described by recurrence relations among problems and sub-problems
Possibly time-variant i.e. a dynamic system
In the CS setting, often time-invariant Coined by Bellman in the 1950s
![Page 20: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/20.jpg)
DP: From Problemto Table to Algorithm
Start with a problem definition Devise a minimal description for any problem instance and sub-problem Define recurrence to specify the relationship of problems to sub-
problems i.e., Define the conceptual DAG on sub-problems
Embed the DAG in a table Use the index variables
2-D case: index rows and columns Two possible strategies
1. Fill in the sub-problem cells, proceeding from the smallest to the largest2. Draw the DAG in the table from the top problem down to the smallest sub-
problems; solve the relevant sub-problems in their table cells, from smallest to largest. i.e., solve top problem recursively, using the table as a memory function
![Page 21: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/21.jpg)
Making Change using DP Given , what’s the smallest number of coins required to make
change for ? Example:
m = 4 different denominations of coins Values d = [1, 2, 4, 7] – Look familiar?
In general, does the greedy algorithm always give the optimal answer?
Greedy can fail. Example:
d = [1, 10, 18], = 20
Goal: Solve every case using some other strategy
How about Dynamic Programming?
![Page 22: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/22.jpg)
Making Change using DP Problem: Given , what’s the smallest number of coins to make change for ? Define sub-problem representation:
Let min. number of coins needed to make change for using coins of type 1 through .
Coins have denominations in the array ; assume Basic ideas for relationship among sub-problems as a recurrence relation:
If , then take coins If , either take a coin of value or do not take, and solve the problem (make
change) for the balance Take out a piece of scratch paper and write a mathematical version of
the recurrence relation
![Page 23: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/23.jpg)
Making Change using DP If , then take coins If , either take a coin of value or do not
0 1/ 1
( , ) min{ ( , 1&) 1, ( 1( 1, ) 1& 0
0 &
, )
01
i
i i
i
ij d i
C i j C i j d iC i j djC i j i j d
i j
![Page 24: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/24.jpg)
Making Change using DP
Design the table to contain the sub-problem results from the recurrence relation
Design the DP algorithm to walk the table properly
Solve an example by running the DP algorithm
Modify the resulting algorithm for space and time, if necessary
![Page 25: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/25.jpg)
Given j, what’s the smallest number of “coins” to make j in change?
0 1/ 1
( , ) min{ ( , 1&) 1, ( 1( 1, ) 1& 0
0 &
, )
01
i
i i
i
ij d i
C i j C i j d iC i j djC i j i j d
i j
Example
![Page 26: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/26.jpg)
Making Change
Amount 0 1 2 3 4 5 6 7
senine=1 0 1 2 3 4 5 6 7
seon=2 0 1 1 2 2 3 3 4
shum=4 0 1 1 2 1 2 2 3
limnah=7 0 1 1 2 1 2 2 1
j
i
How much space? How much time?
![Page 27: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/27.jpg)
Algorithm in Pseudo-codefunction coins(d, N)Input: Array d[1..n] specifies the coinage;
N is the number of units for which to make changeOutput: Minimum number of coins needed to make change for N units using coins from d
array c[1..n,0..N]for i=1 to n do
c[i,0] = 0
for i=1 to n dofor j=1 to N do
if i=1 and j<d[i] thenc[i,j] = +infinity
else if i=1 thenc[i,j] = 1+c[1,j-d[1]]
else if j<d[i] thenc[i,j] = c[i-1,j]
elsec[i,j] = min(c[i-1,j],1+c[i,j-d[i]])
return c[n,N]
Easy translation from table + recurrence to pseudo-code!
![Page 28: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/28.jpg)
Pre-computing vs. on-the-fly
Eager: Pre-computing Fill the table bottom-up, then extract solution
Lazy: On-demand Build the DAG (in the table) top-down Solve only the necessary table entries (again
from bottom-up)
![Page 29: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/29.jpg)
Comparison
How does DP algorithm compare to greedy? speed space correctness simplicity
![Page 30: CS 312: Algorithm Analysis](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816717550346895ddb87c2/html5/thumbnails/30.jpg)
Assignment
HW #15, #16 Greedy: Divisible Knapsack Dynamic Programming: (coin problem)
Project #5 Gene Sequence Alignment (DP)