LECTURE 5: DYNAMIC PROGRAMMING ADVANCED...
Transcript of LECTURE 5: DYNAMIC PROGRAMMING ADVANCED...
ADVANCED ALGORITHMS LECTURE 5: DYNAMIC PROGRAMMING
�1
LECTURE 5
ANNOUNCEMENTS
▸ Homework 1 due on Monday Sep 10, 11:59 PM
▸ Instructor office hours Thursday -> Friday (1 - 2 pm)
▸ Comment on notes — JeffE’s material
▸ Solutions to HW0 on course page
▸ Pecking order problem of HW1
�2
vertices birdsO O
could be cycles
O O
LECTURE 5
LAST CLASS
�3
▸ Divide and conquer
▸ Break instance into smaller pieces, solve pieces separately, combine solutions (*)
▸ Merge-sort, multiplying n digit numbers (time << n2)
▸ Median (and k’th smallest element) in O(n) time (no sorting!)
▸ Key idea: approximate median to break array into “nearly equal” pieces
▸ All analyses by writing recursion, solving (tree, plug-and-chug, …)
n58
1Master theorem
Notes onhomepageAkira Bazzitheorem Jeff E's
notes
LECTURE 5
OVERLAP IN RECURSIVE CALLS
�4
▸ Divide & conquer: usually sub-problems don’t interact “much”
▸ What if they do? Can run into wastefulness — solving same sub-problem many times…
▸ Motivation for dynamic programming
LECTURE 5
TODAY’S PLAN
�5
▸ Continue example — subset sum
▸ Knapsack problem
▸ Traveling salesman
LECTURE 5
RECAP: SUBSET SUM
�6
Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.
▸ Trivial algorithm: 2n time — check all subsets
▸ Sometimes better: divide into two halves — check all possibilities for how sum is divided
T(n) = S ⋅ T(n/2) ⟹ T(n) = Slog2 n = nlog2 S
Sn sL s I
Alo All f Aln i
It fl nitlogg2
5 30 i 5 7 4 15 38 221 29
LECTURE 5
TRY 2
�7
▸ See if first element is picked or not
Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.
timeonwhenwehauf.in
Suppose we can write S Afi 1 Afia tafiacorrectnessis
Alo isNOT Alo isused immediate we
usedtowrite fw arepdqkijy.ua.lt
f ESee ifwe canwrite
seeif S Alo can bewritten as
5 using All Aln i using Afi An i
I
LECTURE 5
RUNNING TIME
�8
c n runtime when we have
m Afit's
1 n 2T n l I Thi
Th F fintheomania In Fast
2 2 t t 2 t ln
nEE 2 2 almost
as bad as bruteforce
LECTURE 5
LOOKING MORE CLOSELY — EXAMPLE
�9
A[] = {1, 2, 3, 5, 7, 9, 10, 11}; S = 20TEt is
3,5 20 E3 B b 19
3 5.3 s 20 3,5 18 3 3,51 17N L I I
sit 17 5,7 s 17
1start index _2Sum needed 17
LECTURE 5
AVOIDING MULTIPLE SOLVES ON “SAME” INSTANCE
�10
▸ Store answers!
▸ How much are we storing?
▸ Running time
Howmany differentsub problems
can one have 7 cc 2
if I
any n possibilities possible
of distinct instances is E n sa
Modi thm
In each recursive step firstcheck if we have solved thisinstance before If so read offthe answer
If not we store the answer aftercomputation
Answers to computation can be stored in ann Sti array
start index Sam needed
LECTURE 5
IS THIS POLYNOMIAL?
�11
subtle point
when we say polytime algorithm we mean
polynomial in INPUT SIZE go
01 all an i S S 2
og att flog aft flog.amif ilgRunning time n.SE e udaiistn
Input size an Cfznz.sn andsaySERunning time n I fan n 2
LECTURE 5
“KNAPSACK” / COIN CHANGE
�12
Problem: suppose we are given coins of denominations 1c, 5c, 10c, 20c, 25c, 50c. Make change for $1.90 using the fewest # of coins.
▸ Heuristics?
a
206 201C
50 50d 50k 254,104,5
GreedyAtways use the largest coin that youCAN use
E
LECTURE 5
RECURSIVE ALGORITHM
�13
denominations do d dn
target sum S
Son'tused use do once The dotwice
1eachis a recursive call
s using d idnB and
7 do using it 3 tankandHans 2 S 2do 3
i2tans3n
LECTURE 5
RUNNING TIME
�14
All recursive calls are of the type
S set of denominationsI1
Sum needed suffix of doD
Space needed to store answers 01ns
Time needed Ofn S Edo
LECTURE 5
GENERAL IDEA OF DYNAMIC PROGRAMMING
�15
▸ Write down recursive algorithm
▸ Observe “common sub-problems” in recursion
▸ Capture sub-problems “succinctly”
LECTURE 5
TRAVELING SALESMAN PROBLEM
�16
Problem: given an undirected graph with edge weights, find a path that visits every vertex precisely once and returns to start.
LECTURE 5
TRAVELING SALESMAN PROBLEM
�17
Problem: given an undirected graph with edge weights, find a path that visits every vertex precisely once and returns to start.
▸ Trivial algorithm?
LECTURE 5
TRAVELING SALESMAN PROBLEM
�18
Problem: given an undirected graph with edge weights, find a path that visits every vertex precisely once and returns to start.
▸ Trivial algorithm?
▸ Many heuristics for geometric cases
LECTURE 5
RECURSIVE ALGORITHM
�19
LECTURE 5
RECURSIVE ALGORITHM — CORRECTNESS
�20
LECTURE 5
RECURSIVE ALGORITHM — CORRECTNESS
�21
LECTURE 5
SUB-PROBLEMS, RUNNING TIME
�22
LECTURE 5
SUB-PROBLEMS, RUNNING TIME
�23
LECTURE 5
CAN WE DO BETTER?
�24
▸ Likely not — NP hard problem
▸ We will see that finding “approximately good” solution possible!