CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1...
Transcript of CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1...
![Page 1: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/1.jpg)
CSC165 Week 8Larry Zhang, October 28, 2014
![Page 2: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/2.jpg)
an informal, anonymous and very useful survey of your learning experience
http://goo.gl/forms/AyC01bEk7g
Exclusive for Tuesday evening section
![Page 3: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/3.jpg)
some “scary” announcements
➔ A2 due next Monday, 10:00pm➔ term test 2 next Tuesday
◆ 6:10pm~7:00pm, in BA1130◆ aid sheet: 8.5”x11”, double-sided, handwritten◆ what’s in the test: three proofs
➔ extended office hours:◆ Thursday 4-6pm, BA4262◆ Friday & Monday 4-6pm, BA5287
➔ deadline for dropping: Nov. 3
![Page 4: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/4.jpg)
some tips for Assignment 2
![Page 5: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/5.jpg)
proof for statement with multiple quantifiers
∀x ∈ X, ∃y ∈ Y, ∀z ∈ Z, ∃w ∈ W, P => Qassume generic x ∈ X pick some y ∈ Y assume generic z ∈ Z pick some w ∈ W assume P … then Q then P => Q then ∃w ∈ W, P => Q then ∀z ∈ Z, ∃w ∈ W, P => Q then ∃y ∈ Y, ∀z ∈ Z, ∃w ∈ W, P => Qthen ∀x ∈ X, ∃y ∈ Y, ∀z ∈ Z, ∃w ∈ W, P => Q
∀ - assume generic
∃ - pick / let … be
P => Qassume P then Q
![Page 6: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/6.jpg)
the floor function
“jump”
![Page 7: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/7.jpg)
How to prove a = b ?➔ Prove (a ≤ b) ∧ (a ≥ b)
Assume two integers a < b➔ a ≤ b - 1 # two integers differ by at least 1
When the statement seems kind-of trivial but proving directly looks messy➔ try contradiction
![Page 8: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/8.jpg)
today’s outline
➔ formal definition of O, Ω➔ worst-case analyses of two algorithms
➔ problem solving session
![Page 9: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/9.jpg)
formal definitions of O and Ω
![Page 10: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/10.jpg)
recap O(n²)
set of functions that grow no faster than n² ➔ count the number of steps➔ constant factors don’t matter➔ only highest-order term matterThese functions are in O(n²)
![Page 11: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/11.jpg)
the formal definition of O(n²)
Beyond breakpoint B, f(n) is upper-bounded by cn², where c is some wisely chosen constant multiplier.
cn²
f(n)
B
![Page 12: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/12.jpg)
“chicken size” is in O(“turkey size”)
A chicken grows slower than a turkey in the sense that, after a certain breakpoint, a chicken will always be smaller than a turkey.
B
![Page 13: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/13.jpg)
the formal definition of O(n²)
Simple example: prove 700n² ∈ O(n²)
![Page 14: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/14.jpg)
the formal definition of Ω(n²)
O(n²): set of functions that grow no faster than n²Ω(n²): set of functions that grow no slower than n²ϴ(n²): set of functions that are in both O(n²) and Ω(n²) (functions growing as fast as n²)
upper-bound
lower-bound
![Page 15: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/15.jpg)
examples
![Page 16: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/16.jpg)
growth rate ranking of typical functions
grow slowly
grow fast
![Page 17: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/17.jpg)
analyse a sorting algorithm
![Page 18: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/18.jpg)
insertion sort
➔ grow a sorted list inside an unsorted list
➔ in each iteration◆ remove an element from the unsorted part
◆ insert it into the correct position in the sorted part
![Page 19: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/19.jpg)
insertion sort
see animation at: http://en.wikipedia.org/wiki/Insertion_sort
![Page 20: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/20.jpg)
insertion sortdef IS(A):
‘’’sort the elements in A in
non-decreasing order’’’
1. i = 1
2. while i < len(A):
3. t = A[i] # take red square out
4. j = i
5. while j > 0 and A[j-1] > t:
6. A[j] = A[j-1] # shift
7. j = j - 1
8. A[j] = t # put red square in
9. i = i + 1 # next element to be red-squared
i=1...n-1, that’s n-1 iterations, + 1 final loop guard
n: size of A
j=i, …, 1, in worst casethat’s i iterations, +1 final loop guard, total lines to run: 3i + 1
each iteration has (3i + 1) + 5 lines to execute
![Page 21: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/21.jpg)
insertion sortdef IS(A):
‘’’sort the elements in A in
non-decreasing order’’’
1. i = 1
2. while i < len(A):
3. t = A[i] # take red square out
4. j = i
5. while j > 0 and A[j-1] > t:
6. A[j] = A[j-1] # shift
7. j = j - 1
8. A[j] = t # put red square in
9. i = i + 1 # next element to be red-squared
n: size of A
each iteration has (3i + 1) + 5 lines to execute
loop guard
Line #1
i=1...n-1, that’s n-1 iterations, + 1 final loop guard
j=i, …, 1, in worst casethat’s i iterations, +1 final loop guard, total lines to run: 3i + 1
![Page 22: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/22.jpg)
insertion sort worst-case running time
![Page 23: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/23.jpg)
Prove the worst case complexity of insertion sort is O(n²)
Proof:# pick a big c, could also be 2, 3, 4, 12, 35, 600, ...
# make sure 11n² dominates for n ≥ B
![Page 24: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/24.jpg)
Proof:# pick a small c
# make sure (½) n² is smaller for n ≥ B
![Page 25: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/25.jpg)
analysis result
The worst case time complexity of insertion sort is in O(n²) and in Ω(n²), i.e., it’s in ϴ(n²)
![Page 26: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/26.jpg)
summary
➔ we first derived the exact form of WIS(n), then determined it’s upper and lower bounds
➔ don’t alway have to derive the exact form, as we will see soon
![Page 27: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/27.jpg)
analyse another algorithm
![Page 28: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/28.jpg)
maximum slice
➔ input: L, a list of numbers
➔ output: the maximum sum over slices of L
L = [-2, -3, 4, -1, 6, -3]
max = 4 + (-1) + 6 = 9
useful algorithm for stockbrokers
![Page 29: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/29.jpg)
def max_sum(L):
‘’’ maximum sum over slices of L’’’
1. max = 0
2. i = 0
3. while i < len(L):
4. j = i + 1
5. while j <= len(L):
6. sum = 0
7. k = i
8. while k < j:
9. sum = sum + L[k]
10. k = k + 1
11. if sum > max:
12. max = sum
13. j = j + 1
14. i = i + 1
15. return max
computing sum of slice [i:j]
enumerating all possible slices [i:j]
How it works:Enumerate all possible slices, compute the sum for each slice, and keep the max.
![Page 30: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/30.jpg)
def max_sum(L):
‘’’ maximum sum over slices of L’’’
1. max = 0
2. i = 0
3. while i < len(L):
4. j = i + 1
5. while j <= len(L):
6. sum = 0
7. k = i
8. while k < j:
9. sum = sum + L[k]
10. k = k + 1
11. if sum > max:
12. max = sum
13. j = j + 1
14. i = i + 1
15. return max
upper bound WMS(n)
# line 8-10, at most n iters
# line 5-13, at most n iters
# line 3-14, at most n iters
loop guard
loop guard
loop guard
# line 1-15, add 3 more lines
WMS(n) is in O(n³)
![Page 31: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/31.jpg)
def max_sum(L):
‘’’ maximum sum over slices of L’’’
1. max = 0
2. i = 0
3. while i < len(L):
4. j = i + 1
5. while j <= len(L):
6. sum = 0
7. k = i
8. while k < j:
9. sum = sum + L[k]
10. k = k + 1
11. if sum > max:
12. max = sum
13. j = j + 1
14. i = i + 1
15. return max
lower bound WMS(n)
WMS(n) is in Ω(n³)
L:
# first loop (i), at least iterations.(pick i in red zone)
# second loop (j), at least iterations(pick j in blue zone)
# third loop (k), at least iterations
Overall, it’s at least
0 n-1
This analysis can be made more precise by considering
(slice [i:j] is at least this long)
![Page 32: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/32.jpg)
summary
➔ we didn’t derive the exact form of WMS(n), we did different calculations for upper and lower bounds
➔ when finding upper-bound, it is OK to over-estimate the number of steps
➔ when finding lower-bound, it is OK to under-estimate the number of steps
![Page 33: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/33.jpg)
def max_sum(L):
‘’’ maximum sum over slices of L’’’
1. max = 0
2. i = 0
3. while i < len(L):
4. j = i + 1
5. while j <= len(L):
6. sum = 0
7. k = i
8. while k < j:
9. sum = sum + L[k]
10. k = k + 1
11. if sum > max:
12. max = sum
13. j = j + 1
14. i = i + 1
15. return max
This O(n³) algorithm is pretty stupid…
➔ exercise for home: design a max_sum that runs in O(n²)
◆ think about computing the sum of [0:4] right after getting the sum of [0:3]
➔ challenge for home: design a max_sum that runs in O(n)
![Page 34: CSC165 Week 8ylzhang/csc165f14/slides/lec08.pdf · 2015. 11. 24. · 12. max = sum 13. j = j + 1 14. i = i + 1 15. return max computing sum of slice [i:j] enumerating all possible](https://reader036.fdocuments.net/reader036/viewer/2022071409/61037152b52bf32e297508e8/html5/thumbnails/34.jpg)
problem solving: penny piles