Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… ·...
-
Upload
phungnguyet -
Category
Documents
-
view
215 -
download
1
Transcript of Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… ·...
![Page 1: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/1.jpg)
1/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
Adam Smith
Data Structures and Algorithms
CMPSC 465
LECTURES 1 & 2 Analysis of Algorithms
• Course information
• What are algorithms?
• Why study them?
![Page 2: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/2.jpg)
Course information
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
1. Staff
2. Prerequisites
3. Lectures
4. Recitations
5. Textbook
6. Course website
7. Homework
8. Grading policy
9. Collaboration policy
![Page 3: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/3.jpg)
Etymology of “Algorithm”
Abu Abdullah Muhammad ibn Musa al-Khwarizmi (c. 780 -- 850 AD)
• Persian astronomer and mathematician
• lived in Baghdad, father of algebra
• “On calculating with hindu numerals” a treatise in Arabic, 825
• “Agoritmi de numero Indorum” translation into Latin, 12th century
• author’s name, mistaken for a plural noun, came to mean “calculation methods”
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
![Page 4: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/4.jpg)
Algorithm Design and Analysis
Theoretical study of how to solve computational
problems
• sorting a list of numbers
• finding a shortest route on a map
• scheduling when to work on homework
• answering web search queries
(Generally: precisely defined set of inputs and,
for each input, acceptable outputs)
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
![Page 5: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/5.jpg)
Algorithms
• Definition: Finite set of unambiguous
instructions for solving a problem.
– An algorithm is correct if on all legitimate inputs, it
outputs the right answer in a finite amount of time
• Can be expressed as
– pseudocode
– flow charts
– text in a natural language (e.g. English)
– computer code
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
![Page 6: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/6.jpg)
Data Structures
• Data structures are ways to store information
for which there are algorithms for performing
particular operations (retrieving/manipulating
information), e.g.
– linked lists
– hash tables
– arrays
– trees
– heaps
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
![Page 7: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/7.jpg)
Course Objectives
• classical algorithms and
data structures
• analysis of algorithms
• standard design techniques
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
![Page 8: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/8.jpg)
Why study algorithms?
• a language for talking about program behavior
• standard set of algorithms and design techniques
• feasibility (what can and cannot be done)
– halting problem, NP-completeness
• analyzing correctness and resource usage
• successful companies (Google, Mapquest, Akamai)
• computation is fundamental to understanding the world
– cells, brains, social networks, physical systems all can be
viewed as computational devices
• IT IS FUN!!!
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
![Page 9: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/9.jpg)
Performance isn’t everything
• Typical goal: Find most space- and time-efficient
algorithm for given problem.
• What else is important?
– modularity
– correctness
– maintainability
– functionality
– robustness
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
– user-friendliness
– programmer time
– simplicity
– extensibility
– reliability
![Page 10: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/10.jpg)
Performance isn’t everything
• Typical goal: Find most space- and time-efficient
algorithm for given problem.
• Even performance has many facets:
– type of memory access
– cache usage
– network usage
– parallelism
• This course: simple models, general skills
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
![Page 11: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/11.jpg)
The problem of sorting
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
Input: sequence a1, a2, …, an of numbers.
Example:
Input: 8 2 4 9 3 6
Output: 2 3 4 6 8 9
Output: permutation a'1, a'2, …, a'n such
that a'1 a'2 … a'n .
![Page 12: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/12.jpg)
Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
INSERTION-SORT (A, n) A[1 . . n]
for j 2 to n
do key A[ j]
i j – 1
while i > 0 and A[i] > key
do A[i+1] A[i]
i i – 1
A[i+1] = key
“pseudocode”
![Page 13: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/13.jpg)
Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
INSERTION-SORT (A, n) A[1 . . n]
for j 2 to n
do key A[ j]
i j – 1
while i > 0 and A[i] > key
do A[i+1] A[i]
i i – 1
A[i+1] = key
“pseudocode”
sorted
i j
key
A: 1 n
![Page 14: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/14.jpg)
Example of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
8 2 4 9 3 6
![Page 15: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/15.jpg)
Example of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
8 2 4 9 3 6
![Page 16: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/16.jpg)
Example of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
8 2 4 9 3 6
2 8 4 9 3 6
![Page 17: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/17.jpg)
Example of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
8 2 4 9 3 6
2 8 4 9 3 6
![Page 18: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/18.jpg)
Example of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
![Page 19: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/19.jpg)
Example of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
![Page 20: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/20.jpg)
Example of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
![Page 21: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/21.jpg)
Example of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
![Page 22: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/22.jpg)
Example of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
![Page 23: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/23.jpg)
Example of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
![Page 24: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/24.jpg)
Example of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
![Page 25: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/25.jpg)
Unit 1
Basics of reasoning about algorithms
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson L1.25
![Page 26: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/26.jpg)
Review: mathematical induction
• Theorem: All humans have the same height.
• Inductive formulation: For all integers n 1,
– P(n): Every set of n people contains people of a single height.
• Base case P(1): When a set contains only 1 person, everyone
in the set has the same height.
• Inductive step: Assume P(k) holds for some k 1.
– Consider a set of k+1 people, h1,…, hk+1
– Since P(k) holds, sets A and B contain people of the same height
– Since A and B overlap, people h1,…, hk+1 all have the same height
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson L1.26
A B
What’s wrong with this proof?
![Page 27: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/27.jpg)
Reasoning about Algorithms
• Focus of this course: thinking about algorithms
• Basic tool: mathematical induction
– Look for iterative structure in the algorithm
– Try to identify guarantees that (should) hold at each
step of the algorithm
– Use induction to show that the guarantees hold at
every step of the algorithm’s execution
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson L1.27
![Page 28: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/28.jpg)
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
Insertion Sort
INSERTION-SORT (A, n) A[1 . . n]
for j 2 to n
do key A[ j]
i j – 1
while i > 0 and A[i] > key
do A[i+1] A[i]
i i – 1
A[i+1] = key
“pseudocode”
sorted
i j
key
A: 1 n
![Page 29: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/29.jpg)
Correctness of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
Loop Invariant: After execution of execution j of for loop
1. A[1 . . n] is a permutation of the input array, and
2. A[1 . . j] is sorted
![Page 30: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/30.jpg)
Loop Invariants
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
A tool for analyzing iterative algorithms
(example of inductive reasoning)
Usually, we prove 3 statements
• Initialization: invariant holds on first execution
• Maintenance: if invariant held on all previous passes through the loop, it holds on current pass
• Termination: if invariant holds at the end, then some desired property holds (e.g. algorithm is correct).
![Page 31: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/31.jpg)
Correctness of Insertion Sort
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
• Initialization: A[1] is sorted.
• Maintenance: If A[1.. j-1 ] is sorted before pass j through the for loop, then A[1.. j] is sorted after the pass. This holds because A[j] is inserted in the correct place in A[1..j-1].
– Proving this formally requires looking carefully at while loop.
• Termination: If loop invariant holds at termination (j = n), Insertion Sort is correct. Loop invariant states that A[1..n] is sorted when the for loop exits. Since the array elements were never changed (only permuted), A now contains the sorted version of the input.
![Page 32: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/32.jpg)
How to measure running time?
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
• Parameterize the running time by the size of the
input, denoted by n, since short sequences are
easier to sort than long ones.
• Issue: the running time depends on the input: an
already sorted sequence is easier to sort.
• Generally, we seek upper bounds on the running
time
![Page 33: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/33.jpg)
Kinds of analyses
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
Worst-case: (usually) • T(n) = maximum time of algorithm
on any input of size n. Average-case: (sometimes)
• T(n) = expected time of algorithm over all inputs of size n.
• Requires assumption about distribution of inputs.
Best-case: (bogus!) • Cheat with a slow algorithm that
works fast on some input.
![Page 34: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/34.jpg)
Machine-independent time
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
What is Insertion Sort’s worst-case time?
• It depends on the speed of our computer:
• relative speed (on the same machine),
• absolute speed (on different machines).
BIG IDEA:
• Ignore machine-dependent constants.
• Look at growth of T(n) as n .
“Asymptotic Analysis”
![Page 35: Data Structures and Algorithms - Penn State College of …ads22/courses/cmpsc465/S12/lec-n… · · 2012-01-131/12/12 S. Raskhodnikova and A. Smith; based on slides by E. Demaine](https://reader034.fdocuments.net/reader034/viewer/2022051803/5b021bcf7f8b9ad85d8f1878/html5/thumbnails/35.jpg)
Insertion sort analysis
1/12/12
S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson
Worst case: Input reverse sorted.
T(n) = c ( j 1) = cn(n 1) /2 = n2( )j= 2
n
The “ ” notation ignores constants and
“low-order” terms. Defined next lecture.
Is insertion sort a fast sorting algorithm?
• Moderately so, for small n.
• Not at all, for large n.
[arithmetic series]