Villanova University Department of Computing Sciences
CSC 8301 – Design and Analysis of Algorithms:
Introduction
Professor Henry CarterFall 2016
Villanova University Department of Computing Sciences
Greatest Common Divisor
• Problem: Find gcd(m,n), the greatest common divisor of two nonnegative, not both zero integers m and n
• Ex.: gcd(60,24) = gcd(10,50) = gcd(47,0) =
Villanova University Department of Computing Sciences
Euclid’s Algorithm
• Euclid’s algorithm is based on repeated application of this equality
• gcd(m, n) = gcd(n, m mod n)
• Ex.: gcd(60,24) =
• while n ≠ 0 do r ← m mod nm← n n ← r return m
Villanova University Department of Computing Sciences
What is an algorithm?
• An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.
• Muhammad ibn Musa al-Khwarizmi – 9th century Islamic mathematician
Villanova University Department of Computing Sciences
What is an algorithm?
problem
“computer” input
algorithm
output
Villanova University Department of Computing Sciences
This course ...
• This course is a theory course covering basic algorithm analysis and design principles. We will investigate a series of specific algorithms designed to solve common problems and consider their broader applicability. Topics will include:
‣ analysis of recursive and nonrecursive algorithms; design techniques: brute force, greedy, divide-and-conquer, dynamic programming, backtracking, branch-and-bound, space and time tradeoffs; NP-completeness; approximation algorithms; computational problems: sorting, searching, string processing, graphs, arithmetic, linear algebra.
6
Villanova University Department of Computing Sciences
Who cares about algorithms?
• Computer Scientists
‣ Foundational theory for the field
• Computer Engineers/Developers
‣ Significant impact on application performance and correctness
• Logistical planning in general
‣ Scheduling? Sorting? Routing? Puzzles and games?
Villanova University Department of Computing Sciences
You need to understand ...
• Basic programming and data structure concepts
• Discrete math and logic
• Logical proofing techniques
8
Villanova University Department of Computing Sciences
Should you take this course?
• Students without the necessary background.
• Students who are not willing to put forth significant effort.
• Students who are not interested in learning.
9
Villanova University Department of Computing Sciences
Goals
• My goal: to provide you with the tools to analyze algorithms and use common techniques to build programs.
‣ Speed/Space/Correctness/Optimality analysis
‣ Common techniques for search, sort, etc.•This is going to be a hard course. The key to success is
sustained effort. Failure to keep up with readings and homework will likely result in poor grades, and ultimately little understanding of the course material.
•Pay-off: A strong knowledge of algorithm design is among the most desirable abilities for anycomputer scientists
•Technical interviews test precisely this!!!
10
Villanova University Department of Computing Sciences
Course Materials
• Website - I am maintaining the course website at
‣ http://www.csc.villanova.edu/~carterh/Courses/csc8301/f16/
• Course assignments, slides, and other artifacts will be made available on the course website.
• Course textbook
‣ Anany Levitin, Design and Analysis of Algorithms, 3rd edition
11
Villanova University Department of Computing Sciences
Course Calendar
• The course calendar as all the relevant readings, assignments and test dates
• The calendar page contains electronic links to assignments and online papers assigned for course readings.
• Please check the website frequently for announcements and changes to the schedule. Students are responsible for any change on the schedule (I will try to make announcements in class).
12
6/20/2016 CSC 8301 - Design and Analysis of Algorithms - Professor Henry Carter
file:///Users/carterh/Sites/csc8301/f16/schedule.html 1/1
COURSE CALENDAR
Below is the calendar for this semester course. This is the preliminary schedule, which may need to be alteredas the semester progresses. It is the responsibility of the students to frequently check this webpage forschedule, readings, and assignment changes. As the professor, I will attempt to announce any change to theclass, but this webpage should be viewed as authoritative. If you have any questions, please contact me(contact information is available at the course homepage).
Date Topic Assignments Due Readings/Discussions (do readings before class) Slides
08/25/16 Introduction Course syllabus (link)
Textbook, chapter 1
Assignment #0 Introductions
08/30/16 Algorithm Analysis Assignment #0 Textbook, Chapter 2.12.2
09/01/16 Nonrecursive Analysis Textbook, Chapter 2.3
09/06/16 Recursive Analysis Textbook, Chapter 2.42.5
09/08/16 Bruteforce Textbook, Chapter 3.13.2
09/13/16 Exhaustive Search Textbook, Chapter 3.4
09/15/16 Tree Searching Textbook, Chapter 3.5
09/20/16 Insertion and Topological Sort Textbook, Chapter 4.14.2
09/22/16 Binary Search Textbook, Chapter 4.4
09/27/16 Variable Decrease Algorithms Textbook, Chapter 4.5
09/29/16 Merge and Quick Sort Textbook, Chapter 5.15.2
10/04/16 Further Divideandconquer Textbook, Chapter 5.35.5
10/06/16 Midterm Exam10/11/16 Fall Break No Class10/13/16 Fall Break No Class10/18/16 Presorting and Gaussian Elimination Textbook, Chapter 6.16.2
10/20/16 Balanced Search Trees Textbook, Chapter 6.3
10/25/16 Representation Change Textbook, Chapter 6.46.5
10/27/16 Problem Reduction Textbook, Chapter 6.6
11/01/16 String Matching Textbook, Chapter 7.17.2
11/03/16 Dynamic Programming Textbook, Chapter 8.18.2
11/08/16 Dynamic Programming Textbook, Chapter 8.38.4
11/10/16 Prim's and Kruskal's Algorithms Textbook, Chapter 9.19.2
11/15/16 Dijkstra's and Huffman's Algorithm Textbook, Chapter 9.39.4
11/17/16 Iterative Improvement Textbook, Chapter 10.110.2
11/22/16 Iterative Improvement Textbook, Chapter 10.310.4
11/24/16 Thanksgiving No Class11/29/16 Lower Bound Arguments and Decision Trees Textbook, Chapter 11.111.2
12/01/16 P, NP, NPComplete and Numerical Algorithms Textbook, Chapter 11.311.4
12/06/16 Backtracking and Branch and Bound Textbook, Chapter 12.112.2
12/08/16 Approximating NPHard Problems Textbook, Chapter 12.3
12/17/16 Final Exam 8:00am 10:50am
Villanova University Department of Computing Sciences
Grading
• Grading in this class will be distributed as follows:30% Mid-term Exam 40% Final Exam25% Quizzes5% Class Participation
• You need to average above 50% across both exams to pass
• You get the grade that you earn, so be surethat you earn a grade you like.
13
Villanova University Department of Computing Sciences
Homework
• Homeworks are designed to be difficult
‣ Expect to sweat
‣ Preparation for the exams
• Collaboration is allowed
‣ Talk to your fellow students and me!
14
Villanova University Department of Computing Sciences
Readings
• There are a large amount of readings in this course covering various topics. These assignments are intended to:
‣ Support the lectures in the course (provide clarity)
‣ Augment the lectures and provide a broader exposure to security topics.
• Students are required to do the reading!
• The material covered in class is not enough to fully understand the complex concepts of algorithm design and analysis. The reading is necessary to gain a complete understanding of the material and to do well on the homework and exams!
15
Villanova University Department of Computing Sciences
Quizzes
• 10 minute quizzes will take place every Friday
• Your 10 best quiz grades will determine 25% of your final grade
• These quizzes will be quick, so practice beforehand is critical to success
Villanova University Department of Computing Sciences
A Note on Academic Integrity
• As scientists and engineers, we must trust each other to make progress.
• Numerous examples exist to show the consequences of this breakdown.
‣ Jan Hendrik Schon...
• Academic dishonesty, whether from cheating, copying, or through any other dishonest practice will not be tolerated.
‣ I take this very personally - you should too.
17
Villanova University Department of Computing Sciences
Assignment 0 - Introduction
• Your first assignment is to write a one-page summary about yourself. Details can be found on the course webpage.
• A PDF of your write up is due on Blackboard on Monday.
18
Villanova University Department of Computing Sciences
Analyzing an algorithm
• Efficiency
• Correctness
‣ Precision
‣ Accuracy
• Optimality
Villanova University Department of Computing Sciences
Efficiency
• Speed efficiency
‣ Clock time
‣ Processor cycles
‣ Operation count
• Space (storage) efficiency
‣ Bytes of memory
Villanova University Department of Computing Sciences
Example: sequential search
• Input: a list of items and a keyOutput: the item matching the specified keyThink: Names and grade records
• Algorithm: Check each name in order through the entire list of names. Return either the matching record or “not present”
• What impacts the speed most?
Villanova University Department of Computing Sciences
Example: Euclid’s algorithm
• Euclid’s algorithm is based on repeated application of equality
• gcd(m, n) = gcd(n, m mod n)
• Ex.: gcd(60,24) =
• while n ≠ 0 do r ← m mod nm← n n ← r return m
Villanova University Department of Computing Sciences
Input size
• Does measuring strictly input size tell us anything?
‣ How does this map to speed?
‣ How does this compare across algorithms?
• Input size impacts speed, doesn’t directly determinespeed
Villanova University Department of Computing Sciences
Counting Operations
• Basic operation count determines speed
• Exact formula:
‣ e.g., C(n) = n(n-1)/2
• Formula indicating order of growth with specific multiplicative constant:
‣ e.g., C(n) ≈ 0.5 n2
• Formula indicating order of growth with unknown multiplicative constant:
‣ e.g., C(n) ≈ cn2
Villanova University Department of Computing Sciences
Additional Examples
Problem/algorithm Input size measure Basic operation
Search sequentially for a key in a list of n items Number of items n Key comparison
Multiply two n× nmatrices of real numbers
Matrix order n ornumber of elements 2n2
Multiplication of two real numbers
Primality checking of given integer n
#bits in n’s binary repres.b = log2(n) 1=
log2(n+1)Integer division
Graph traversal #vertices (and edges) Visiting a vertex
Villanova University Department of Computing Sciences
Orders of growth
• The relation of basic operations to size of input determines how efficient an algorithm performs as n grows
• Basically: does increasing the input size increase the operation count a LOT or a little?
Villanova University Department of Computing Sciences
Common orders of growth
1 – constant
log n – logarithmic
n – linear
n logn – “n-log-n” (linearithmic)
n2 – quadratic
n3 – cubic
2n – exponential
n! – factorial
Villanova University Department of Computing Sciences
Example values
n log2n n n log2n n² n³ 2ⁿ n!
10 3.3 101 3.3⋅10 102 103 103 3.6⋅106
102 6.6 102 6.6⋅102 104 106 1.3⋅1030 9.3⋅1015
103 10 103 10⋅103 106 109
104 13 104 13⋅104 108 1012
105 17 105 17⋅105 1010 1015
106 20 106 20⋅106 1012 1018
Villanova University Department of Computing Sciences
Basic Analysis Framework
• Efficiency based on input size and “basic operation count”
• This framework applies to both speed and storage analysis
• One assumption we are missing…
Villanova University Department of Computing Sciences
Example: sequential search
• Input: a list of items and a keyOutput: the item matching the specified keyThink: Names and grade records
• Algorithm: Check each name in order through the entire list of names. Return either the matching record or “not present”
• What operation do we count?
• What conditions impact the speed?
Villanova University Department of Computing Sciences
Case-wise efficiency
• Best case: what is the absolute fastest efficiency for a good input size n?
• Average case: out of several executions, what is the average efficiency?
• Worst case: what is the absolute slowest efficiency for a bad input of size n?
• Amortized efficiency: does repeating the algorithm many times reduce the per execution speed?
Villanova University Department of Computing Sciences
Example: sequential search
• Best case: 1
• Average case: n/2
‣ With one caveat
• Worst case: n
Villanova University Department of Computing Sciences
Recap
• Time and storage are most critical
• We measure efficiency in basic operation counts with respect to the input size
• Always consider: how fast does the speed/storage requirement grow (i.e. order of growth)?
• Different “qualities” of inputs affect the efficiency
Top Related