CES 512 Theory of Software Systems

B. Ravikumar (Ravi) Office: 141 Darwin Hall

664 3335E-mail: ravi.kumar@sonoma.edu

Course Web site: http://linux.cs.sonoma.edu/~ravi/ces512su04

Course Goals:

• develop critical thinking about problem solving (in the context of software design)• emphasis on innovation, avoiding dull and routine line of thinking whenever possible.• become exposed to interesting problems (some for pure fun, others for purpose).• focus on models – understanding the context of a problem: specialization, generalization.• tools to create algorithmic solutions, implement and test etc.

Context of the course in Master’s program MS-CES

• I created this course mainly because I felt that the students who want to do a thesis in Software Design need to know this course material. (More specifically, the thesis topics I would like to supervise will require background similar to this.) • The emphasis (creative thinking, awareness of current trends and developments etc.) is based on my belief that one can excel only by consciously demanding high expectation of him/herself. • Emphasis on theoretical ideas and models. But at every stage, we will examine the relevance of the model to practical applications.

Precursors to this course at other institutions

• Algorithms in the Real-World (CMU)• Applied Algorithms (University of

Washington).

Algorithm Engineering.

International conferences and workshops in which modeling, theory (mathematical analysis) as well as implementations and engineering issues are given due emphasis

Course Goals:• tools for algorithm design

• divide and conquer, induction• graph searching, network flow, matching• linear programming• dynamic programming• scheduling and other optimization problems• heuristics and NP-completeness

• tools for analysis• recurrence relations, summation of series• approximations, inequalities • reductions between problems• experimental analysis

• algorithmic modeling • case studies

• formal systems and their applications• logic, grammars, automata etc.

Goals for today’s lecture

• Course outline• Discuss Course work

• homework • quiz • tests, final exam • course project

• Some examples of programming/algorithm design problems • Cover Chapters 1-3 of the text.

Algorithm Design and Analysis• Efficient algorithms are a crucial part of software design

• tinkering with implementation and coding details can’t get us very far. root of the problem may be an inefficient algorithm.• optimization problems allow multiple “correct” solutions, but only one correct and optimal solution.• correctness of algorithms can be tricky to establish.• predicting the behavior of algorithm is an important part of software design.

• Some standard approaches to systematic algorithm design

• identify commonly occurring problems (such as sorting, searching, multiplication, breadth-first search etc.) and design the most efficient algorithms for them.• devise general-purpose techniques (induction, divide and conquer, backtracking, dynamic programming etc.) and study when they are effective by looking at many examples.• create and study models (such as trees, graphs, probabilistic algorithms) that can be used to represent problems and solutions.

• Analysis of algorithms• time taken to solve a problem as a function of problem size usually behaves as a well-defined function (logarithmic, linear, quadratic, n log n, exponential, etc.)• when the time complexity function is complicated, it can be approximated to a well-defined function.• problems can be classified based on the time complexity into easy (polynomial time solvable), provably intractable (not polynomial time solvable) and hard (not known to be polynomial time solvable, and not likely to be so).

Sample Problems

1. How to compress a given text? 2. How to enhance the quality of a digital

photograph ? 3. How is a program like mapquest

implemented? 4. GRE recently introduced a computer

test that adaptively chooses questions based on how well the candidate has performed thus far. How should the questions be selected to make the test fair and reliable?

5. Based on the performance of N teams playing against each other, we want to rank the teams. How should we go about this process?

Course overview:• Mathematical preliminaries

• induction• summation, approximation, estimation• inequalities, upper and lower bounds• O, and notation• recurrence relations• review of discrete math

• Sorting and related problems• quadratic time sorting algorithms• heap sort and merge sort• quick sort• selection problem• non-comparison based algorithms

• Searching• binary search tree• AVL trees• application to a geometric problem• hashing

• Design Techniques• divide and conquer• dynamic programming• greedy method• probabilistic method• backtracking• genetic algorithms, neural networks etc.(?)

• Graph Problems• basic definitions and models• DFS, BFS and applications• minimum spanning tree• shortest path problem• matching in a bipartite graph• Network flow

• Linear Programming • simplex method• duality • applications

• Heuristics• genetic algorithms • simulated annealing

• Applications • cryptography • bioinformatics

Insertion Sorting• Efficient algorithm for sorting a small number of elements. Insert(A,i){// pre-condition: A[0 .. i-1] is sorted. //post-condition: A[0 .. i] is sorted. temp = A[i]; while (A[i-1] > temp) {A[i] = A[i-1]; i--;} A[i] = temp;}

InsertionSort(A){ for (j = 2; j<= A.size; ++j) Insert(A,j)}

• Our pseudo-code style is a little different from that of the text. (Ours will be a lot closer to Java or C++.)• Analysis of InsertionSort:

• Focus on a specific operation. (key comparison). Text considers every operation.• Measured as a function of parameter(s). In this case, it is the size of the array.• Cost of the operation is a constant (1).• Cases of interest:

• worst-case• best-case• average-case

Insert(A,i){// pre-condition: A[0 .. i-1] is sorted. //post-condition: A[0 .. i] is sorted. temp = A[i]; while (A[i-1] > temp) {A[i] = A[i-1]; i--;} A[i] = temp;}

worst-case: i best-case: 1 average: will do later.InsertionSort(A){ for (j = 1; j<= A.size; ++j) Insert(A,j)} worst-case: n(n-1)/2 best-case: n

Review of background materialMathematical Preliminaries• induction- proof technique for assertion of the form “ for all integer n, ... ” Example 1: For all n, n3 + 5n is divisible by 6. Example 2. 1 + r + … + rk = (rk+1 – 1)/(r-1)Proofs will be presented in class

Inequalities, O, and notation• For any positive real constants c and d, cn > nd for all sufficiently large n.• There are positive constants c1 and c2 such that for all n, c1 n log n <= log n! <= c2 n log n.Definition: Let f(n) and g(n) be such that f(n) <= c g(n) for all large enough n. Then, we say f(n) = O(g(n)). • If <= is replaced by >=, we have f(n) = (g(n)). • If both hold, f(n) = (g(n)).

From previous slide, we have: log n! = (n log n)Exercise:1) 1 + 1/2+ 1/3 + ... + 1/n = (log n)

2) 1 + 1/22 + 1/32 + ... + 1/n2 = (1)

3) Estimate 1/2 + 2/22 + 3/23 ... + n/2n

Summation formulas for arithmetic, geometric and arithmetic-geometric series