AlgorithmsWhat are they?How to analyze?When are they “reasonable”?

The word “Algorithm”

Abu Abd Allah Muhammad al-Khwarizmi

Persian native of Khwarizm about 810 C.E. Originally “algorism”.

Algorithm (dictionary)

“procedure for solving a mathematical problem (as of finding the greatest common devisor) in a finite number of steps that frequently involves repetition of an operation; broadly, a step-by-step procedure for solving a problem or accomplishing some end, esp. by a computer.”

Algorithm (Knuth)A finite set of rules that gives a sequence of operations for solving a specific

type of problem, with five important features:

finite: must always terminate after a finite number of steps

definite: each step precisely defined; actions to be carried out must be rigorously and unambiguously specified for each case

input: zero or more inputs (quantities given to it before the algorithm begins)

output: one or more outputs (quantities with specified relation to inputs)

effective: operations are sufficiently basic that they can in principle be done exactly and in a finite length of time by someone using pencil and paper

Algorithm (short definition)

well-defined sequence of steps guaranteed to terminate

{int n; int i; // no stepsint sum = 0; // 1 initializationcin >> n; // 1 readfor (i=1; i<=n; i++); // 1 init, n tests, n increments{sum =sum + i;} // n additions

cout << sum; // 1 output}

Very simple example

Computational Problems (definition)

Definition 3.1 A Computational Problemspecifies a relation between two sets of finite sequences of symbols.

The domain of a computational problem is a set of questions called the input. The range is a set of answers called the output.

A computational problem, then, describes how questions are related to answers.

Computational Problems (specifics)

The finite sequences are often – but not always – numbers.

Computations are relations, not functions – one input sequence can have multiple output sequences.

Computational Problems (Example 1)

The Sorting Problem:

INPUT: A sequence of n integers <a1, a2, …, an>.

OUTPUT: A permutation <a′1, a′2, …, a′n> of the input such that a′1 ≤ a′2 ≤ … ≤ a′n.

Algorithms for solving the problem:Bubblesort, Mergesort, Heapsort, etc…

Computational Problems (Example 1, continued)

INPUT: <51, 23, 97, 82, 23, 45, 97>

OUTPUT: <231, 232, 45, 51, 82, 971, 972> or

<232, 231, 45, 51, 82, 971, 972> or

<232, 231, 45, 51, 82, 972, 971> or...

Computational Problems (Example 2)

The Searching Problem:

INPUT: A sequence of n integers <a1, a2, …, an>.

OUTPUT: YES if the input sequence is sorted, NO otherwise

Algorithm for solving the problem: You can write one.

Computational Problems (Example 3)

The Computational Problem:

INPUT: The sequence of words in Shakespeare’s Hamlet (that is, the script of the play)

OUTPUT: Cluster analysis or other word analysis of the script

Algorithm for solving the problem:http://www.textanalysis.info/

Computational Problems (Example 4)

The Not-A-Computational Problem:

INPUT: Two real numbers r1 and r2.

OUTPUT: The sum s = r1 + r2.

Algorithm for solving the problem: ???This is not a computational problem! Why?

Computational Problems (for this course)

1. Given an array of N integers, is the array sorted?

2. Given a C++ program of N lines and input file F, will the program go into an infinite loop if run on F?

3. Given a number D, and a map with N cities, with distances between each pair of cities given, can one find a path to visit all N cities of total distance less than D?

Decision Problems (definition)

Definition 3.3 A Decision Problem is a computational problem with output from the set {yes, no}

That is, it is a question where the only two possible answers are “yes” or “no”.

Examples: Does the string “aabb” have length 5?Is the shortest car route from San Francisco to San

Jose less than 25 miles?Do all triangles have three sides?

EfficiencyThe efficiency of an algorithm A with regard to a

resource R is the measure of how much of R is required to solve an instance of A.

We will consider only time and space efficiency in Computational Complexity.

AnalysisAnalyzing an algorithm means predicting the

resources that an algorithm requires.

This is a straight-forward idea, but in complex in practice.

For example…

Analysis (difficulties)

int foo1 (int a, int N){int i, temp;

temp = 1;for (i = 1; i <= N; i++) {

temp = a*temp;}return(temp);

}

int foo2 (int a, int N){if (N == 1) return(a);else return(foo2(a, N/2) *

foo2(a, N/2));}

What do these two programs do? Do they compute the same thing?If so, which program is more efficient? Or are they equally efficient?

AnalysisOnce an algorithm is given for a problem (and

decided to be correct), analysis is used to determine how much in the way of resources (time or space) the algorithm will require. We focus mainly on time in this course. Usually need only a very rough analysis. Upper bounds on time will be important.

Comparing Algorithms We need a theoretical framework upon which

we can compare algorithms. The idea is to establish a relative order among

different algorithms, in terms of their relative rates of growth. The rates of growth are expressed as functions, which are generally in terms of the number of inputs N.

Definition 3.4 : An algorithm is called efficient if it requires at most polynomially many steps (as a function of input size)

Some decision problems with efficient algorithms:

1. Given a list and a key, is the key in the list? 2. Given an array and a quantity q, is q smaller than all

array elements?3. Given a string, is it a palindrome?

Definition 3.5 : For functions f and g defined on the positive integers, f is O(g) if there exist positive integers C and N such that |f(n)| ≤ C |g(n)| for all n ≥ N.

Algorithms analysis:

1. Count the steps in the algorithm2. Find an upper bound on this count

Example 1: Does a list contain a certain element, the key?

Algorithm to solve: Find an element key in a list of n elements, if presentAlgorithm: (1) read(key)(2) element = first element(3) while (element != key) and (element <= last)

(4) element := next element

Analysis: initialization: 2 steps, one read, one assignmentmoving through the list: 2 tests and 1 move, done 1…n times

O = 2+3n = O(n)

Is d = the greatest common divisor of integers m and n?

Euclid’s algorithm for GCD (n,m), n,m positive integers.while m != 0 do // how many times? { r := n mod m // one step

n := m // one stepm := r } // one step

return n

1. Size of input?2. Cost of n mod m?3. Number of iterations?

Time required for an algorithm (I)

Time required for an algorithm (II)

ORDER n =10 n =20 n =50 n =100 n =1000n 10-8 2x10-8 5x10-8 10-7 10-6

n log2 n 3.3x10-8 9x10-8 2.8x10-7 7x10-7 10-5

n2 10-7 4x10-7 20 10-5 .001n3 10-6 8x10-6 20 10-3 1n10 10 2.84 3.1 31.7 3.2x1011

nn log n 2.1x10-6 4.2x10-4 3.88 5.39 2.5x1011

2n 10-6 10-3 1.86 4x1011 4x10282

3n 6x10-5 3.49 2x105 1.6x1029 4x10458

n! 3.6x10-3 77 9.6x1047 3x10139 1.3x102349

Assuming 10-9 seconds per instruction

seconds hours weeks years centuries

A Comparison of Growth-Rate Functions (cont.)

WE CAN AGREE THAT SOME OF THESE ARE NOTREASONABLE?

THAT IS, INTRACTABLE?

Definition 3.8: A problem is tractable (or feasible) if it has an algorithm of O(p(n)) for some polynomial p(n) in n, the length of the input, that is, there is a constant K so that no input of size n requires more than Kp(n) steps. The class Pwill be the set of decision problems that have O(p(n)) algorithms, for a polynomial p.

Definition 3.9: An problem is intractable (or infeasible) if it has no algorithm of O(p(n)) for any polynomial p(n).