01 intro to algorithm--updated 2015

Instructor: Sadia Arshid

ANALYSIS OF ALGORITHM

Introduction to Algorithms

ANALYSIS OF ALGORITHM Detailed study of the basic notions of the design

of algorithms and the underlying data structures

Several measures of complexity are introduced

Emphasis on the structure, complexity, and efficiency of algorithms.

Prepare the Students for Analyzing algorithms solving same problems & making them capable of writing optimize algorithms


ANALYSIS OF ALGORITHM Text Book:

The Design And Analysis of Algorithm by Anany Levitin

Fundamentals of Algorithm by Richard Neopolitan

Reference Material:Algorithm Design by Micheal T. Good Rich

Introduction to Algorithms /2E, T. H. Cormen, C. E. Leiserson, and R. L. Rivest, MIT Press, McGraw-Hill, New York, NY, 2001.

Grading policy


Assignments 5%

Quizzes 10%

Project 15%

Midterm 20%

Final 50%

WHAT IS AN ALGORITHM? An algorithm is a set of rules for carrying out

calculation either by hand or on a machine.

An algorithm is a finite step-by-step procedure to achieve a required result.

An algorithm is a sequence of computational steps that transform the input into the output.

An algorithm is a sequence of operations performed on data that have to be organized in data structures.

An algorithm is an abstraction of a program to be executed on a physical machine (model of Computation).


WHAT IS AN ALGORITHM?

An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.


WHAT IS AN ALGORITHM?


Problem

algorithm

“computer”

input output

ALGORITHM

• Problem

• problem instance and solution• Sort list of n numbers in ascending order

• Determine whether x is in list of n numbers

• Algorithm is a general solution which solves all instances of a problem.


• Generality

• Finiteness

• Non-ambiguity

• EfficiencyInstructor: Sadia Arshid

What properties an algorithm should have ?

• The algorithm applies to all instances of input data not only for particular instances

Example:

Let’s consider the problem of increasingly sorting a sequence of values.

For instance:

(2,1,4,3,5) (1,2,3,4,5)

input data resultInstructor: Sadia Arshid

Generality

Method:


Generality (cont’d)

2 1 4 3 5Step 1:

1 2 4 3 5

1 2 4 3 5

1 2 3 4 5

Step 2:

Step 3:

Step 4:

Description:

- compare the first two elements: if there are not in the desired order swap them- compare the second and the third element and do the same…..- continue until the last two elements were compared

The sequence has been ordered

GENERALITY (CONT’D)

• Is this algorithm a general one ? I meant, does it assure the ordering of ANY sequence of values ?


• Answer: NO Counterexample:

3 2 1 4 5 2 3 1 4 5 2 1 3 4 5 2 1 3 4 5

In this case the method doesn’t work, it isn’t a general sorting algorithm

FINITENESS

• An algorithm have to terminate, i.e. to stop after a finite number of steps

Example

Step1: Assign 1 to x;

Step2: Increase x by 2;

Step3: If x=10 then STOP;

else GO TO Step 2

How does this algorithm work ?


FINITENESS (CONT’D)

How does this algorithm work and what does it produce?



Step3: If x=10

then STOP;

else Print x; GO TO Step 2;Instructor: Sadia Arshid

x=1

x=3 x=5 x=7 x=9 x=11

The algorithm generates odd numbers but it never stops !

FINITENESS (CONT’D)

The algorithm which generate all odd naturals smaller than 10:



Step3: If x>=10

then STOP;

else Print x; GO TO Step 2Instructor: Sadia Arshid

NON-AMBIGUITY

The operations in an algorithm must be rigorously specified:• At the execution of each step one have to know exactly

which is the next step which will be executed

Example:

Step 1: Set x to value 0

Step 2: Either increment x with 1 or decrement x with 1

Step 3: If x[-2,2] then go to Step 2; else Stop.

As long as does not exist a criterion by which one can decide if

x is incremented or decremented, the sequence above cannot be

considered an algorithm.


NON-AMBIGUITY (CONT’D)

Let’s modify the previous algorithm as follows:

Step 1: Set x to value 0

Step 2: Flip a coin

Step 3: If one obtains head

then increment x with 1

else decrement x with 1

Step 3: If x[-2,2] then go to Step 2, else Stop.

• This time the algorithm can be executed but … different executions can be different

• This is a so called random algorithm


EFFICIENCY

• The need to design efficient algorithms

• What is efficiency?

• Example

• Sequential search Vs. Binary Search


ALGORITHM ANALYSIS

• How to analyze an algorithm?

• Predicting the resources that the algorithm requires.

• Memory

• Communications Bandwidth

• Logic gates etc

• Most important is Computational Time


ALGORITHM ANALYSIS

• Important thing before analysis

• Model of the machine upon which the algorithms is executed.

• Random Access Machine (RAM) (Sequential)

• Running Time: No. of primitive operations or “steps executed”.


RUNNING TIME

Most algorithms transform input objects into output objects.

The running time of an algorithm typically grows with the input size.

Average case time is often difficult to determine.

We focus on the worst case running time.Easier to analyzeCrucial to applications such

as games, finance and robotics

0

20

40

60

80

100

120

Runnin

g T

ime

1000 2000 3000 4000

Input Size

best caseaverage caseworst case


EXPERIMENTAL STUDIES

Write a program implementing the algorithm

Run the program with inputs of varying size and composition

Use a method in “Time.h” to get an accurate measure of the actual running time

Plot the results

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

0 50 100

Input Size

Tim

e (

ms)


LIMITATIONS OF EXPERIMENTS

• It is necessary to implement the algorithm, which may be difficult

• Results may not be indicative of the running time on other inputs not included in the experiment.

• In order to compare two algorithms, the same hardware and software environments must be used


THEORETICAL ANALYSIS

• Uses a high-level description of the algorithm instead of an implementation

• Characterizes running time as a function of the input size, n.

• Takes into account all possible inputs

• Allows us to evaluate the speed of an algorithm independent of the hardware/software environment


ALGORITHM ANALYSIS

• Running Time

• We need a measure that is independent of computer, programming language and the programmer.

• Concept of basic operation

• In sorting comparison is a basic operation.


ALGORITHM ANALYSIS

• How much time each construct / keyword of a pseudo code takes to execute. Assume it

takes ti (the ith construct)

• Sum / Add up the execution time of all the constructs / keywords. if there are m constructs then


m

iitT

1

ALGORITHM ANALYSIS

• That will be the execution time for a given input size (say n)


m

iin tT

1

Running time as the function of the input size T(n)

ALGORITHM ANALYSIS

• What are the constructs / Keywords.

• Time for each construct

• Total Time

• Total time as a function of input size


ALGORITHM ANALYSIS

• Construct:

• Sequence

• Selection• Iterations• Recursion


ALGORITHM ANALYSIS Sequence Statements: Just add the running time

of the statements If-Then-Else: if (condition) S1 else S2

Running time of the test plus the larger of the running times of S1 and S2.

Iteration is at most the running time of the statements inside the loop, including the times the number of iterations.

Nested Loops: Analyze these inside out. The total Running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the size of all the loops.

Function Calls: Analyzing from inside to out. If there are function calls, these must be analyzed first.


EXAMPLE


Algorithm arrayMax(A, n)Input array A of n integersOutput maximum element of A

currentMax A[0]for i 1 to n 1 do

if A[i] currentMax thencurrentMax A[i]

return currentMax

Example: find max element of an array

COUNTING PRIMITIVE OPERATIONS

By inspecting the pseudocode, we can determine the maximum number of primitive operations executed by an algorithm, as a function of the input size

Algorithm arrayMax(A, n) # operationscurrentMax A[0] 2for i 1 to n 1 do 1 n

if A[i] currentMax then 2(n 1)

currentMax A[i] 2(n 1){ increment counter i } 2(n 1)return currentMax 1

Total 7n 2Instructor: Sadia Arshid

ESTIMATING RUNNING TIME• Algorithm arrayMax executes 7n 2 primitive operations in the

worst case.

• This notation is complex to use. To simplify it we will be using order of growth that is dominating term of expression; which is n (will be discuss in detail later)

• Instead of computing running time of every statement of the algorithm we can further simplify it by calculating running time of basic operation only(Time Complexity) (will be discuss in detail later)

• In This example • basic operation is comparison

• Executed n-1 number of times

• Order of growth is n

• Same as order of growth of computing running time of every statement.Instructor: Sadia Arshid

IMPORTANT PROBLEM TYPES

• Sorting

• Sorting keys

• Stable

• Inplace

• Searching

• String processing


IMPORTANT PROBLEM TYPES

• Graph problems

• Combinatorial Problems

• Geometric Problems

• Numerical Problems


FUNDAMENTAL DATA STRUCTURES Linear data Structures

Arrays

Link list

Stack

Queue

GraphsAdjacency Matrix

Adjacency Link list

Trees

Set & Dictionaries


01 intro to algorithm--updated 2015

Technology

Transcript of 01 intro to algorithm--updated 2015