1

Algorithm Analysis

2

Question Suppose you have 2 programs that

will sort a list of student records and allow you to search for student information.

Question: How do you judge which is a better program?Answer: By running programs or Algorithm Analysis

3

Algorithm Analysis Is a methodology for estimating

the resource (time and space) consumption of an algorithm.

Allows us to compare the relative costs of two or more algorithms for solving the same problem.

4

How to Measure “Betterness” Critical resources in computer

Time Memory space

For most algorithms, running time depends on size n of data.

Notation: T(n) Time T is a function of data size n.

5

Two approaches Two approaches to obtaining

running time: Measuring under standard

benchmark conditions (running programs)

Estimating the algorithms performance (analyzing algorithms)

6

Estimation Assumptions Estimation of running time is based

on: Size of the input Number of basic operations

The time to complete a basic operation does not depend on the value of its operands.

7

Example: largest()

// Return position of largest value// in array

int largest(int array[], int n) { int posBig = 0; // 1 for (int i = 1; i < n; i++)

if (array[posBig] < array[i]){ // 2 posBig = i; // 3 } return posBig; }

8

Example: largest() Step 1: T(n) = c1

Step 2: It takes a fixed amount of time to do one comparison T(n) = c2n

Step 3: T(n) = c3n, at most Total time: T(n) = c1 + c2n+ c3n

≈ c1 + (c2 + c3)n ≈ cn

9

Example: Assignment

int a = array[0];

The running time is T(n) = c This is called constant running time.

10

Example: total()

sum = 0;for (i = 1; i <= n; i++) for (j = 1; j < n; j++) sum++;} What is the running time for this code? Analysis:

The basic operation is sum++ The cost of a sum operation can be bundled into

some constant time c. Inner loop takes c1n; outer loop takes c2n. The total running time is T(n) = c1n * c2n

= (c1 + c2)n = c3n2

11

Growth Rate of Algorithm—Big-O Natation Growth Rate for an algorithm

is the rate at which the cost of the algorithm grows as the data size n grows.

Constant: T(n) = c O(1) Linear Growth: T(n) = n O(n) Quadratic Growth: T(n) = n2 O(n2) Exponential Growth: T(n) = 2n O(2n)

Assumptions Growth rates are estimates. They have meaning when n is large.

Growth Rate Graph

12

13

Growth Rate Graph

n

T(n)

c

Log n

14

Characteristics of Growth Rates

Growth Rates Constant— T(n) = c

Independent of n Linear— T(n) = cn

Constant slope Logarithmic— T(n) = c log n

Slope flattens out quickly Quadratic— T(n) = cn2

Increasing slope Cubic— T(n) = cn3

Exponential— T(n) = c2n

Growth Rate—Array Operations

15

bool search(int list[], int n, int key) found = false Loop (for i = 0 to i < n) If (list[i] == key) Then found = true End If End loop return found End // search

Growth Rate—Array Operations

16

• Linear Search• Insert at Back• Insert at Front• Remove from Back• Remove from Font

Linear Search

17

bool search(int list[], int n, int key) found = false // c1 Loop (for i = 0 to i < n) If (list[i] == key) Then // nc2

found = true (at most) End If End loop return found // c3

End // searchT(n) = c1 + nc2 + c3 = c + nc2 ≈ nc2 ∴ O(n)

Insert at Back

18

void insertAtBack(int list[], int &n, item) If (!isFull) Then list[n] = item n++ End IfEnd // insertAtBack

Insert at Back

19

void insertAtBack(int list[], int &n, item) If (!isFull) Then list[n] = item // c1 n++ // c2 End IfEnd // insertAtBack

T(n) = c1 + c2 = c ∴ O(1)

Insert At Front

20

void insertAtFront(int list[], int &n, int item) If (!isFull) Then // shift to right Loop (for i = n downto 1) list[i] = list[i – 1] End loop // assign list[0] = item count++ End IfEnd // insertAtFront

Insert At Front—Growth Rate?

21

void insertAtFront(int list[], int &n, int item) If (!isFull) Then Loop (for i = n downto 1) list[i] = list[i – 1] // c1n (at most) End loop

list[0] = item // c2

count++ // c3

End IfEnd // insertAtFront

T(n) = c1n + c2 + c3 = nc1 + c ≈ nc1 ∴ O(n)

Remove From Back

22

int removeFromBack(list[], int &n) result = NIL If (!isEmpty) Then result = list[n – 1] n = n - 1 End If return resultEnd // removeFromBack

Remove From Front

23

int removeFromFront(list[], int &n) result = NIL If (!isEmpty) Then result = list[0] // shift left Loop (for i = 0 to n – 2) list[i] = list[i + 1] End Loop n = n - 1 End If return resultEnd // removeFromFront

Growth Rate—Linked-List Operations

24

• Linear Search• Insert at Back• Insert at Front• Remove from Back• Remove from Font

Linear Search

25

bool linearSearch(int key){ bool found = false; c1

if (head != NULL){ node* temp = head; c2

while (temp != NULL){ if (temp=>getData() != key) found = true; else temp = temp->getNext(); c3n } } return found;} T(n) = c1 + c2 + c3n ≈ cn ∴ Growth Rate: O(n)

insertAtBack()

26

void insertAtBack(int item){ node *newP = new node(item, NULL); node *temp = head; c1

if (head == NULL) head = newP; else while (temp->getNext() != NULL){ temp = temp->getNext(); c2n } temp->setNext(newP); c3

count++; c4

} T(n) = c1 + c2n + c3 + c4 ≈ c + c2n ∴ G.R. = O(n)

insertAtFront()

27

void insertAtFront(int item){ node *newP = new node(item, NULL); c1

newP->setNext(head); c2

head = newP; Next(newP); c3

count++; c4

}

T(n) = c1 + c2 + c3 + c3 = c ∴ G.R. = O(1)

removeFromBack()

28

void removeFromBack(){ node *temp = head; c1

node *tail = NULL; c2

while (temp != NULL){

trail = temp; nc3

temp = temp->getNext(); nc4 } count++; c5

} T(n) = c1 + c2 + nc3 + nc4 + c5 = c6 + n(c3 + c4) ≈ cn ∴ G.R. = O(n)

removeFromFront()

29

void removeFromBack(){ node *newP = new node(item, NULL); c1

newP->setNext(head); c2

head = newP; Next(newP); c3

count++; c4

} T(n) = c1 + c2 + c3 + c3 = c ∴ G.R. = O(1)

30

Best, Worst, Average Cases

For an algorithm with a given growth rate, we consider Best case Worst case Average case

For example:

Sequential search for K in an array of n integers

Best case: The first item of the array equals K Worst case: The last position of the array equals K Average case: Match at n/2

31

Which Analysis to Use The Worst Case.

Useful in many real-time applications. Advantage:

PredictabilityYou know for certain that the algorithm must perform at least that well.

Disadvantage: Might not be a representative measure of

the behavior of the algorithm on inputs of size n.

32

Which Analysis to Use? The average case.

Often we prefer to know the average-case running time.

The average case reveals the typical behavior of the algorithm on inputs of size n.

Average case estimation is not always possible.

For the sequential search example, it assumes that the key value K is equally likely to appear in any position of the array. This assumption is not always correct.

33

Which Analysis to Use The Best Case

Normally, we are not interested in the best case, because:

It is too optimistic Not a fair characterization of the

algorithms’ running time Useful in some rare cases, where the best

case has high probability of occurring.

34

The moral of the story If we know enough about the

distribution of our input we prefer the average-case analysis.

If we do not know the distribution, then we must resort to worst-case analysis.

For real-time applications, the worst-case analysis is the preferred method.

Your Turn (Stack w/array)

35

Given Stack Class:

…private: elemType data[MAX]; int top;}

Stack (w/array)

36

What is the growth rate (big O) of the push operation?

void push(elemType item){ if (!isFull()){ top++; data[top] = item; }}

Stack (w/array)

37

What is the growth rate (big O) of the pop operation?

elemType pop (){ elemType result = NIL; if (!isEmpty()){ result = data[top]; top--; }}

Stack (w/array)

38

What is the growth rate (big O) of the clear operation?

void clear(){ top = -1;}

Stack (w/linked list)

39

Given: Stack class with linked list

…private: Node *top;}

Stack (w/linked list)

40

What is the growth rate (big O) of the push operation?

void push(elemType item){ Node *temp = new Node(item, NULL); top->setNext(temp); top = temp;}

Stack (w/linked list)

41

What is the growth rate (big O) of the pop operation?

elemType pop () { elemType result = NIL; Node *temp = top; if (!isEmpty()){ result = top-<getData(); top = top->getNext(); delete temp; } return result;}

Stack (w/linked list)

42

What is the growth rate (big O) of the clear operation?

Void clear () { Node *temp = top; while (top != NULL) { top = top->getNext(); delete temp; temp = top; }}

Growth Rate of Binary Search (w/array)

43

Comparisons Number left

0 n1 n(1/2)2 n(1/2)2

3 n(1/2)3

4 n(1/2)4

… …k-1 n(1/2)k-1

k n(1/2)k

Solution n(1k) nn(1/2)k = ------- = ---- = 1 (2k) (2k)2k = n log(2k) = log(n)k log(2) = log(n)k = log(n)/log(2)k = c1 log(n)T(n) = c1 c2 log(n)O(log n)

Analysis of Bubble Sort

void bubbleSort(int a[], int count) pass count – 1 last pass – 1 Loop (for i from 1 to pass) Loop (for j from 0 to last) If (a[j] > a[j + 1]) Then swap a[j] and a[j = 1] End If End Loop last last – 1 End Loop

• Suppose: count = 10maximum pass = 9

pass comparisons1 92 83 74 65 56 47 38 29 1

Analysis of Bubble Sort

• Total number of comparisons 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 = ? 9 + 8 + 7 + 6 + 5 + 4 + 3 + 2 + 1 = ?--------------------------------------------------10+10 +10+……………………+10 +10 = 9x10

• (n-1) + (n-2) + (n-3)… + 3 + 2 + 1 1 + 2 + 3 …+ (n-3) + (n-2) + (n-1) n + n + n … + n + n + n

• = n(n-1)• Therefore, ? = n(n – 1)/2

= cn2 – cn ≈ cn2

• Growth Rate: O(n2)