1 Chapter 3: Efficiency of Algorithms Quality attributes for algorithms Correctness: It should do...

1

Chapter 3: Efficiency of Algorithms

Quality attributes for algorithms Correctness:

It should do things right No flaws in design of the algorithm

Maintainability If we discover it is incorrect, efforts for modifying

and/or extending it must be minimized Central aspect in real-life programs

Understandability Good structure Name things (variables, conditions etc.) based on their

roles in the algorithm/program

2


Elegance Search for intuitive and intelligent solutions Example: Adding 1 to n

1. Get value of n 2. Set Sum to 0 and x to 1 3. While x <= n do

4. Add x to Sum 5. Add 1 to x

End of the loop 6. Print value of Sum 7. Stop

But more elegant is the Gauss method (treated earlier): 1. Get the number n 2. Set m to n+1 3. Set n to n multiplied by m 4. Divide n by 2 5. Print the result n

3


Efficiency How does the algorithm use resources Resources are: time and memory Major computer science concern Memory efficiency:

How much memory is used in processing compared with the size of the input data

Time efficiency: Amount of processing work in dependence of the

size of data input Inherent time (in)efficiency cannot significantly be

influenced by the use of more or less efficient machines

4


The choice of algorithms For the same problem there may be different

algorithmic solutions possible. The algorithm attributes, above all the efficiency

criterion, can be used in order to choose the “best” solution for the respective problem.

Example for a problem with 3 different solutions: A survey of ages of persons in a city is undertaken in

order to get the average age of the population in that city.

Persons who don’t wish to publish their ages write 0 as a fictive age.

Given such a survey, print the average age in the city.

5


Formally: Input: A list of ages including 0’s

E.g. [0, 24, 16, 0, 36, 42, 23, 21, 0, 27] Idea of the solution:

Successively eliminate any 0 in the list. Get the number r of the “real” (no fictive ones, no 0’s) ages. Sum up real ages and divide by r in order to get the average.

Idea of Solution A: Set r to size of the list (here 10). Looking at the list from left to right and pointing with two

fingers. Whenever a 0 is encountered at the position of the left finger,

we copy ALL items to the right of the right finger one cell to the left and we decrement r.

If the left finger points on a real age, both fingers are moved to the right.

6


Example: Initially:

r = 10 [0, 24, 16, 0, 36, 42, 23, 21, 0, 27]

After copying the 9 items (after the right finger): r = 9 [24, 16, 0, 36, 42, 23, 21, 0, 27, 27]

Processing a “real” age pointed to by the left finger: r = 9 [24, 16, 0, 36, 42, 23, 21, 0, 27, 27]

Copy operations needed whenever a 0 is processed: Here: 9 + 7 + 3 = 19 copy operations

7


Algorithm A: Get the n size of the list Set r to n, left to 1, right to 2 While left <= r do

If left points to a non-zero item then Add 1 to left and right

Else Decrease r by 1 While right <= n do

Copy item at position right one cell to the left Add 1 to right

End loop Set right to left+1

End loop Efficiency:

Copying operations -> time efficiency is rather low Only memory for the input is required (+ tiny quantities for

variables like n, left, and right) -> memory-efficient

8

Chapter 3: Efficiency of Algorithms Idea of solution B:

Provide a new (output) list. Read the input from left to right and copy any non-zero

item the output list Algorithm B:

Get n the size of the list Set left to 1 and pos to 1 While left <= n do

If left points to a non-zero item then Copy item to position pos Increment left and pos

Else Increase left by 1

End loop Example:

Input list: [0, 24, 16, 0, 36, 42, 23, 21, 0, 27] Output list: [24, 16, 36, 42, 23, 21, 27]

9


Efficiency of algorithm B: Every location is examined and non-zero ones are copied

-> less copies than solution A (or more time-efficient than A) -> each non-zero element is copied

Extra memory for the output list -> less memory-efficient than A

Idea of Solution C r is set to the size of the list The left finger slides from left to right (beginning from the first

position) The right finger slides from right to left (beginning from the last

position) Whenever a 0 is encountered by the left finger the item of the

right finger is copied to that place and r is decremented The process is repeated until “left” is higher than “right”

10


Example: Initially:

r = 10 [0, 24, 16, 0, 36, 42, 23, 21, 0, 27]

After copying the 27 to the first position and decreasing right and r:

r = 9 [27, 24, 16, 0, 36, 42, 23, 21, 0, 27]

Now left increases until 0: r = 9 [27, 24, 16, 0, 36, 42, 23, 21, 0, 27]

In turn a copy and a decrease of both r and right: r = 8 [27, 24, 16, 0, 36, 42, 23, 21, 0, 27]

11

Chapter 3: Efficiency of Algorithms Algorithm C

Get n the size of the list Set r to n, left to 1, right to n While left < right do

If item of left is not 0 then increase left by 1 Else

Decrement r Copy item of right to position of left Decrement right

End loop If item of left is 0 then decrement r

Efficiency: Fewer copies than B, number of copies is number of zeros No memory requirement like B

Comparison in our example: A: Too many copies, no extra memory requirement B: Fewer copies than A, more memory requirement C: Fewer copies than B (few 0’s !), no extra memory

requirement C “seems” to be the best choice !

12

Chapter 3: Efficiency of Algorithms Measuring Efficiency

What is really a “good” algorithm for a given problem? Algorithms A, B, and C have the same result but do things

differently. How to judge algorithms objectively? Algorithm analysis: Time and memory complexity

Sequential search (telephone book) 1. Get values of N, n, N1, …, Nn, T1, …, Tn 2. Set I to 1 and mark N as not yet found 3. Repeat 4 through 7 until either N is found or I>n

4. If N = Ni then 5. Print phone number Ti 6. Mark N as found

Else 7. Increment I

8. If N is not marked as found then Print ‘Sorry, could not find the name’

13


Good efficiency criterion in this case is: counting the number of comparisons.

Best case: N is in the first place only 1 comparison

Worst case: N is in the last place or it does not exist at all n comparisons

Average case: N is somewhere in the middle of the list About n/2 comparisons

Example: NY city’s population 20 millions (n), computer can do say 50000

comparisons/second In average: 3.33 minutes to find the phone number (n/2*1/50000) Worst case: When the name is not in the book almost 7 minutes are needed!

Memory: Space for the list is required (n) Additional cells for variable like I, and n memory-efficient

14


Algorithm efficiency is measured using the size of input (n) as parameter

Order of Magnitude E.g. sequential search:

worst case: n comparisons c peripheral work per comparison (e.g. adding a variable) worst case is c*n (in this case rather 2*n)

Difference between n and 2*n 1 2 2 4

3 6 and so on

Graph of 2*n in dependence of n Graphs of c = 1, c = ½, and c = 2 in comparison All of them follow the basic straight line shape

15


Order of magnitude f(n) (-> written O(f(n)) ) Any function g(n) that follows the basic shape of f(n)

Example: Sequential search needs at least n comparisons

-> O(n) in both average and worst case Another problem: A city has 4 districts

Phone calls between districts are kept in a table

1 2 3 41 333 33 45 562 44 12 89 2223 567 47 11 1234 35 68 99 59

16


Process the information in the table (e.g. print out content) Algorithm

For each row 1 to 4 do For each column 1 to 4

Print the entry in this row and column

Number of print operations: 4 * 4 = 16

Similarly: Given n districts, number of print operations would be:

n * n = n2 operations Work does not grow in the same rate as the input (like

sequential search) It grows at a rate equal to the square of the input

17


An algorithm having c*n2 amount of work to do is said to be of order of magnitude n2 or O(n2) algorithm.

In our example c = 1 Graphs of c*n2 for different values of c c*n2 is different in shape from c*n We can ignore the actual value of c

Example from real life: Driving is faster than walking, running, and biking Driving has another order of magnitude!

Graph of n and n2 in comparison Computers often solve problems with large problem sizes (n is

very high) Humans are often able to solve problems of small problem

size, they don’t really need a computer therefor.

18

Chapter 3: Efficiency of Algorithms Suppose you have 2 solutions for the same problem:

A: O(n) B: O(n2)

For large n, the O(n) algorithm will always perform better even if the constant factor c is high

Compare: 0.0001*n2 and 100*n In the long run: 100*n will perform better than 0.0001*n2

-> constants do not affect the inherent (in)efficiency of algorithms

-> constants may play a role for small n only Important rule when designing algorithms

No assumptions about the size of the problem !!! If any, this should be documented, because any

assumption may affect the efficiency tremendously

19


Machine speed does not help avoid inefficiency of algorithms! Example:

Mac PowerPC 601, price $2000, speed = 15 million operations /sec Cray T3D, price $31 millions, speed = 300 billion operations /sec Let us give the Mac an O(n) and the Cray an O(n2) algorithm,

respectively Result:

n Mac O(n) Cray O(n2)15.000 0.001 sec 0.00075 sec150.000 0.01 sec 0.075 sec15.000.000 1 sec 12.5 min150.000.000 10 sec 20.83 hours

-> The difference in order of magnitude has slowed down the very powerful Cray machine (for large n)

Were the two algorithms of same order of magnitude, were the Cray better in performance

20


Sequential search: O(n) What is if the list is alphabetically ordered?

Example: [Adam, Catherine, David, John, Nancy, Peter, Thomas]

Searching for Peter? Sequential search: 6 operations needed Why not begin in the middle of the list (like searching in a

telephone book) and then branch to the right or the left depending on the name to be looked for?

1. Step: Compare Peter with John failure 2. Step: Branch to the right, and compare Peter with Peter

Success! Only two comparison operations are needed (sequential

search needed 6) In general: O(log(n)) because the length of the list is

successively divided by 2. Logarithmic search (binary search) is more efficient than

sequential search.

21

Chapter 3: Efficiency of Algorithms Sorting:

Direct sort: O(n2) More efficient sort: O(n*log(n))

Difficult Problems: O(2n) Examples:

Given a network of cities connected by direct paths, find the minimum path between two cities.

Search for Hamiltonian cycles in a network of cities. Find the minimum number of bins needed to fill n objects. These problems are “realistically” not solvable, they need

many thousands of years for relative small problem sizes. Remedy:

Heuristics (sub-optimal solutions) E.g. first fit for bin-packing problem

1 Chapter 3: Efficiency of Algorithms Quality attributes for algorithms Correctness: It should do...

Documents

Transcript of 1 Chapter 3: Efficiency of Algorithms Quality attributes for algorithms Correctness: It should do...