Prof Saroj Kaushik

Two algorithms may perform the same task but one ismore "efficient" than the other.Efficiency means "use of fewer resources“.What is a resource? Resource includes◦ CPU Cycles (time)◦ Computer Memory (space)

Most "coding tricks" provide insignificantimprovements in efficiency.Our concern is with significant difference inefficiency.Focus must be on the underlying approach to solvingthe problem as opposed to the implementation.Otherwise we might end up with an efficientimplementation of an inefficient algorithm.It is very convenient to classify algorithms based onthe relative amount of time or space they require.

The growth of time /space requirements is specifiedas a function of the input size.There are two notions of complexity expressed as afunction of the size of input:◦ Time Complexity: Running time of the program◦ Space Complexity: Amount of computer memory required

during the program executionTime and space are both important, but we areusually more interested in time efficiency rather thanspace efficiencyThese days - relative memory prices are quite lowcompared to the past.

The performance of program can be affected by◦ the language in which it is written,◦ the machine it is executed on and◦ the data it processes.We confine our analysis to the algorithm itself to avoidabove problems.The time efficiency of an algorithm is analyzed byestimating how many "high level" instructions will getexecuted.It can be expressed as a function of the amount of datait processes.

Algorithm:input n;i = 0;sum = 1;while (i < n) { i = i + 1;

sum = sum + i ;}output fact

The algorithm will require following 3 operationsplus the initial assignments, input and an outputstatements

The time taken by above algorithm will be dependenton the amount of input data ‘n’.Number of statements executed is: 3 * n + 4Here for large value of ‘n’, the extra four instructionsare insignificant.

n = 100 304n = 1,000 3,004n = 10,000 30,004

We can ignore the constant 4 as the differencebetween 3,000 and 3,004 is not really significant.So when dealing with large problem sizes, smalldifferences aren't really an issue.

Interest is to know how quickly the time grows as thesize of the problem grows (called growth rate).The growth rate can usually be expressed as afunction f(n) sometimes called computing time.Algorithm’s growth function is expressed in terms ofthe order of magnitude of the growth rate.For example, if f(n) = 3 * n + 4, then order ofmagnitude is O(n).This is known as "Big O" Notation

Can we ignore low order terms and multiplicativeconstants of an algorithm's growth rate function?◦ Answer YesPerformance-wise for sufficiently large n◦ the following functions are of quadratic growth rates

f(n) = 300 * n2 + 3 * n + 42 andf(n) = 20 * n2

Order is represented using dominant term in thepolynomial.

Consider a polynomial of degree k asP(n) = aknk + ak-1nk-1 + …+ a1n + a0 , where ak ≠ 0

Let f(n) represents growth rate of an algorithmrepresented as a polynomial.Order of growth rate for such algorithm is written as O(nk) Order of growth is also called time complexity of algorithm.

Consider two algorithms performing the same task.◦ For algorithm1

F(n) = 10*n ; Complexity: O(n)◦ For algorithm2

F(n) = n2 / 2; Complexity: = O(n2)Initially, algorithm2 seems to be better for values ofn=1,..,20.But for n > 20, algorithm1 is better.Hence algorithm1 having complexity O(n) is alwaysbetter.

Constant: O(1)◦ for example: f(n) = 300◦ It does not depend on the size of problem.Logarithmic: O(log2n)◦ for example: f(n) = 5 log2(n)+20◦ time to perform algorithm increases in proportion to the

logarithm of the size of the problem.Linear: O(n)◦ for f(n) = 2n +10◦ time to perform algorithm is proportional to the size of the

problem.

Quadratic: O(n2)◦ for example: f(n) = 5n2+100◦ time to perform the algorithm is proportional to the

square of the size of the problemCubic: O(n3) ◦ For example: f(n) = 4n3 + 200N logarithmic: O(n*log2n) ◦ For example: f(n) = n * log2(N) + 10Exponential : O(2n) ◦ For example: f(n) = 300 * 2n◦ time to perform the algorithm is proportional to some

number raised to the size of the problem.

Example: ex series : 1+x+x2/2! +x3/3! +.....+xn/n!Version1 : Complexity : O(n2) (Inefficient)

Fixed number of terms to be added.Each term is independantly computed

input x;i = 0; e =1;while (i < n){ i = i+1

e = e + (x ** i) /fact(i)}print e

ex series : 1+x+x2/2! +x3/3! +.....+xn/n!+...Version2 : Complexity : O(n2) – (Inefficient)

Variable number of terms to be added.Each term is independantly computed

input x;i = 1; e =1; while (term > terminating_val){ term = x ** i /fact(i)

e = e + termi = i+1

}print e

ex series : 1+x+x2/2! +x3/3! +.....+xn/n!+...Version3 : Complexity : O(n) – (Efficient)

Variable number of terms to be added.Each term is computed using previous term. Factorial computation is avoided

input x;i = 1; e =1; term = 1while (term > terminating_val){ term = (term* x) / i

e =e + termi = i+1

}print e

Simple Version of algorithm

input n;i = 2; found = false; while (i

Strategy development:◦ Consider n= 36◦ Complete set of exact divisors:{2,3,4,6,9,12,18}Smaller factor Larger factor

2 183 124 96 6

Notice that smallest exact divisor

If n is even then smallest exact factor = 2If n is odd then no need to check for i= 4,6,8,… √n. Only check for i = 3,5,7,9,.. √n.◦ Complexity : O(n). Most efficient in terms of computing time

input n;if (n mod 2 = 0) then output (2) else{ i = 3; found = false; while (i

Formula: xn = x* x * …. *x n timesinput x, n;i = 1; found = false; power =1;while (i

Inductive definition of power is:

1, if n = 0power(x, n) =

x * power(x, n-1), otherwise

We can write function ‘power’ that contains the following statementIf n=1 then return 1 else return x * power(x, n-1)

Inductive definition for fast method:

1, if n = 0power(x,n) = x * power(x,n-1), if odd(n)

power(x,n/2) * power(x,n/2), if even(n)

Time Complexities : O(log2n)

We know that each integer can be written as a product of prime numbers.

◦ For example, 60 = 2*2*3*5Therefore, each integer n can be written as product of prime numbers such as

n = p1*p2*…*pi, n>1 and p1≤ p1…≤pi Simple approach:◦ Start with prime number 2 and repeatedly reduce n by

factor of 2 until reduced number is not divisible by 2.◦ Repeat this process for next prime number till number is

reduced to 1.n : 120 60 30 15 5 1prime: 2 2 2 3 5

input n;prime =2; quot = n; i=1;while (prime

Input n, m # dimensionsInput a(i, j), i = 1, n; j = 1, mInput b(i, j), i = 1, n; j = 1, mfor i=1 to n{ for j = 1 to m

c(i, j) = a(i, j) + b(i, j) }print c

Consider

1 2 5 1A = 3 4 6 V = 2

3

Ci = ∑ (Aij * Vj, j = 1 to m ), i = to n

C = A * V = 2029

In matrix-vector multiplication, if the matrix is N x M, then the vector must have a dimension, M. In other words, the vector will have M entries. The product of the matrix and the vector is a resultant vectorthat has a dimension of N.

read n, m # dimensions of matricesread A(i, j), i = 1, n; j = 1, mread V(i), i = 1, mfor i=1 to n{ sum = 0

{ for j = 1 to msum = sum +A(i, j) * V(j) }

c(i) = sum}print c

a11 a12 .… a1ma21 a22 …. a2m

a (n X m) an1 an2 … anm

b11 b12 … b1pb21 b22 …. b2p

b (m X p) bm1 bm2 … bmp

c (n X p) = a (n X m) * b (m X p)Each element of product matrix c is computed as:

cij = ∑ (aik * bkj, k = 1 to m), i =1 to n; j = 1 to p

read n, m, p # dimensions of matricesread a(i, j), i = 1, n; j = 1, mread b(i, j), i = 1, m; j = 1, pfor i=1 to n{ for j = 1 to p

sum = 0{ for k = 1 to m

sum = sum +a(i, k) * b(k, j) }c(i,j) = sum}

print c

Algorithm for creating transpose matrix

read nread amatrixfor i=1 to n{ for j=1 to n

{ bmatrix(j,i)=amatrix(i,j) }Print amatrix, bmatrix

Python Program

n = input('Input dim\n')def trans(a,n):

for i in range (n):for j in range (n):

b[i][j] = a[j][i]return b

read(X,n)print X, trans(X,n)

input ninput a(i,j), i=1,n; j=1,nfor i=1 to n

{ for j=i+1 to n{ temp = a(i,j)

a(i,j) = a(j,i)a(j,i) = temp

}print a

n = input('Input dim\n')def trans(a,n):

for i in range (n):for j in range (i+1,n):

temp=a[i][j]a[i][j] = a[j][i]a[j][i] = a[i][j]

return bread(X,n)print trans(X,n)

Efficient Algorithm DesignPerformance Evaluation of AlgorithmContd…Contd…Contd…Example: Sum of first n integers starting from 0Contd…Contd…Contd…Contd…IllustrationVarious order of magnitudesContd..Series SummationSeries SummationSeries SummationSmallest exact divisor of an integer nEfficient Algorithm Further improvementAlgorithm for computing xnComputing nth power of xEfficient Algorithm for computing powerComputing prime factors of an integerPrime Factorization AlgorithmAlgorithm for Adding two matricesMatrix–Vector MultiplicationAlgorithm for Matrix-Vector Multiplication Multiplying two matricesAlgorithm for Multiplying two matricesTranspose of a square matrixEfficient version of Transposing a square matrix (same matrix gets transposed)