AaDS - W01b Foundations.pdf

7/27/2019 AaDS - W01b Foundations.pdf

http://slidepdf.com/reader/full/aads-w01b-foundationspdf 1/23

AaDS 2010/2011

Fundamental principles of

algorithms analysis Algorithms

and Data Structures



AaDS 2010/2011

Algorithm definitions

• A step-by-step problem-solving procedure, especially an established,recursive computational procedure for solving a problem in a finite number of steps. (The American Heritage dictionaries)

• An algorithm is a set of precise (i.e., unambiguous) rules that specify how tosolve some problem or perform some task (LINFO).

• Properties of algorithms generally include: input/output (i.e., receives input

and generates output), precision (each step is precisely stated),determinism (the intermediate results of each step of execution are uniqueand are determined only by inputs and results of the previous step),termination (ends after a finite number of instructions execute),correctness (the output generated is correct) and generality (eachalgorithm applies to a set of inputs). (LINFO)

• Informally, an algor i thm is any well-defined computational procedure that

takes some value, or set of values, as i nput and produces some value, or set of values, as output . An algorithm is thus a sequence of computationalsteps that transform the input into the output (Cormen at al.)



AaDS 2010/2011

Algorithms

• Are in every computer program. But also in anynon-computer procedure: – Instruction of tools

– Recipe (for food, for medicine)

– Technology – Calculation in science

• Can be expressed as: – Colloquial language

– Code in known programming language – Pseudocode

– Diagrams

– Mix of above



AaDS 2010/2011

Assumptions

• Problems – decidable , undecidable

– decision problems, computational (function) problems

• Algorithms

– correct : An algorithm is said to be correct if, for every inputinstance, it halts with the correct output. We say that a correctalgorithm solves the given computational problem.

– incorrect : An incorrect algorithm might not halt at all on someinput instances, or it might halt with an answer other than thedesired one. Contrary to what one might expect, incorrectalgorithms can sometimes be useful, if their error rate can becontrolled

– f in i te runt ime



AaDS 2010/2011

Fundamental notions

• elementary operation (step): – algebraic operation (addition, substraction, …)

– comparison (<, <=,…)

– coping, assigment (a=b)

• size of problem (n) – number of input data: – theory: number of bits

– real number: number of numbers to sort

– more useful number: size of matrix

– more than one number: for graph• complexity of problem/algorithm is represented

as a function of n, i.e. f (n)



AaDS 2010/2011

The fact

• The simplest algorithm for solving a

problem is often not the most efficient.

Therefore, when designing an algorithm,

do not settle for just any algorithm thatwork.


http://slidepdf.com/reader/full/aads-w01b-foundationspdf 7/23 AaDS 2010/2011

Improving algorithm – an example

• How to calculate nk ? (k in R)

• Simple – from definition:

Power(n,k)Result=1;

While(k>0)

{

Result=Result*n;

k--; // k=k-1

}return Result;

k multiplications

n0=1

nk=n*n(k-1) for k>=1



nk – better algorithm (1)

• n89=n(n44)2, n44=(n22)2, n22=(n11)2,

n11=n(n5)2, n5=n(n2)2, n2=n*n

n2k=(nk)2

n2k+1=n(nk)2

Power(n,k){

if(k==0) return 1;

if(k==1) return n;

result=Power(n,k/2);

if(k%2==0)

return result*result;else

return n*result*result;

}2*log(k) multiplications



nk – better algorithm (2)

• n89=n64*n16*n8*n1=(((((n2)2)2)2)2)2)*(((n2)2)2)2

*((n2)2)2*n

101100222289 0346

nnnnnn

Power(n,k)

{ Result=1;

pow=n;

while(k>0)

{

if((k%2)==1)

Result*=pow;pow*=pow;

k/=2;

}

return Result;

}

2*log(k) multiplications



AaDS 2010/2011

Asymptotic Notation – Big Theta

For a given function g(n), we denote by Θ(g(n)) the set of functions f(n) having the property that there exists positive

constants c 1,c 2 , and n0 such that for all n>=n0 ,

c 1g(n) ≤ f(n) ≤ c 2 g(n)

nn0

f(n)

c1g(n)

c2g(n)

f ( x ) has o rder g ( x )

Θ determines

an equivalence

relation



Big Theta - example

• f(x)=x2+100x+1000

• f(x)= Θ (x 2 ) , because:

– for n=1000,c 1=1, c

2 =10 we have for x>=n:

• x2+100x+1000>x2

• x2+100x+1000<10x2

(1.000.000+100.000+1000<10*1.000.000)

AaDS 2010/2011



AaDS 2010/2011

Asymptotic Notation – Big Oh

For a given function g(n), we denote by O(g(n)) the set of functions f(n) having the property that there exists positive

constants c and n0 such that for all n>=n0

f(n) ≤ c∙g (n)

nn0

f(n)

cg(n)



AaDS 2010/2011

Asymptotic Notation – Big Omega

For a given function g(n), we denote by (g(n)) the set of functions f(n) having the property that there exists

positive constants c and n0 such that for all n>=n0 ,

c∙g(n) ≤ f(n)

n

n0

f(n)

cg(n)



AaDS 2010/2011

Dependences

O n( )2

( )n2

( )n2

55 nn nlog

2 3n

6 32n6 2n n n nlog

n2

n n

4 32 n

n n!

Functions that havea smal ler o rder than

n2

Functions that have

a l arger o rder than

n2



AaDS 2010/2011

Complexity of problem/algorithm

• time complexity of a problem is the number of

steps that it takes to solve an instance of the

problem as a function of the size of the input ,

using the most efficient algorithm• space complexity of a problem is a related

concept, that measures the amount of space, or

memory required by the algorithm

• time complexity and space complexity can be

considered for a chosen algorithm.



AaDS 2010/2011

Hierarchy of orders – complexity classes

O( )1

O n(log )

O n( )

O n( )

O n n( log )

O n( )2

O n( )3

O n( )2

O nn( )2

O n( !)

O nn

( )

NP Problems

NP-complete problems

P Problems



AaDS 2010/2011

Relationships

• transitivity

• reflexivity

• symmetry

• transitive symmetry

))(()(implies))(()(i))(()(

))(()(implies))(()(i))(()(

))(()(implies))(()(i))(()(

nhn f nhn g n g n f

nhn f nhn g n g n f

nhn f nhn g n g n f

f n f n

f n f n

f n f n

( ) ( ( ))

( ) ( ( ))

( ) ( ( ))

))(()(if onlyandif ))(()( n f n g n g n f

))(()(if onlyandif ))(()( n f n g n g n f



AaDS 2010/2011

Relationships (cont.)

• Others property

nm = O(nk ), where m ≤ k

O(f (n))+O(g (n)) = O(|f (n)|+|g (n)|)

c ∙O(f (n)) = O(f (n)), where c is constant

O(O(f (n)) = O(f (n))

O(f (n))∙O(g (n)) = O(f (n)∙g (n))

O(f (n)∙g (n)) = f (n)∙O(g (n))



AaDS 2010/2011

Problems classification• P problems

– Computation complexity is at most

polynomial

• sorting

• matrix multiplication

• shortest path

• NP problems – Computation complexity is at least

exponential

• simplex method

• monk’s puzzle problem

• NP-complete problems• boolean satisfiability problem (SAT)

• Hamilton’s cycle

• set division

P

NP

NP-complete



AaDS 2010/2011

Complexity comparisonComplexity

function

Size n

10 20 30 40 50 60

n 0,00000001s 0,00000002s 0,00000003s 0,00000004s 0,00000005s 0,00000006s

n2 0,0000001s 0,0000004s 0,0000009s 0,0000016s 0,0000025s 0,0000036s

n5 0,0001s 0,0032s 0,0243s 0,1024s 0,3125s 0,7776s

n10

10s 2,8h 6,8 days 121,4

days

3,1 years 19,2

years

2n 0,0000001s 0,001s 1s 18,3 min 13 days 36,6

years

3n 0,000006s 3,5s 2,4 days 385

years

2*107

years

1,3*1012

years

n! 0,003s 77 years 8*1015

years

2*1032

years

9*1047

years

2*1065

years

age of universe = 13,7*109 years



AaDS 2010/2011

Complexity comparison (cont.)Complexity

function

Size n

10 100 1000 10000 100000 1000000

n 0,00000001s 0,0000001s 0,000001s 0,00001s 0,0001s 0,001s

nlog2n 0,00000003s 0,0000006s 0,00001s 0,0001s 0,0016s 0,02s

n2 0,0000001s 0,00001s 0,001s 0,1s 10s 16,6min

n3 0,000001s 0,001s 1s 16,6min 11,5 days 31,7

years



AaDS 2010/2011

Complexity comparison (cont.)Complexity

function

Size of the biggest problem solved during 1h

By current computer By computer

100 times faster

By computer

1000 times faster

n N1 100N1 1000N1

n2

N2 10N2 31,6N2

n3

N3 4,64N3 10N3

n5

N4 2,5N4 3,98N4

2n

N5 N5+6,64 N5+9,97

3n N6 N6+4,19 N6+6,29

AaDS - W01b Foundations.pdf

Documents

Transcript of AaDS - W01b Foundations.pdf