CS 2210 Discrete Structures
Transcript of CS 2210 Discrete Structures
CS2210DiscreteStructuresAlgorithmsandComplexity
Fall2017SukumarGhosh
Whatisanalgorithm
Afiniteset(orsequence)ofpreciseinstructions
forperformingacomputation.
Example:Maximafindingproceduremax (a1,a2,…,an:integers)max :=a1fori :=2 ton
ifmax <ai thenmax :=aireturnmax {thelargestelement}
Flowchartformaximafinding
max:=a1
i:=2
max<ai
max=ai
i =n?
i:=i +1
start
yes
no
end
yes
no
Givenn elements,canyoucountthetotalnumberofoperations?
TimecomplexityofalgorithmsCountsthelargestnumberofbasic operations requiredtoexecuteanalgorithm.
Example:Maximafindingproceduremax (a1,a2,…,an:integers)max :=a1 1operationfori :=2 ton 1operationi:=2
ifmax <a1 thenmax :=ai {n-1times}{2ops+1optocheckifi >n +1optoincrementi}
returnmax {thelargestelement}
Thetotalnumberofoperationsis4(n-1)+2=4n-2
TimecomplexityofalgorithmsExampleoflinearsearch(Searchx inalist )
k :=1 {1op}whilek ≤n do {n opsk ≤n}
ifx =ak thenfoundelsek:=k+1} {2nops+1op}
Themaximum numberofoperationsis3n+2.Ifwearelucky,thensearchcanendeveninthefirstiteration.
Timecomplexityofalgorithms
{searchfailed}
Binarysearch(Searchx inasortedlist )
Howmanyoperations?Roughlylogn.Why?
BubbleSortprocedurebubblesort (A:listofitems)n =length(A)repeat
fori =1ton-1doifA[i-1]>A[i]thenswap(A[i-1],A[i])endif
endforn:=n - 1
until n=0endprocedure
BubbleSort
BubbleSort
Theworstcasetimecomplexityis(n-1)+(n-2)+(n-3)+…+2=n(n-1)/2-1
TheBig-OnotationItisameasureofthegrowthoffunctionsandoftenusedtomeasurethecomplexityofalgorithms.
DEF.Letf andg befunctionsfromthesetofintegers(orrealnumbers)tothesetofrealnumbers.Thenf isO(g(x))ifthereareconstantsC andk,suchthat
|f(x)|≤C|g(x)| forallx >k
Intuitively,f(x)grows“slowerthan”somemultipleofg(x)asx growswithoutbound.ThusO(g(x))definesanupperboundoff(x).
TheBig-Onotation
1 2
1
2
3
4y =x2
y=4x2
Definesanupperboundofthegrowthoffunctions
TheBig-Ω (omega)notation
DEF.Letf andg befunctionsfromthesetofintegers(orrealnumbers)tothesetofrealnumbers.Thenf isΩ(g(x))ifthereareconstantsCandk,suchthat
|f(x)|≥ C|g(x)| forallx >k
Example.7x2 +9x+4isΩ(x2),since7x2 +9x+4≥1.x2 forallxThus Ω definesthelowerboundofthegrowthofafunction
Question.Is7x2 +9x+4Ω(x)?
TheBig-Theta(Θ)notationDEF.Letf andg befunctionsfromthesetofintegers(orrealnumbers)tothesetofrealnumbers.Thenf isΘ(g(x))ifthereareconstantsC1 andC2 apositiverealnumberk,suchthat
C1.|g(x)|≤|f(x)|≤C2.|g(x)| forallx >k
Example. 7x2 +9x+4isΘ(x2),since1.x2 ≤7x2 +9x+4≤8.x2 forallx >10
AveragecaseperformanceEXAMPLE.Computetheaveragecasecomplexityofthelinearsearch algorithm.
Classificationofcomplexity
Complexity TerminologyΘ(1) ConstantcomplexityΘ(log n)Θ(log n)c
LogarithmiccomplexityPoly-logarithmiccomplexity
Θ(n) LinearcomplexityΘ(nc) PolynomialcomplexityΘ(bn)(b>1) ExponentialcomplexityΘ(n!) Factorialcomplexity
WealsousesuchtermswhenΘ isreplacedbyO(big-O)
ExerciseComplexityofn5 O(2n) Trueorfalse?Complexityof2n O(n5) Trueorfalse?Complexityoflog(n!) Θ(n logn) Trueorfalse?Complexityof12+22+32+…+n2 Ω(n3) Trueorfalse?”
LetS={0,1,2,…,n}.ThinkofanalgorithmthatgeneratesallthesubsetsofthreeelementsfromS,andcomputeitscomplexityinbig-Onotation.
GreedyAlgorithms
Inoptimizationproblems,manyalgorithmsthatusethebest
choiceateachsteparecalledgreedyalgorithms.
Example.Deviseanalgorithmformakingchangeforn cents
usingquarters,dimes,nickels,andpenniesusingtheleast
numberoftotalcoins?
GreedyChange-makingAlgorithm
Let bethedenominationofthecoins,
(and)
for i:=1tor
while n ≥ci
begin
addacoinofvalueci tothechange
n :=n- ci
end
Question.Isthisoptimal?Doesitusetheleastnumberofcoins?
Letthecoinsbe1,5,10,25cents.Formaking38cents,youwilluse
1quarter1dime3cents
Thetotalcountis5,anditisoptimum.
GreedyChange-makingAlgorithm
Butifyoudon’tuseanickel,andyoumakeachangefor
30centsusingthesamealgorithm,theyouwilluse1quarter
and5cents(total6coins).Buttheoptimumis3coins
(use3dimes!)
So,greedyalgorithmsproduceresults,buttheresults
maybesub-optimal.
GreedyRoutingAlgorithm
A B
IfyouneedtoreachpointBfrompointAinthefewestnumberofhops,
Thenwhichroutewillyoutake?Iftheknowledgeislocal,thenyouare
temptedtouseagreedyalgorithm,andreachBin5hops,although
itispossibletoreachBinonlytwohops.
C
Otherclassificationofproblems• Problemsthathavepolynomialworst-casecomplexityare
calledtractable.Otherwisetheyarecalledintractable.
• Problemsforwhichnosolutionexistsareknownasunsolvable problems(likethehaltingproblem).Otherwisetheyarecalledsolvable.
• Manysolvableproblemsarebelievedtohavethepropertythatnopolynomialtimesolutionexistsforthem,butasolution,ifknown,canbecheckedinpolynomialtime.ThesebelongtotheclassNP(asopposedtotheclassoftractableproblemsthatbelongtoclassP)
Estimationofcomplexity
Source:D.Harel.Algorithmics:TheSpiritofComputing.Addison-Wesley,2ndedition,1992
TheHaltingProblem
TheHaltingproblem asksthequestion.
Givenaprogramandaninputtotheprogram,determineiftheprogram
willeventuallystopwhenitisgiventhatinput.
• Runtheprogramwiththatinput.Iftheprogramstops,thenweknowit
stops.
• Butiftheprogramdoesn'tstopinareasonableamountoftime,thenwe
cannotconcludethatitwon'tstop. Maybewedidn'twaitlongenough!
Thequestionisnotdecidableingeneral!
TheTravelingSalesmanProblem
Startingfromanode,youhavetovisiteveryothernodeandreturnToyoustartingpoint.Findtheshortestroute?NP-complete
3-SatisfiabilityProblem
Doesthereexistanassignmentofvaluesofx,y,zsothatthisformulaistrue?NP-Completeproblem!
Consideranexpressionlikethis: