Chapter 3 The Efficiency of Algorithms

Chapter 3Chapter 3

The Efficiency of AlgorithmsThe Efficiency of Algorithms

Algorithmic problem solving

Attributes of AlgorithmsAttributes of Algorithms Are some algorithms better than others?Are some algorithms better than others? We expect We expect correctness correctness from our algorithmsfrom our algorithms Ease of understanding; EleganceEase of understanding; Elegance

Analysis of AlgorithmsAnalysis of Algorithms EfficiencyEfficiency

Term used to describe an algorithm’s careful use of resourcesTerm used to describe an algorithm’s careful use of resources BenchmarksBenchmarks

Useful for rating one machine against another and for rating Useful for rating one machine against another and for rating how sensitive a particular algorithm is with respect to how sensitive a particular algorithm is with respect to variations in input on one particular machinevariations in input on one particular machine

Measuring EfficiencyMeasuring Efficiency

Analysis of algorithmsAnalysis of algorithms The study of the efficiency of algorithmsThe study of the efficiency of algorithms An important part of computer scienceAn important part of computer science

Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 33

Sequential SearchSequential Search

Search for Search for NAMENAME among a list of among a list of nn names names

Start at the beginning and compare Start at the beginning and compare NAMENAME to each entry until a match is to each entry until a match is foundfound

Invitation to Computer Science, Invitation to Computer Science, 5th Edition5th Edition 55

Figure 3.1 Sequential Search Algorithm

Figure 3.2 Number of Comparisons to Find NAME in a List of n Names Using Sequential Search

Order of Magnitude - Order Order of Magnitude - Order nn

Order of magnitude Order of magnitude nn Anything that varies as a constant times Anything that varies as a constant times

n n (and whose graph follows the basic (and whose graph follows the basic shape of shape of n)n)

Sequential searchSequential search An Θ(An Θ(nn) algorithm in both the worst case ) algorithm in both the worst case

and the average caseand the average case

Figure 3.3 Work = 2n

Figure 3.4 Work = cn for Various Values of c

Figure 3.5 Growth of Work = cn for Various Values of c

Selection SortSelection Sort Selection sort algorithmSelection sort algorithm

Sorts in ascending orderSorts in ascending order Subtask within selection sortSubtask within selection sort

Task of finding the largest number in a listTask of finding the largest number in a list When selection sort algorithm beginsWhen selection sort algorithm begins

The largest-so-far value must be compared to The largest-so-far value must be compared to all the other numbers in the listall the other numbers in the list

If there are If there are n n numbers in the list, numbers in the list, n n – 1 – 1 comparisons must be donecomparisons must be done

Figure 3.6 Selection Sort Algorithm

Figure 3.7 Comparisons Required by Selection Sort

Selection Sort (continued)Selection Sort (continued) Selection sort algorithm Selection sort algorithm

Does Does n n exchanges, one for each position in the exchanges, one for each position in the list to put the correct value in that positionlist to put the correct value in that position

Space efficiency of the selection sortSpace efficiency of the selection sort Original list occupies Original list occupies n n memory locationsmemory locations Storage is needed for: Storage is needed for:

The marker between the unsorted and sorted sectionsThe marker between the unsorted and sorted sections Keeping track of the largest-so-far value and its Keeping track of the largest-so-far value and its

location in the listlocation in the list

Figure 3.8 An Attempt to Exchange the Values at X and Y

Figure 3.9 Exchanging the Values at X and Y

Order of Magnitude - Order Order of Magnitude - Order nn22

Order of magnitude Order of magnitude nn22, or Θ(, or Θ(nn22)) An algorithm that does An algorithm that does cncn22 work for any work for any

constant constant cc Selection sortSelection sort

An Θ(An Θ(nn22) algorithm (in all cases) ) algorithm (in all cases) Sequential search Sequential search

An Θ(An Θ(nn) algorithm (in the worst case)) algorithm (in the worst case)

Figure 3.10 Work 5 cn2 for Various Values of c

Figure 3.11 A Comparison of n and n2

Figure 3.12 For Large Enough n, 0.25n2 Has Larger Values Than 10n

Figure 3.13 A Comparison of Two Extreme Q(n2) and Q(n) Algorithms

Analysis of AlgorithmsAnalysis of Algorithms

Data cleanup algorithmsData cleanup algorithms The Shuffle-Left AlgorithmThe Shuffle-Left Algorithm The Copy-Over Algorithm The Copy-Over Algorithm The Converging-Pointers AlgorithmThe Converging-Pointers Algorithm

The Shuffle-Left AlgorithmThe Shuffle-Left Algorithm

Scans list from left to rightScans list from left to right When a zero is found, copy each When a zero is found, copy each

remaining data item in the list one remaining data item in the list one cell to the leftcell to the left

Value of Value of legitlegit Originally set to the length of the listOriginally set to the length of the list Is reduced by 1 every time a 0 is Is reduced by 1 every time a 0 is

encounteredencountered

Figure 3.14 The Shuffle-Left Algorithm for Data Cleanup

The Shuffle-Left Algorithm The Shuffle-Left Algorithm (continued)(continued)

To analyze time efficiency:To analyze time efficiency: Identify the fundamental units of work Identify the fundamental units of work

the algorithm performsthe algorithm performs Copying numbersCopying numbers

Best case occurs when the list has no 0 Best case occurs when the list has no 0 values because no copying is requiredvalues because no copying is required

Worst case occurs when the list has all Worst case occurs when the list has all 0 values0 values

The Shuffle-Left Algorithm The Shuffle-Left Algorithm (continued)(continued)

Worst case Worst case Occurs when the list has all 0 valuesOccurs when the list has all 0 values An Θ(An Θ(nn22) algorithm ) algorithm

Space-efficient Space-efficient Only requires four memory locations to Only requires four memory locations to

store the quantities store the quantities nn,, legit legit,, left left,, and and rightright

The Copy-Over AlgorithmThe Copy-Over Algorithm Scans the list from left to right, copying every Scans the list from left to right, copying every

legitimate (non-zero) value into a new list that it legitimate (non-zero) value into a new list that it createscreates

Every list entry is examined to see whether it is 0 Every list entry is examined to see whether it is 0 Every non-zero list entry is copied onceEvery non-zero list entry is copied once Best case Best case

Occurs if all elements are 0Occurs if all elements are 0 Θ(Θ(nn) in time efficiency) in time efficiency No extra space is usedNo extra space is used

Figure 3.15 The Copy-Over Algorithm for Data Cleanup

The Copy-Over Algorithm The Copy-Over Algorithm (continued)(continued)

Worst case Worst case Occurs if there are no 0 values in the listOccurs if there are no 0 values in the list Algorithm copies all Algorithm copies all n n non-zero elements non-zero elements

into the new list and doubles the space into the new list and doubles the space requiredrequired

Θ(Θ(nn) in time efficiency) in time efficiency Time/space tradeoffTime/space tradeoff

You gain something by giving up You gain something by giving up something elsesomething else

The Converging-Pointers The Converging-Pointers AlgorithmAlgorithm

Swap zero values from left with values from Swap zero values from left with values from right until pointers converge in the middleright until pointers converge in the middle

Best case Best case A list containing no 0 elementsA list containing no 0 elements

Worst caseWorst case A list of all 0 entriesA list of all 0 entries Θ(Θ(nn) in time efficiency) in time efficiency Is space-efficientIs space-efficient

Figure 3.16 The Converging-Pointers Algorithm for Data Cleanup

Figure 3.17 Analysis of Three Data Cleanup Algorithms

The Converging-Pointers The Converging-Pointers Algorithm (continued)Algorithm (continued)

In an Θ(In an Θ(nn) algorithm:) algorithm: The work is proportional to The work is proportional to nn

In an Θ(In an Θ(nn22) algorithm:) algorithm: The work is proportional to the The work is proportional to the square square

of of nn

Binary SearchBinary Search ProcedureProcedure

First looks for First looks for NAME NAME at roughly the halfway point in at roughly the halfway point in listlist

If name equals If name equals NAMENAME,, search is oversearch is over If If NAME NAME comes alphabetically before name at comes alphabetically before name at

halfway point, search is narrowed to the front half halfway point, search is narrowed to the front half of listof list

If If NAME NAME comes alphabetically after name at halfway comes alphabetically after name at halfway point, search is narrowed to the back half of the listpoint, search is narrowed to the back half of the list

Algorithm halts when Algorithm halts when NAME NAME is foundis found

Figure 3.18 Binary Search Algorithm (list must be sorted)

Binary Search (continued)Binary Search (continued) Logarithm of Logarithm of n n to the base 2to the base 2

Number of times a number Number of times a number n n can be cut in half and can be cut in half and not go below 1not go below 1

As As n n doubles:doubles: lg lg n n increases by only 1, so lg increases by only 1, so lg n n grows much more grows much more

slowly than slowly than nn Worst case and average caseWorst case and average case

Θ(lg Θ(lg nn) comparisons) comparisons Works only on a list that has already been sortedWorks only on a list that has already been sorted

Figure 3.19 Binary Search Tree for a 7-Element List

Figure 3.20 Values for n and lg n

Figure 3.21 A Comparison of n and lg n

Pattern-MatchingPattern-Matching

Usually involves a pattern length that Usually involves a pattern length that is short compared to the text lengthis short compared to the text length That is, when That is, when m m is much less than is much less than nn

Best caseBest case Θ(Θ(nn))

Worst caseWorst case Θ(Θ(m m * * nn))

When Things Get Out of When Things Get Out of HandHand

Polynomially bound algorithmsPolynomially bound algorithms Work done is no worse than a constant Work done is no worse than a constant

multiple of multiple of nn22

GraphGraph A collection of nodes and connecting A collection of nodes and connecting

edgesedges Hamiltonian circuitHamiltonian circuit

A path through a graph that begins and A path through a graph that begins and ends at the same node and goes through ends at the same node and goes through all other nodes exactly onceall other nodes exactly once

Figure 3.22 Order-of-Magnitude Time Efficiency Summary

Figure 3.23 Four Connected Cities

Figure 3.24 Hamiltonian Circuits among All Paths from A in Figure 3.23 with Four Links

When Things Get Out of When Things Get Out of HandHand

Exponential algorithmExponential algorithm An Θ(2An Θ(2nn) algorithm) algorithm

Brute force algorithmBrute force algorithm One that beats the problem into submission by One that beats the problem into submission by

trying all possibilitiestrying all possibilities Intractable problemIntractable problem

No polynomially bounded algorithm existsNo polynomially bounded algorithm exists Approximation algorithmsApproximation algorithms

Do not solve the problem, but provide a close Do not solve the problem, but provide a close approximation to a solutionapproximation to a solution

Figure 3.25 Comparisons of lg n, n, n2, and 2n

Figure 3.26 Comparisons of lg n, n, n2, and 2n for Larger Values of n

Figure 3.27 A Comparison of Four Orders of Magnitude

Chapter 3 The Efficiency of Algorithms

Documents

Transcript of Chapter 3 The Efficiency of Algorithms

INVITATION TO Computer Science 1 Chapter 3 The Efficiency of Algorithms.

Chapter 3: The Efficiency of Algorithms Invitation to Computer Science, Java Version, Third Edition.

Efficiency Enhancement of Estimation of Distribution Algorithms

Algorithms - Kent State Universityssteinfa/classes/intro.sp05/lect/chap03.pdf · 1 Chapter 3: The Efficiency of Algorithms Invitation to Computer Science, C++ Version, Third Edition

The Efficiency of Algorithms Chapter 9 Slides by Steve Armstrong LeTourneau University Longview, TX 2007, Prentice Hall.

Introduction to Algorithms Chapter 16: Greedy Algorithms.

Algorithms and Efficiency of Algorithms February 4th.

Chapter 11 Analysis of Algorithms. Chapter Scope Efficiency goals The concept of algorithm analysis Big-Oh notation The concept of asymptotic complexity.

Chapter 13: Population size, building blocks, fitness ...ebrary.free.fr/Genetic Algorithms Handbook/The_Practical_Handbook… · fitness landscape and genetic algorithm search efficiency

Novel Algorithms for Induction Motor Efficiency Estimation

MULTI-OBJECTIVE EVOLUTIONARY ALGORITHMS FOR ENERGY-EFFICIENCY IN

The Efficiency of Algorithms Chapter 7. 2 Chapter Contents Motivation Measuring an Algorithm's Efficiency Big Oh Notation Formalities Picturing Efficiency.

Data Structure and Algorithms. Algorithms: efficiency and complexity Recursion Reading Algorithms.

Chapter 2 Recursive Algorithms. 2 Chapter Outline Analyzing recursive algorithms Recurrence relations Closest pair algorithms Convex hull algorithms Generating.

Chapter 2: Fundamentals of the Analysis of Algorithm ...s.pdf · Fundamentals of Analysis of algorithm efficiency . Analysis of algorithms Issues: correctness time efficiency space

Data Structures and Algorithms 10 - edX · Ming Zhang “Data Structures and Algorithms” Chapter 10 Retrieval Basis Concepts •Retrieval •The efficiency of retrieval is very

Efficiency of Algorithms

1 Chapter 3: Efficiency of Algorithms Quality attributes for algorithms Correctness: It should do things right No flaws in design of the algorithm Maintainability.

CMPUT101 Introduction to Computing(c) Yngvi Bjornsson & Jia You1 The Efficiency of Algorithms Chapter 3 Topics: Attributes of Algorithms A Choice of Algorithms.

Data Structures and Algorithms 10...3 目录页 Ming Zhang “Data Structures and Algorithms” Chapter 10 Retrieval Basis Concepts •Retrieval •The efficiency of retrieval is very