Cliff Shaffer Computer Science
description
Transcript of Cliff Shaffer Computer Science
![Page 1: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/1.jpg)
Cliff ShafferComputer Science
Computational Complexity
![Page 2: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/2.jpg)
Computer Performance
![Page 3: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/3.jpg)
Computer PerformanceDo we need to care about
performance when computers keep getting faster?
![Page 4: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/4.jpg)
Computer PerformanceDo we need to care about
performance when computers keep getting faster?Our history is to do bigger problems,
not the same ones faster.
![Page 5: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/5.jpg)
Computer PerformanceDo we need to care about
performance when computers keep getting faster?Our history is to do bigger problems,
not the same ones faster.More complex problems are less tied
to our everyday “common sense” experience
![Page 6: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/6.jpg)
Algorithm AnalysisWe could compare two programs
by running them side-by-side
![Page 7: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/7.jpg)
Algorithm AnalysisWe could compare two programs
by running them side-by-sideBut that means we have to implement
them!
![Page 8: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/8.jpg)
Algorithm AnalysisWe could compare two programs
by running them side-by-sideBut that means we have to implement
them!We want a way to easily evaluate
programs before they are writtenLook at the algorithm, not the
program
![Page 9: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/9.jpg)
Algorithm AnalysisWe could compare two programs
by running them side-by-sideBut that means we have to implement
them!We want a way to easily evaluate
programs before they are writtenLook at the algorithm, not the
programAlgorithm Analysis estimates
problem cost as a function of growth rate
![Page 10: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/10.jpg)
Simple SearchFind the record with key value
1005.Sequential search: Look through
each record in turn.If there are n records, we do work
proportional to n (unless we are lucky).
![Page 11: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/11.jpg)
Simple SearchFind the record with key value
1005.Sequential search: Look through
each record in turn.If there are n records, we do work
proportional to n (unless we are lucky).
The growth rate of this problem (in the average or worst cases) is linear on n.
![Page 12: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/12.jpg)
Sorting: Insertion Sort• For each record• Insert it into the sorted list made from the records already seen.
• Might have to look at each such record already in the (sorted) list – n work.• Since we do this for n records, this
is n*n in the worst (and average) cases.• So the cost is proportional to n2
(unless we are really lucky).
![Page 13: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/13.jpg)
Sorting: Merge SortFor a list of n records:
Split the list in half. Merge each half (using merge sort) Merge the records together (needs n
work)Total cost: proportional to n log n
![Page 14: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/14.jpg)
Sorting DemoURL:
http://www.cs.ubc.ca/spider/harrison/Java/
Compare Insertion, Shell, Heap, Quick sorts
![Page 15: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/15.jpg)
Does It Matter?• 1000 records:• Insertion sort: 1,000,000• Mergesort: 10,000• Factor of 100 difference
• 1,000,000 records• Insertion sort: 1,000,000,000,000• Mergsort: 20,000,000• Factor of 50,000 difference• Hours vs. seconds on a real computer
![Page 16: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/16.jpg)
Tractable vs. Intractable• Cost n is better than cost n log n,
which is better than cost n2.
![Page 17: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/17.jpg)
Tractable vs. Intractable• Cost n is better than cost n log n,
which is better than cost n2.• These are all polynomial: a faster
computer gives you a bigger problem in an hour by some factor.
![Page 18: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/18.jpg)
Tractable vs. Intractable• Cost n is better than cost n log n,
which is better than cost n2.• These are all polynomial: a faster
computer gives you a bigger problem in an hour by some factor.• Exponential growth: 2n.
![Page 19: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/19.jpg)
Tractable vs. Intractable• Cost n is better than cost n log n,
which is better than cost n2.• These are all polynomial: a faster
computer gives you a bigger problem in an hour by some factor.• Exponential growth: 2n.• Making input one unit bigger
doubles the cost.
![Page 20: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/20.jpg)
Tractable vs. Intractable• Cost n is better than cost n log n,
which is better than cost n2.• These are all polynomial: a faster
computer gives you a bigger problem in an hour by some factor.• Exponential growth: 2n.• Making input one unit bigger
doubles the cost.• Running twice as fast only gives
you one more problem unit.
![Page 21: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/21.jpg)
Tractable vs. Intractable• Cost n is better than cost n log n,
which is better than cost n2.• These are all polynomial: a faster
computer gives you a bigger problem in an hour by some factor.• Exponential growth: 2n.• Making input one unit bigger
doubles the cost.• Running twice as fast only gives
you one more problem unit.• Exponential-time algorithms are
“intractable”.
![Page 22: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/22.jpg)
Problems• Problems have many algorithms
(sorting)• What does “cost of a problem”
mean?
![Page 23: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/23.jpg)
Problems• Problems have many algorithms
(sorting)• What does “cost of a problem”
mean?• We say the problem’s cost is that of
the best algorithm.• But we can’t know all the
algorithms!
![Page 24: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/24.jpg)
Problems• Problems have many algorithms
(sorting)• What does “cost of a problem”
mean?• We say the problem’s cost is that of
the best algorithm.• But we can’t know all the
algorithms!• It is possible (though difficult) to
figure out lowest cost for any algorithm to solve the problem• Sorting: n log n lower bound.
![Page 25: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/25.jpg)
Traveling Salesman ProblemGiven n cities, find a tour for all
the cities that is of shortest length.
![Page 26: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/26.jpg)
Traveling Salesman ProblemGiven n cities, find a tour for all
the cities that is of shortest length.Nobody knows a polynomial-time
algorithm, only exponential algorithms.
![Page 27: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/27.jpg)
Traveling Salesman ProblemGiven n cities, find a tour for all
the cities that is of shortest length.Nobody knows a polynomial-time
algorithm, only exponential algorithms.
We don’t KNOW that this problem needs exponential time.
![Page 28: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/28.jpg)
Traveling Salesman ExampleURL:
http://itp.nat.uni-magdeburg.de/~mertens/TSP/TSP.html
Nearest Neighbor Heuristic
![Page 29: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/29.jpg)
NP-Completeness• Many, many problems are like
traveling salesman – we know no polynomial algorithm, and have no proof they need exponential time.
![Page 30: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/30.jpg)
NP-Completeness• Many, many problems are like
traveling salesman – we know no polynomial algorithm, and have no proof they need exponential time.• It is possible to “cheaply” convert
any problem from this collection into any other.
![Page 31: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/31.jpg)
NP-Completeness• Many, many problems are like
traveling salesman – we know no polynomial algorithm, and have no proof they need exponential time.• It is possible to “cheaply” convert
any problem from this collection into any other.• So if we had a polynomial time
algorithm for any of them, we’d have one for all.
![Page 32: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/32.jpg)
NP-Completeness• Many, many problems are like
traveling salesman – we know no polynomial algorithm, and have no proof they need exponential time.• It is possible to “cheaply” convert
any problem from this collection into any other.• So if we had a polynomial time
algorithm for any of them, we’d have one for all.• These are called NP-complete
problems.
![Page 33: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/33.jpg)
NP-Completeness• Many, many problems are like
traveling salesman – we know no polynomial algorithm, and have no proof they need exponential time.• It is possible to “cheaply” convert
any problem from this collection into any other.• So if we had a polynomial time
algorithm for any of them, we’d have one for all.• These are called NP-complete
problems.• NP problems are those problems for
which we can quickly verify that a proposed solution is correct.
![Page 34: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/34.jpg)
Examples of NP-complete problems• Find the cheapest way to wire up
telephones in a city• Find the largest clique in a graph• Find a way to assign values to a
boolean expression to make it true• Find the largest matching between
workers and compatible jobs• Find the least number of boxes
needed to pack some goods
![Page 35: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/35.jpg)
What do you do?… when you must solve an NP-
complete problem?
![Page 36: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/36.jpg)
What do you do?… when you must solve an NP-
complete problem?ApproximationOptimization
![Page 37: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/37.jpg)
What do you do?… when you must solve an NP-
complete problem?ApproximationOptimizationMany engineering problems are
optimization problems
![Page 38: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/38.jpg)
What do you do?… when you must solve an NP-
complete problem?ApproximationOptimizationMany engineering problems are
optimization problemsExamples: Aircraft design, “best”
decision
![Page 39: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/39.jpg)
Why is optimization hard?• Imagine a 2d problem – find the
highest hill.
![Page 40: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/40.jpg)
Why is optimization hard?• Imagine a 2d problem – find the
highest hill.• Imagine a 10-parameter problem• Just checking the “high” and “low”
values would give 1024 combinations.
![Page 41: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/41.jpg)
Why is optimization hard?• Imagine a 2d problem – find the
highest hill.• Imagine a 10-parameter problem• Just checking the “high” and “low”
values would give 1024 combinations.• Imagine a 10d “cube”… 1024
corners.• The goal is to find the best point in
the cube, for a complex function.
![Page 42: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/42.jpg)
Why is optimization hard?• Imagine a 2d problem – find the
highest hill.• Imagine a 10-parameter problem• Just checking the “high” and “low”
values would give 1024 combinations.• Imagine a 10d “cube”… 1024
corners.• The goal is to find the best point in
the cube, for a complex function.• Many problems have higher
dimension
![Page 43: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/43.jpg)
Why is optimization hard?• Imagine a 2d problem – find the
highest hill.• Imagine a 10-parameter problem• Just checking the “high” and “low”
values would give 1024 combinations.• Imagine a 10d “cube”… 1024
corners.• The goal is to find the best point in
the cube, for a complex function.• Many problems have higher
dimension• Whole branches of
CS/Math/Engineering devoted to optimization
![Page 44: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/44.jpg)
Uncomputable ProblemsNot all problems that we can think
of can be solved.Abstractly, not all functions can be
computed.
![Page 45: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/45.jpg)
Uncomputable ProblemsNot all problems that we can think
of can be solved.Abstractly, not all functions can be
computed.The number of computable
programs is countably infinite.The number of integer functions is
uncountably infinite.
![Page 46: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/46.jpg)
The Halting ProblemProblem: Given a particular
program P on particular input I, does P halt when run on I?
![Page 47: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/47.jpg)
The Halting ProblemProblem: Given a particular
program P on particular input I, does P halt when run on I?
Can be proved that this is impossible to determine in all cases.
![Page 48: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/48.jpg)
The Halting ProblemProblem: Given a particular
program P on particular input I, does P halt when run on I?
Can be proved that this is impossible to determine in all cases.
Lots of problems like this that try to determine program behavior.
![Page 49: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/49.jpg)
The Halting ProblemProblem: Given a particular
program P on particular input I, does P halt when run on I?
Can be proved that this is impossible to determine in all cases.
Lots of problems like this that try to determine program behavior.
Does this program contain a virus?
![Page 50: Cliff Shaffer Computer Science](https://reader036.fdocuments.net/reader036/viewer/2022062310/5681680d550346895ddd95e1/html5/thumbnails/50.jpg)
Does this terminate for all n?(n is an integer)While n > 1 do if Odd(n) then n = 3n + 1 else n = n/2