7.8 What is an algorithm?Intractability...7 Literal.A Boolean variable or its negation. Clause.A...
Transcript of 7.8 What is an algorithm?Intractability...7 Literal.A Boolean variable or its negation. Clause.A...
Introduction to Computer Science • Robert Sedgewick and Kevin Wayne • Copyright © 2006 • http://www.cs.Princeton.EDU/IntroCS
7.8 Intractability
2
Overview
Q. What is an algorithm?
A. Definition formalized using Turing machines.
Q. Which problems can be solved on a computer?
A. Computability.
Q. Which algorithms will be useful in practice?
A. Analysis of algorithms.
Q. Which problems can be solved in practice?
A. Intractability.
3
Q. Which algorithms are useful in practice?
A working definition. [von Neumann 1953, Godel 1956, Cobham 1960, Edmonds 1962]
! Measure running time as a function of input size N.
! Efficient = polynomial time for all inputs.
! Inefficient = "exponential time" for some inputs.
Ex. Dynamic programming algorithm for edit distance takes N2 steps.
Ex. Brute force algorithm for TSP takes N! steps.
Theory. Definition is broad and robust; huge gulf between polynomial
and exponential algorithms.
Practice. Exponents and constants of polynomials that arise are small !
scales to huge problems.
Properties of Algorithms
4
Exponential Growth
Exponential growth dwarfs technological change.
! Suppose each electron in the universe had power of today's
supercomputers…
! And each works for the life of the universe in an effort to solve
one TSP problem via brute force.
! Will not help solve 1,000 city TSP problem via brute force.
Quantity
Supercomputer instructions per second
Age of universe in seconds †
Number
1013
1017
Electrons in universe † 1079 † Estimated
1000! >> 101000 >> 1079 " 1017 " 1013
5
P
Def. P is the set of all yes-no problems solvable in poly-time on a
deterministic Turing machine.
Remark. Algorithm typically also solves the related search problem.
34, 5134, 39Euclid (300 BCE)Are x and y relatively prime?RELPRIME
5351Agarwal-Kayal-
Saxena (2002)
Does x have a factor other
than 1 and itself?COMPOSITE
acgggt
ttttta
niether
neither
Dynamic
Programming
Is the edit distance between
strings x and y less than 5?
EDIT-
DISTANCE
Is there a vector x that
satisfies Ax = b?
Description
Gauss-Edmonds
elimination
Algorithm
LSOLVE
Problem NoYes
!
0 1 1
2 4 "2
0 3 15
#
$
% % %
&
'
( ( (
,
4
2
36
#
$
% % %
&
'
( ( (
!
1 0 0
1 1 1
0 1 1
"
#
$ $ $
%
&
' ' '
,
1
1
1
"
#
$ $ $
%
&
' ' '
6
Extended Church-Turing Thesis
Extended Church-Turing thesis.
! P = yes-no problems solvable in poly-time in this universe.
! If computable by a piece of hardware in time T(N) for input of size
N, then computable by TM in time (T(N))k for some constant k.
Evidence supporting thesis.
! True for all physical computers.
! k = 2 for random access machines. [your laptop]
Implication. To make future computers more efficient, only need to
focus on improving implementation of existing designs.
Law of physics? A new constraint on what is possible?
Possible counterexample? Quantum computers. like 2nd law of thermodynamics
7
Literal. A Boolean variable or its negation.
Clause. A disjunction of 3 distinct literals.
Conjunctive normal form. A propositional
formula # that is the conjunction of clauses.
3-SAT. Given a CNF formula # consisting of k clauses over n literals,
does it have a satisfying truth assignment?
3-Satisfiability
!
Cj = x1" x
2" x
3
!
xi or x
i
!
" = C1#C
2# C
3# C
4 # C
5
!
x1" x
2" x
3( ) # x1" x
2" x
3( ) # x1" x
2" x
3( ) # x1" x
2" x
4( ) # x2" x
3 " x
4( )
!
x1 = true, x2 = true, x3 = false, x4 = true
a satisfiable formula with 5 clauses and 4 variables
8
Q. How to solve an instance of 3-SAT.
A1. Try all 2n truth assignments.
A2. Do something substantially more clever???
Conjecture. No poly-time algorithm for 3-SAT.
Intractability of 3-SAT
"intractable"
9
Reductions
Q. Which problems won't we be able to solve in practice?
A. No easy answers, but theory helps.
Def. Problem X polynomial reduces to problem Y if arbitrary instances
of problem X can be solved using:
! Polynomial number of standard computational steps, plus
! One call to subroutine for Y as last step.
Consequence. If no poly-time algorithm for X, then no poly-time
algorithm for Y.3-SAT
your research problem
10
Planar 3-Color
Given a planar map, can it be colored using 3 colors so that no adjacent
regions have the same color?
YES instance.
11
Planar 3-Color
Given a planar map, can it be colored using 3 colors so that no adjacent
regions have the same color?
NO instance.
12
3-Colorability
3-COLOR. Given a graph (need not be planar), is there a way to color
the vertices red, green, and blue so that no adjacent vertices have the
same color?
Applications. Register allocation, Potts model in physics, …
yesinstance
13
3-Colorability
3-COLOR. Given a graph, is there a way to color the vertices
red, green, and blue so that no adjacent vertices have the same color?
14
3-Colorability
Claim. 3-SAT polynomial reduces to 3-COLOR.
Pf. Given 3-SAT instance #, we construct an instance of 3-COLOR that
is 3-colorable iff # is satisfiable.
Construction.
i. Create one vertex for each literal.
ii. Create 3 new vertices T, F, and B; connect them in a triangle,
and connect each literal to B.
iii. Connect each literal to its negation.
iv. For each clause, attach a gadget of 6 vertices and 13 edges.
to be described next
15
3-Colorability
Claim. Graph is 3-colorable iff # is satisfiable.
Pf. ! Suppose graph is 3-colorable.
! Consider assignment that sets all T-colored literals to true.
! (ii) ensures each literal is same color as T or F.
! (iii) ensures a literal and its negation are opposites.
true false
baseB
T F
!
x1
!
x1
!
x2
!
x2
!
xn
!
xn
!
x3
!
x3
16
3-Colorability
Claim. Graph is 3-colorable iff # is satisfiable.
Pf. ! Suppose graph is 3-colorable.
! Consider assignment that sets all T-colored literals to true.
! (ii) ensures each literal is same color as T or F.
! (iii) ensures a literal and its negation are opposites.
! (iv) ensures at least one literal in each clause is same color as T.
T F
!
Ci
= x1V x
2V x
3
6-node gadget
true false
B
!
x1
!
x2
!
x3
17
3-Colorability
Claim. Graph is 3-colorable iff # is satisfiable.
Pf. ! Suppose graph is 3-colorable.
! Consider assignment that sets all T-colored literals to true.
! (ii) ensures each literal is same color as T or F.
! (iii) ensures a literal and its negation are opposites.
! (iv) ensures at least one literal in each clause is same color as T.
T F
B
!
x1
!
x2
!
x3
not 3-colorable if all are red
true false
contradiction!
Ci
= x1V x
2V x
3
6-node gadget
18
3-Colorability
Claim. Graph is 3-colorable iff # is satisfiable.
Pf. $ Suppose 3-SAT formula # is satisfiable.
! Color all true literals same color as T.
! Color vertex below green node F, and vertex below that B.
! Color remaining middle row vertices B.
! Color remaining bottom vertices T or F as forced. !
T F
B
!
x1
!
x2
!
x3
a literal set to true in 3-SAT assignment
true false
!
Ci
= x1V x
2V x
3
6-node gadget
19
More Poly-Time Reductions
3-SAT
3DM VERTEX COVER
HAM-CYCLECLIQUE
INDEPENDENT SET
3-COLOR
PLANAR-3-COLOREXACT COVER
HAM-PATHSUBSET-SUM
PARTITION INTEGER PROGRAMMING
KNAPSACK
Dick KarpTuring award (1985)
3-SAT
reduces to 3-COLOR
TSP
BIN-PACKINGConjecture: no poly-time algorithm for 3-SAT.
(and hence none of these problems)20
Still More Hard Computational Problems
Aerospace engineering. Optimal mesh partitioning for finite elements.
Biology. Protein folding.
Chemical engineering. Heat exchanger network synthesis.
Chemistry. Chemical synthesizability.
Civil engineering. Equilibrium of urban traffic flow.
Economics. Computation of arbitrage in financial markets with friction.
Electrical engineering. VLSI layout.
Environmental engineering. Optimal placement of contaminant sensors.
Financial engineering. Find minimum risk portfolio of given return.
Game theory. Find Nash equilibrium that maximizes social welfare.
Genomics. Phylogeny reconstruction.
Mathematics. Given integer a1, …, an, compute
Mechanical engineering. Structure of turbulence in sheared flows.
Medicine. Reconstructing 3-D shape from biplane angiocardiogram.
Operations research. Traveling salesperson problem.
Physics. Partition function of 3-D Ising model in statistical mechanics.
Politics. Shapley-Shubik voting power.
Pop culture. Minesweeper consistency.
Statistics. Optimal experimental design.
6,000+ scientific papers per year.
21
Implications of Intractability
Proving a problem intractable guide scientific inquiry.
! 1926: Ising introduces simple model for phase transitions.
! 1944: Onsager find closed form solution to 2D case in tour de force.
! 19xx: Feynman and other top minds seek 3D solution.
! 2000: Istrail reduces 3-SAT to 3D version of problem.
a holy grail of statistical physics
search for closed formula appears doomed
22
Cook's Theorem
3-SAT
3DM VERTEX COVER
HAM-CYCLECLIQUE
INDEPENDENT SET
3-COLOR
PLANAR-3-COLOREXACT COVER
HAM-PATHSUBSET-SUM
PARTITION INTEGER PROGRAMMING
KNAPSACK
3-COLOR
reduces to 3-SAT
TSP
BIN-PACKINGAll of these problems (any many more)polynomial reduce to 3-SAT.
Stephen CookTuring award (1982)
23
Cook + Karp
3-SAT
3DM VERTEX COVER
HAM-CYCLECLIQUE
INDEPENDENT SET
3-COLOR
PLANAR-3-COLOREXACT COVER
HAM-PATHSUBSET-SUM
PARTITION INTEGER PROGRAMMING
KNAPSACK
3-COLOR
reduces to 3-SAT
TSP
BIN-PACKINGAll of these problems are different manifestations
of one "really hard" problem.
3-SAT
reduces to 3-COLOR
24
Complexity Classes
P. Set of yes-no problems solvable in poly-time on a deterministic TM.
EXP. Same as P, but in exponential-time.
NP. Same as P, but on non-deterministic Turing machine.
Cook-Levin Theorem (1960s). ALL NP problems reduce to 3-SAT.
NP-complete. All NP problems to which 3-SAT reduces.
Implications.
! If efficient algorithm for 3-SAT, then P = NP.
! If efficient algorithm for any NP-complete problem, then P = NP.
! If no efficient algorithm for some NP problem, then none for 3-SAT.
if we can solve 3-SAT,we can solve any of them
if we can solve any ofthem, we can solve 3-SAT
25
Certificates
Alternate (and Equivalent) Definition. NP is set of all yes-no problems
for which you can check in poly-time that it is a yes instance (given a
certificate).
3-COLOR. Given a graph, is there a way to color the vertices
red, green, and blue so that no adjacent vertices have the same color?
instance certificate
26
Certificates
Alt. Def. NP is set of all yes-no problems for which you can check in
poly-time that it is a yes instance (given a certificate).
FACTOR. Given two integers x and y, does x have a factor
between 2 and y?
x = 2773
y = 50
instance
certificate
47 ! 59
x = 147573952589676412927
y = 147573952589676412927
193707721 ! 761838257287
instance
certificate
267 - 1
27
The Main Question
Does P = NP?
! Is there a poly-time algorithm for SAT?
! Does nondeterminism help you solve problems faster?
! Clay $1 million prize.
If yes: Efficient algorithms for 3-COLOR, TSP, FACTOR, . . .
If no: No efficient algorithms possible for 3-COLOR, TSP, . . .
Consensus opinion on P = NP? Probably no.
would break modern cryptographyand collapse economy
EXP NP
P
If P % NP If P = NP
NP-complete
EXP
P = NP
Jack Edmonds, 1962
28
Implications of NP-Completeness
Classify problems according to their computational requirements.
! NP-complete: SAT, all Karp problems, thousands more.
! P: RELPRIME, COMPOSITE, LSOLVE.
! Unclassified: FACTOR is in NP, but unknown if NP-complete or in P.
Computational universality.
! All known algorithms for NP-complete problems are exponential.
! If any NP-complete problem proved exponential, so are rest.
! If any NP-complete problem proved polynomial, so are rest.
Proving a problem is NP-complete can guide scientific inquiry.
! 1926: Ising introduces simple model for phase transitions.
! 1944: Onsager solves 2D case in tour de force.
! 19xx: Feynman and other top minds seek 3D solution.
! 2000: Istrail proves 3D problem NP-complete.
31
Coping With Intractability
Relax one of desired features.
! Solve the problem in poly-time.
! Solve the problem to optimality.
! Solve arbitrary instances of the problem.
Complexity theory deals with worst case behavior.
! Instance(s) you want to solve may be "easy."
! Concorde algorithm solved 13,509 US city TSP problem.
32
Coping With Intractability
Relax one of desired features.
! Solve the problem in poly-time.
! Solve the problem to optimality.
! Solve arbitrary instances of the problem.
Develop a heuristic, and hope it produces a good solution.
! No guarantees on quality of solution.
! Ex: TSP assignment heuristics.
! Ex: Metropolis algorithm, simulating annealing, genetic algorithms.
Design an approximation algorithm.
! Guarantees to find a nearly-optimal solution.
! Ex: Euclidean TSP tour guaranteed to be within 1% of optimal.
! Active area of research, but not always possible!
Sanjeev Arora (1997)
33
Coping With Intractability
Relax one of desired features.
! Solve the problem in poly-time.
! Solve the problem to optimality.
! Solve arbitrary instances of the problem.
Exploit intractability.
! Cryptography. [stay tuned]
Keep trying to prove P = NP.
can do any 2 of 3
34
Summary
Many fundamental problems are intractable.
! TSP, 3-SAT, 3-COLOR.
! 3D Ising model.
Theory says: we probably won't be able to design efficient algorithms
for them.
! You will encounter such problems in your scientific lives.
! If you know about intractability, you can identify them and
avoid wasting time and energy.