An Introduction to Computational Complexity Lance Fortnow University of Chicago.

An Introduction to An Introduction to Computational Computational

ComplexityComplexity

Lance FortnowLance Fortnow

University of ChicagoUniversity of Chicago

The Round Table ProblemThe Round Table Problem

Sir Lancelot

Sir Percivale

Sir Galahad Sir Gareth

Sir PericvaleSir Bevidere

The Round Table ProblemThe Round Table Problem

Map ColoringMap Coloring

5 Colors5 Colors

All Maps are Four ColorableAll Maps are Four Colorable

Can we use only 3 colors?Can we use only 3 colors?

SudokuSudoku

All The Same ProblemAll The Same Problem

Solve one of these problems, you solve them Solve one of these problems, you solve them all.all.

And Thousands More…And Thousands More…

► MinesweeperMinesweeper► Traveling Traveling

SalespersonSalesperson► CliquesCliques► Integer ProgrammingInteger Programming► Quadratic Quadratic

ProgrammingProgramming► Maximum CutMaximum Cut► Longest PathLongest Path

► Bin PackingBin Packing► Job SchedulingJob Scheduling► Filling a KnapsackFilling a Knapsack► Crossword Puzzle Crossword Puzzle

ConstructionConstruction► Code GenerationCode Generation► Sequencing DNASequencing DNA► Finding ArbitrageFinding Arbitrage

Can We Solve These Can We Solve These Problems?Problems?

P = NP?P = NP?

The P versus NP QuestionThe P versus NP Question

►P – Efficiently Computable SolutionsP – Efficiently Computable Solutions►NP – Efficiently Verifiable SolutionsNP – Efficiently Verifiable Solutions

P – Efficiently ComputableP – Efficiently Computable

► Finding the shortest Finding the shortest path from point A to path from point A to point Bpoint B

NP – Efficiently VerifiableNP – Efficiently Verifiable

NP-CompleteNP-Complete

NP

P

NPC

If P ≠ NPIf P ≠ NP

NP

P

NPC

If P = NPIf P = NP

NP

P

NPC

P versus NPP versus NP

►General Consensus: P ≠ NPGeneral Consensus: P ≠ NP►Major Open Problem: Formally prove P ≠ Major Open Problem: Formally prove P ≠

NPNP►First to do so wins $1,000,000First to do so wins $1,000,000►One of the seven “Millennium” Problems One of the seven “Millennium” Problems

of the Clay Mathematics Institute.of the Clay Mathematics Institute.►Many approaches, all have failed Many approaches, all have failed

miserably.miserably.

P versus NP: A Short HistoryP versus NP: A Short History

►1936: Alan Turing tries to understand how 1936: Alan Turing tries to understand how a mathematician thinks and developed a a mathematician thinks and developed a theoretical model of computation we now theoretical model of computation we now call the Turing Machine.call the Turing Machine.


►1956: The logician Kurt Gödel, in a 1956: The logician Kurt Gödel, in a letter to John van Neumann, asked a letter to John van Neumann, asked a question mathematically equivalent to question mathematically equivalent to P = NP?P = NP?

►The letter was never followed up and The letter was never followed up and soon forgotten.soon forgotten.


►1962: Juris Hartmanis and Richard 1962: Juris Hartmanis and Richard Stearns develop the notion of Stearns develop the notion of “Computational Complexity” by “Computational Complexity” by suggesting we look at the time a suggesting we look at the time a program takes as a function of the size program takes as a function of the size of the input.of the input.

►They show that given more time, a They show that given more time, a computer can do solve more problems. computer can do solve more problems.


►1965: Independently Alan Cobham and 1965: Independently Alan Cobham and Jack Edmonds suggest that one can Jack Edmonds suggest that one can characterize efficient computation by characterize efficient computation by having the running time run in some having the running time run in some fixed polynomial of the input length.fixed polynomial of the input length.

►This class of problems is later given This class of problems is later given the name P.the name P.


►1972: Steve Cook gave the first NP-1972: Steve Cook gave the first NP-complete problem, Boolean Formula complete problem, Boolean Formula Satisfiability, and formally defined the Satisfiability, and formally defined the P versus NP problem.P versus NP problem.

u v w u w x v w x


►1973: Richard Karp showed that 1973: Richard Karp showed that several natural problems, such as several natural problems, such as Clique and Traveling Salesperson are Clique and Traveling Salesperson are NP-complete.NP-complete.

►The rest is history.The rest is history.

Suppose P = NPSuppose P = NP

►What a beautiful world…What a beautiful world…

We Can Efficiently SolveWe Can Efficiently Solve

► MinesweeperMinesweeper► Traveling Traveling

SalespersonSalesperson► CliquesCliques► Integer ProgrammingInteger Programming► Quadratic Quadratic

ProgrammingProgramming► Maximum CutMaximum Cut► Longest PathLongest Path

► Bin PackingBin Packing► Job SchedulingJob Scheduling► Filling a KnapsackFilling a Knapsack► Crossword Puzzle Crossword Puzzle

ConstructionConstruction► Code GenerationCode Generation► Sequencing DNASequencing DNA► Finding ArbitrageFinding Arbitrage

Occam’s RazorOccam’s Razor

►William of OckhamWilliam of Ockham Entities should not Entities should not

be multiplied beyond be multiplied beyond necessitynecessity..

► The shortest The shortest explanation of the explanation of the data is the best one.data is the best one.

TranslationTranslation► In the beginning God created In the beginning God created

the heavens and the earth. the heavens and the earth. Now the earth was formless Now the earth was formless and empty, darkness was and empty, darkness was over the surface of the deep, over the surface of the deep, and the Spirit of God was and the Spirit of God was hovering over the waters. hovering over the waters. And God said, "Let there be And God said, "Let there be light," and there was light. light," and there was light. God saw that the light was God saw that the light was good, and He separated the good, and He separated the light from the darkness. God light from the darkness. God called the light "day," and called the light "day," and the darkness he called the darkness he called "night." And there was "night." And there was evening, and there was evening, and there was morning—the first day. morning—the first day.

► Im Anfang schuf Gott die Im Anfang schuf Gott die Himmel und die Erde. Und Himmel und die Erde. Und die Erde war wüst und leer, die Erde war wüst und leer, und Finsternis war über der und Finsternis war über der Tiefe; und der Geist Gottes Tiefe; und der Geist Gottes schwebte über den Wassern. schwebte über den Wassern. Und Gott sprach: Es werde Und Gott sprach: Es werde Licht! Und es wurde Licht. Licht! Und es wurde Licht. Und Gott sah das Licht, daß Und Gott sah das Licht, daß es gut war; und Gott schied es gut war; und Gott schied das Licht von der Finsternis. das Licht von der Finsternis. Und Gott nannte das Licht Und Gott nannte das Licht Tag, und die Finsternis Tag, und die Finsternis nannte er Nacht. Und es nannte er Nacht. Und es wurde Abend, und es wurde wurde Abend, und es wurde Morgen: ein Tag. Morgen: ein Tag.

TranslationTranslation► In the beginning God created the heavens and In the beginning God created the heavens and

the earth. Now the earth was formless and the earth. Now the earth was formless and empty, darkness was over the surface of the empty, darkness was over the surface of the deep, and the Spirit of God was hovering over deep, and the Spirit of God was hovering over the waters. And God said, "Let there be light," the waters. And God said, "Let there be light," and there was light. God saw that the light was and there was light. God saw that the light was good, and He separated the light from the good, and He separated the light from the darkness. God called the light "day," and the darkness. God called the light "day," and the darkness he called "night." And there was darkness he called "night." And there was evening, and there was morning—the first day. evening, and there was morning—the first day.

► Im Anfang schuf Gott die Himmel und die Erde. Im Anfang schuf Gott die Himmel und die Erde. Und die Erde war wüst und leer, und Finsternis Und die Erde war wüst und leer, und Finsternis war über der Tiefe; und der Geist Gottes war über der Tiefe; und der Geist Gottes schwebte über den Wassern. Und Gott sprach: schwebte über den Wassern. Und Gott sprach: Es werde Licht! Und es wurde Licht. Und Gott Es werde Licht! Und es wurde Licht. Und Gott sah das Licht, daß es gut war; und Gott schied sah das Licht, daß es gut war; und Gott schied das Licht von der Finsternis. Und Gott nannte das Licht von der Finsternis. Und Gott nannte das Licht Tag, und die Finsternis nannte er das Licht Tag, und die Finsternis nannte er Nacht. Und es wurde Abend, und es wurde Nacht. Und es wurde Abend, und es wurde Morgen: ein Tag. Morgen: ein Tag.

If P = NP…If P = NP…

►Can efficiently find the smallest Can efficiently find the smallest program to translate documents.program to translate documents.

►Recognizing FacesRecognizing Faces►Speech to Text, Pictures to TextSpeech to Text, Pictures to Text►Recognizing MusicRecognizing Music►……

No Such LuckNo Such Luck

► Life can’t be so Life can’t be so good so we all good so we all believebelieveP ≠ NP.P ≠ NP.

►Once we show a Once we show a problem is “NP-problem is “NP-Complete” we Complete” we should stop looking should stop looking for an efficient for an efficient algorithm.algorithm.

What to do about hard What to do about hard problems?problems?

►Solve it anyway.Solve it anyway. Computers are very fast these days.Computers are very fast these days.

►Heuristics.Heuristics. Your specific instance might not be that Your specific instance might not be that

hard.hard.

►Approximation.Approximation. Can’t get best algorithm but perhaps one Can’t get best algorithm but perhaps one

can get a “good enough” solution.can get a “good enough” solution.

Solve It AnywaySolve It Anyway

► There are about There are about 2x102x107575 possible possible numbers to try.numbers to try.

► A simple A simple backtracking backtracking algorithm can solve algorithm can solve the problem on a PC the problem on a PC in a few seconds.in a few seconds.

HeuristicsHeuristics

►Most puzzles are Most puzzles are designed to be designed to be solved by humans.solved by humans.

ApproximationApproximation

Clique ProblemClique Problem

► Find a large group Find a large group of people all of of people all of whom are friends whom are friends with each other.with each other.

► Coming up with a Coming up with a clique anywhere clique anywhere close to the largest close to the largest clique is as hard as clique is as hard as solving NP-complete solving NP-complete problems.problems.

Good News with Hard Good News with Hard ProblemsProblems

►Pseudorandom GeneratorsPseudorandom Generators Hard functions can help us create fake Hard functions can help us create fake

random coins that are as good as real random coins that are as good as real random coins.random coins.

►Public-Key CryptographyPublic-Key Cryptography

Public-Key CryptographyPublic-Key Cryptography

Impossible if P = NPImpossible if P = NP

Possible if Factoring is HardPossible if Factoring is Hard

NP as Proof SystemNP as Proof System

Zero-Knowledge Proof Zero-Knowledge Proof SystemSystem

► Can convince Can convince someone that a someone that a solution exists solution exists without revealing without revealing any details of the any details of the solution.solution.

► Requires Requires cryptography and cryptography and verification process verification process is randomized. is randomized.

Interactive ProofsInteractive Proofs

55

► Can convince Can convince someone there is no someone there is no solution.solution.

► Verification process Verification process is randomized.is randomized.

Proof CheckingProof Checking

Probabilistically Checkable Probabilistically Checkable ProofProof

► There is a proof that There is a proof that a solution exists a solution exists where we need to where we need to only see three only see three randomly chosen randomly chosen bits of the proof.bits of the proof.

► This is the tool that This is the tool that shows that Clique is shows that Clique is hard to hard to approximate.approximate.

Quantum ComputingQuantum Computing

Quantum BitsQuantum Bits

EntanglementEntanglement

InterferenceInterference

►Use entanglement to search many states.Use entanglement to search many states.►Use interference to eliminate the ones that Use interference to eliminate the ones that

don’t solve your problem.don’t solve your problem.

Shor’s AlgorithmShor’s Algorithm

►Factor quickly with a quantum Factor quickly with a quantum computer.computer.

Complexity of FactoringComplexity of Factoring

NP

P

NPC

Complexity of QuantumComplexity of Quantum

NP

P

NPC

Quantum

Grover’s AlgorithmGrover’s Algorithm

► Search a trillion Search a trillion pieces of hay to find pieces of hay to find the needle by the needle by making only a making only a million quantum hay million quantum hay searches.searches.

► Can be used to give Can be used to give quadratic quadratic improvement to improvement to solving NP problems.solving NP problems.

Quantum Computers in Our Quantum Computers in Our Future?Future?

►A very very long way from building A very very long way from building quantum computers that can quantum computers that can outperform current machines in any outperform current machines in any task.task.

►Even if we build quantum computers, Even if we build quantum computers, factoring is only good for breaking factoring is only good for breaking some codes. Quantum computers some codes. Quantum computers could end up being a very specialized could end up being a very specialized tool.tool.

The Future of ComplexityThe Future of Complexity

►Solve the P versus NP problem.Solve the P versus NP problem.►Understanding complexity issues in Understanding complexity issues in

the computational processes outside the computational processes outside of traditional digital computers.of traditional digital computers. Biological Systems (Protein Folding)Biological Systems (Protein Folding) WeatherWeather Social InteractionsSocial Interactions Financial MarketsFinancial Markets

An Introduction to Computational Complexity Lance Fortnow University of Chicago.

Documents

Transcript of An Introduction to Computational Complexity Lance Fortnow University of Chicago.