NP-Completeness, Approximations, & Meta-Heuristics

Teófilo González

Computer Science Department

U of California, Santa Barbara

Soccer

• There are n Soccer teams participating in a soccer tournament. Every team will play against every other team two games (one in each home field).

• Can-Win-Tournament (CWT) Problem:

At mid-season we want to know whether or not a given team X can win the tournament

• Design an efficient algorithm to solve this problem. … Is it possible? …

Old FIFA Point Assignment

• Winning team: 2 points … Loosing team: 0. • Each team in a tie game receives 1 point.

• Under the old FIFA rule, the problem can be

solved efficiently using maximum flow techniques…

• The Point Assignment rule was changed to encourage teams to reduce ties and make games more interesting…

New FIFA Point Assignment

• Winning team: 3 points … Loosing team: 0.

• Each team in a tie game receives 1 point.

• Under the new FIFA rule, the CWT problem is computationally intractable. i.e., it is unlikely that there is an efficient algorithm to solve this problem.

• All known algorithms take exponential time.

Why?

• During the last 50 years we have developed methodologies to solve problems efficiently and techniques to identify computationally intractable problems.

• All of these cannot be fully understood in ones hour, or a few days, or a few months. It takes years to “fully” understand them.

NP-Completeness

• Computationally intractable problems. I.e. problem can be solve computationally, but as far as we know the resources (time) needed for its solution grow exponentially with the size of the problem.

Intractable (Polynomial Time?):

Tractable (Polynomial Time or Efficient)

Intractable vs. Tractable

• Intractable:

• Traveling Salesperson

• Knapsack

• Scheduling

• CWT (3,1,0)

• Rectangular Partitions

• Minesweeper

• Tractable

• Min Cost Spanning Tree

• Sorting

• Shortest Paths

• CWT (2,1,0)

• Guillotine Partitions

Decision Problem

• Problem whose answer is Yes or No (e.g. CWT)

• Shortest Path Problem: Given an edge weighted graph G (representing cities and roads between cities, the weights represent miles), is there a path from vertex s to vertex t with weight at most W (Is the distance between cities s and t at most W?)

• W=5 =>Yes.

• W=4 => No.

NP-Complete Problems

• A class (or group) of decision problems that have the following properties:

• Either all of them can be solved efficiently

or none of them can be solved efficiently.

Conjecture: None of them can be solved efficiently (polynomial time).

Class P

• P: The set of all decision problem that can be solved in polynomial time (O(n), O(n^2), … O(n^k) for some constant k independent of the problem size).

• NP: Set of all decision problems whose YES-answer can be verified (checked) in polynomial time. I.e., we can check correct answers in polynomial time.

• NP-Complete Problems are the “hardest” problems in NP.

• Is P = NP? Central question in Computer Science.

Solvability vs. Checking Yes-Answers

• Solving the CWT(3,1,0) in poly-time means providing an efficient algorithm that given any instance of the problem it decides if its answer is YES or NO.

• Checking a Yes-Answer for the CWT(3,1,0) in poly-time means that given any instance of the problem and an outcome for each of the remaining games, we can check in poly-time if the outcome of the games provided proves that the instance has a YES answer.

• Checking Yes-answers is “easier” than solvability.

Examples

• Solvability is like a student solving an exam.

• Checking (Verifying) is like grading an exam.

• Grading an exam is much simpler than solving an exam.

P = NP?

• Conjecture: P is different than NP. I.e., Solvability is “harder” than Verification.

• If P=NP then programming is really easy. An efficient algorithm to solve the CWT(3,1,0) can be easily derived from an algorithm that given the outcome of the remaining games checks to see if the outcomes proves that the instance is a YES-Instance (has Yes Answer).

Consequences

• If you settle the P=NP question you will be awarded $1,000,000 by the Clay Institute of Mathematics and become world famous for many centuries to come.

• If you come up with a really fast algorithm to solve any NP-Complete problem and have efficient code, then you can start your own company for solving NP-complete instances and become a billionaire quickly and …

you will be able to hire

Bill Gates to clean your windows…

How can we Prove that P = NP?

• Just design an efficient (Poly Time) algorithm to solve the CWT(3,1,0) … Very Difficult Problem.

• The NP-Completeness Theory can then be used to solve all the other NP-Complete problems.

Coping with NP-Complete Problems

• Design efficient algorithm to solve restricted version of NPC problems that arise in practice and are not NPC.

• Approximation Algorithms that provide provable good solutions

• Meta-Heuristics that solve a large number of problem instances.

• …

Approximation Algorithms

• Algorithms that guarantee near-optimal solutions to optimization problems.

Why?

Takes too long to find an optimal solution

e.g.

Input might be noisy (not exact).

GOAL: Find Near-Optimal solutions quickly.

Notation

• P: Problem (Shortest Path)

• I: Problem Instance

• : : Value of optimal solution to I.

• A: Algorithm for P (generates feasible solutions)

• : Value of solution to I generated by A.

Min Problem:

Max Problem:

Example

Bounding the Approximation

Possible Solutions

• K = 3 Three solutions for I

• K = 2 Two solutions for instance I

• K = 1 Only one solution for instance I

• For any K < 20 one solution for instance 10 I

Consequence

• For a large number of problems finding a solution with absolute error at most K, when K is a constant, is as hard as finding an optimal solution!

• Poly Time absolute approximations might not exist for all problems.

Percentage Bounding

• Epsilon is the “approximation or error”

Fully Polynomial time Approx. Scheme

• For some problems and any constant epsilon there are well known efficient approximation algorithms.

• Time Complexity Poly Time in (1/epsilon, n).

• E.g., in O((1/epsilon) n) time

• For example: Knapsack problems, some scheduling problems, …

Poly. Time Approximation Scheme

• For some problems and any constant epsilon there are well known efficient approximation algorithms.

• Time Complexity Poly Time with respect to n.

• E.g., in O(n^(1/epsilon)) time.

• Some scheduling and packing problems.

• Only of theoretical value as they cannot be used in practice.

Single-Point Approximation

• For some value of epsilon it can be approximated efficiently.

• E.g., Metric Travelling Salesperson Problem for epsilon = ½ or 1.

• Many other problems too.

Problems that cannot be Approximated unless P=NP

• Approximating the problem for any constant epsilon is as hard as finding an optimal solution.

• Travelling Salesperson Problem, Clique Problem, …

Approximations

FPTAS – PTAS – Single-Point - Inapproximability

Methodologies for Approximations

• Restriction (Algorithmic or Structural)

• Relaxation (Values or Structure)

• Rounding

• Problem Transformation

• MetaHeuristics

• Combinations of the above techniques

ICS 2016 - Chiayi, Taiwan 2

Restriction • Restrict the set of solutions for the problem and then

find an optimal or suboptimal solution to this

restricted problem.

• Restrict the solution space so that it provides some

structure which can be exploited by an algorithm.

• The restricted problem has an optimal solution value

that is not too far from the true optimal solution

value of the original problem.

• Structural or Algorithmic

ICS 2016 - Chiayi, Taiwan 3

Restriction • Solution Space --- Restricted Solution Space

ICS 2016 - Chiayi, Taiwan 4

Rectangular & Guillotine Partitions

• NPC: Optimal Rectangular Partition.

• Poly-Time: Optimal Guillotine Partition.

• Approximate RP via GP, i.e., Via Restriction.

ICS 2016 - Chiayi, Taiwan 5

0.5-Approximation

ICS 2016 - Chiayi, Taiwan 6

Relaxation • Add a new set of solutions that are not feasible for

the original problem instance in such a way that the

resulting problem can be solved in polynomial time.

• If the solution found is not feasible for the original

instance, then transform it into one.

• (Values) For example if X can only be 0 or 1, allow

0 <= X <= 1. Then round the result …

• (Structure) Instead of visiting each city exactly

once, you may visit it more than once …

ICS 2016 - Chiayi, Taiwan 7

Relaxation • Solution Space ---- Relaxed Solution Space

ICS 2016 - Chiayi, Taiwan 8

Rounding • Round the parameters of the problem in such a

way that the resulting problem can be solved

efficiently. Then use that solution to approximate

the original instance.

• For example, if the profit of an investment is

$1,000,293.45, the only use $1,000,000 instead.

ICS 2016 - Chiayi, Taiwan 9

Rounding • Solution Space --- Rounded Solution Space

ICS 2016 - Chiayi, Taiwan 10

Problem Transformation • Given an instance of problem P, construct an

instance of problem Q for which we can find easily

an optimal or suboptimal solution. Then use that

solution to generate a solution to the original

problem.

• Local Ratio: Make a “local decision” and solve the

“remaining” problem. Local decision subject to

certain constraints (good approximation to local

decision and to resulting problem).

ICS 2016 - Chiayi, Taiwan 11

Transformation • Original Problem --- Transformed Problem

ICS 2016 - Chiayi, Taiwan 12

Primal - Dual • Solution Space --- Primal-Dual Space

ICS 2016 - Chiayi, Taiwan 13

Scheduling, Bin Packing, TSP &

Steiner Trees

ICS 2016 - Chiayi, Taiwan 14

MetaHeuristics • Tabu Search

• Simulated Annealing

• Ant Colony Optimization

• Memetic Algorithms

• Evolutionary Computation

• (Sort of…) Very Large Neighborhood Search,

Reactive Search, Local Search, Stochastic Local

Search, Neural Networks, …

ICS 2016 - Chiayi, Taiwan 15

MetaHeuristics • Single Point vs. Population Based

• Fixed vs. Variable neighborhood

• Memory vs. Memoryless

• Nature-Inspired vs. Algorithmic Based

• Used in practice to solve important problem

instances.

ICS 2016 - Chiayi, Taiwan 16

Basic Approach • Exploration (Diversification) and Exploitation

(Intensification).

ICS 2016 - Chiayi, Taiwan 17

HAAM • A lot of work to be done in this area.

• It is in the boundary of theory and practice.

• 2nd Edition on its way …

ICS 2016 - Chiayi, Taiwan 18