1Introduction

It is the mark of an educated mind to rest satisfied withthe degree of precision which the nature of the subject admits

and not to seek exactness where only an approximation is possible.

— Aristotle

A man only becomes wise when he begins to calculatethe approximate depth of his ignorance.

— Gian Carlo Menotti

When exact solutions are hard to compute, approximation algorithms can help. Inthis chapter, we introduce the basic notions of approximation algorithms. We studya simple optimization problem to demonstrate the tradeoff between the time com-plexity and performance ratio of its approximation algorithms. We also present abrief introduction to the general theory of computational complexity and show howto apply this theory to classify optimization problems according to their approxima-bility.

1.1 Open Sesame

As legend has it, Ali Baba pronounced the magic words “open sesame” and foundhimself inside the secret cave of the Forty Thieves, with all their precious treasureslaid before him. After the initial excitement subsided, Ali Baba quickly realizedthat he had a difficult optimization problem to solve: He had only brought a single

1

2 Introduction

knapsack with him. Which items in the cave should he put in the knapsack in orderto maximize the total value of his find?

In modern terminology, what Ali Baba faced is a resource management problem.In this problem, one is given a fixed amount S of resources (the total volume of theknapsack) and a set of n tasks (the collection of treasures in the cave). Completingeach task requires a certain amount of resources and gains a certain amount of profit.The problem is to maximize the total profit, subject to the condition that the totalresources used do not exceed S. Formally, we can describe Ali Baba’s problem asfollows:

Given n items I1, I2, . . . , In, a volume si and a value ci for each itemIi, 1 ≤ i ≤ n, and an integer S, find a subsetA of items that maximizesthe total value

∑Ii∈A ci, subject to the condition that the total volume∑

Ii∈A si does not exceed S.

We can introduce, for each 1 ≤ i ≤ n, a 0–1 variable xi to represent item Ii inthe following sense:

xi =

1, if Ii ∈ A,

0, if Ii 6∈ A.

Then, Ali Baba’s problem can be reformulated as a 0–1 integer programming prob-lem:

KNAPSACK: Given 2n + 1 positive integers S, s1, s2, . . . , sn andc1, c2, . . . , cn,

maximize c(x) = c1x1 + c2x2 + · · ·+ cnxn,

subject to s1x1 + s2x2 + · · ·+ snxn ≤ S,x1, x2, . . . , xn ∈ 0, 1.

Notation. (1) In this book, we will use the following notation about an optimizationproblem Π: On an input instance I of Π, we write Opt(I) to denote the optimalsolution of the instance I , and opt(I) to denote the optimum value of the objectivefunction on input I . When there is no confusion, we write Opt and opt for Opt(I)and opt(I), respectively. In addition, for convenience, we often write, for an ob-jective function f(x), f∗ to denote the optimum value of the function f , and x∗

to denote the value of x that achieves the optimum value f∗. For instance, for theproblem KNAPSACK above, we write opt or c∗ to denote the maximum value of c(x)under the given constraints, and Opt or x∗ to denote the value of (x1, x2, . . . , xn)that makes

∑ni=1 cixi = c∗.

(2) For the sets of numbers, we write N to denote the set of natural numbers (i.e.,the set of nonnegative integers), Z the set of integers, Z+ the set of positive integers,R the set of real numbers, and R+ the set of positive integers.

Following the above convention, let opt denote the optimum value of the objec-tive function c(x). Without loss of generality, we may assume that sk ≤ S for all

1.1 Open Sesame 3

k = 1, . . . , n. In fact, if sk > S, then we must have xk = 0, and so we need notconsider the kth item at all. This assumption implies that opt ≥ max1≤k≤n ck.

There are many different approaches to attacking the problem KNAPSACK. First,let us use the dynamic programming technique to find the exact solutions for KNAP-SACK.

To simplify the description of the algorithm, we first define some notations. Forany subset I ⊆ 1, . . . , n, let SI denote the sum

∑k∈I sk. For each pair (i, j),

with 1 ≤ i ≤ n, 0 ≤ j ≤∑n

i=1 ci, if there exists a set I ⊆ 1, 2, . . . , n such that∑k∈I ck = j and SI ≤ S, then let a(i, j) denote such a set I with the minimum

SI . If such an index subset I does not exist, then we say that a(i, j) is undefined,and write a(i, j) = nil.

Using the above notation, it is clear that opt = maxj | a(n, j) 6= nil. There-fore, it suffices to compute all values of a(i, j). The following algorithm is based onthis idea.1

Algorithm 1.A (Exact Algorithm for KNAPSACK)

Input: Positive integers S, s1, s2, . . . , sn, c1, c2, . . . , cn.

(1) Let csum ←n∑

i=1

ci.

(2) For j ← 0 to csum do

if j = 0 then a(1, j)← ∅else if j = c1 then a(1, j)← 1 else a(1, j)← nil.

(3) For i← 2 to n do

for j ← 0 to csum do

if [a(i− 1, j − ci) 6= nil] and [Sa(i−1,j−ci) ≤ S − si]and [a(i− 1, j) 6= nil⇒ Sa(i−1,j) > Sa(i−1,j−ci) + si]

then a(i, j)← a(i− 1, j − ci) ∪ ielse a(i, j)← a(i− 1, j).

(4) Output c∗ ← maxj | a(n, j) 6= nil.

It is not hard to verify that this algorithm always finds the optimal solutions toKNAPSACK (see Exercise 1.1).

Next, we consider the time complexity of Algorithm 1.A. Since Ali Baba had toload the treasures and leave the cave before the Forty Thieves came back, he neededan efficient algorithm. It is easy to see that, for any I ⊆ 1, . . . , n, it takes timeO(n logS) to compute SI .2 Thus, Algorithm 1.A runs in time O(n3M log(MS))where M = maxck | 1 ≤ k ≤ n [note that csum = O(nM)]. We note that

1We use the standard pseudocodes to describe an algorithm; see, e.g., Cormen et al. [2001].2In the rest of the book, we write log k to denote log2 k.

4 Introduction

the input size of the problem is n logM + logS (assuming that the input integersare written in the binary form). Therefore, Algorithm 1.A is not a polynomial-timealgorithm. It is actually a pseudo polynomial-time algorithm, in the sense that it runsin time polynomial in the maximum input value but not necessarily polynomial inthe input size. Since the input value could be very large, a pseudo polynomial-timealgorithm is usually not considered as an efficient algorithm. To be sure, if Ali Babatried to run this algorithm, then the Forty Thieves would definitely have come backbefore he got the solution—even if he could calculate as fast as a modern digitalcomputer.

As a compromise, Ali Baba might find a fast approximation algorithm more use-ful. For instance, the following is such an approximation algorithm, which uses asimple greedy strategy that selects the heaviest item (i.e., the item with the greatestdensity ci/si) first.

Algorithm 1.B (Greedy Algorithm for KNAPSACK)Input: Positive integers S, s1, s2, . . . , sn, c1, c2, . . . , cn.

(1) Sort all items in the nonincreasing order of ci/si. Without loss of generality,assume that c1/s1 ≥ c2/s2 ≥ · · · ≥ cn/sn.

(2) Ifn∑

i=1

si ≤ S then output cG←

n∑i=1

ci

else k ← max

j

∣∣∣∣ j∑i=1

si ≤ S <j+1∑i=1

si

;

output cG← max

ck+1,

k∑i=1

ci

.

It is clear that this greedy algorithm runs in time O(n log(nMS)) and hence isvery efficient. The following theorem shows that it produces an approximate solu-tion not very far from the optimum.

Theorem 1.1 Let opt be the optimal solution of the problem KNAPSACK and cG

the approximate solution obtained by Algorithm 1.B. Then opt ≤ 2cG

(and we saythat the performance ratio of Algorithm 1.B is bounded by the constant 2).

Proof. For convenience, write c∗ for opt. If∑n

i=1 si ≤ S, then cG

= c∗. Thus, wemay assume

∑ni=1 si > S. Let k be the integer found by Algorithm 1.B in step (2).

We claim thatk∑

i=1

ci ≤ c∗ <k+1∑i=1

ci. (1.1)

The first half of the above inequality holds trivially. For the second half, we notethat, in step (1), we sorted the items according to their density, ci/si. Therefore, ifwe are allowed to cut each item into smaller pieces, then the most efficient way ofusing the knapsack is to load the first k items, plus a portion of the (k + 1)st itemthat fills the knapsack, because replacing any portion of these items by other items

1.1 Open Sesame 5

decreases the total density of the knapsack. This shows that the maximum total valuec∗ we can get is less than

∑k+1i=1 ci.

We can also view the above argument in terms of linear programming. That is,if we replace the constraints xi ∈ 0, 1 by 0 ≤ xi ≤ 1, then we obtain a linearprogram which has the maximum objective function value c ≥ c∗. It is easy to checkthat the following assignment is an optimal solution to this linear program3:

xj =

1, for j = 1, 2, . . . , k,(S −

∑ki=1 si

)/sk+1, for j = k + 1,

0, for j = k + 2, . . . , n.

Therefore,

c∗ ≤ c =

k∑i=1

ci +ck+1

sk+1

(S −

k∑i=1

si

)<

k∑i=1

ci +ck+1

sk+1sk+1 =

k+1∑i=1

ci.

Finally, it is obvious that, from (1.1), we have

cG

= max

ck+1,

k∑i=1

ci

≥ 1

2

k+1∑i=1

ci >c∗

2.

The above two algorithms demonstrate an interesting tradeoff between the run-ning time and the accuracy of an algorithm: If we sacrifice a little in the accuracyof the solution, we may get a much more efficient algorithm. Indeed, we can furtherexplore this idea of tradeoff and show a spectrum of approximation algorithms withdifferent running time and accuracy.

First, we show how to generalize the above greedy algorithm to get better ap-proximate solutions—with worse, but still polynomial, running time. The idea isas follows: We divide all items into two groups: those with values ci ≤ a andthose with ci > a, where a is a fixed parameter. Note that in any feasible solutionI ⊆ 1, 2, . . . , n, there can be at most opt/a ≤ 2c

G/a items that have values

ci greater than a. So we can perform an exhaustive search over all index subsetsI ⊆ 1, 2, . . . , n of size at most 2c

G/a from the second group as follows: For

each subset I , use the greedy strategy on the first group to get a solution of the to-tal volume no greater than S − SI , and combine it with I to get an approximatesolution. From Theorem 1.1, we know that our error is bounded by the value of asingle item of the first group, which is at most a. In addition, we note that there areat most n2cG/a index subsets of the second group to be searched through, and so therunning time is still a polynomial function in the input size.

In the following, we write |A| to denote the size of a finite set A.

Algorithm 1.C (Generalized Greedy Algorithm for KNAPSACK)Input: Positive integers S, s1, s2, . . . , sn, c1, c2, . . . , cn, and a constant 0 < ε < 1.

3See Chapter 7 for a more complete treatment of linear programming.

6 Introduction

(1) Run Algorithm 1.B on the input to get value cG

.

(2) Let a← εcG

.

(3) Let Ia ← i | 1 ≤ i ≤ n, ci ≤ a. (Without loss of generality, assume thatIa = 1, . . . ,m, where m ≤ n.)

(4) Sort the items in Ia in the nonincreasing order of ci/si. Without loss of gener-ality, assume that c1/s1 ≥ c2/s2 ≥ · · · ≥ cm/sm.

(5) For each I ⊆ m+ 1,m+ 2, . . . , n with |I| ≤ 2/ε do

if∑i∈I

si > S then c(I)← 0

else ifm∑i=1

si ≤ S −∑i∈I

si

then c(I)←m∑i=1

ci +∑i∈I

ci

else k ← max

j

∣∣∣∣ j∑i=1

si ≤ S −∑i∈I

si <

j+1∑i=1

si

;

c(I)←k∑

i=1

ci +∑i∈I

ci.

(6) Output cGG← maxc(I) | I ⊆ m+ 1,m+ 2, . . . , n, |I| ≤ 2/ε.

Theorem 1.2 Let opt be the optimal solution to KNAPSACK and cGG

the approxi-mation obtained by Algorithm 1.C. Then opt ≤ (1 + ε)c

GG. Moreover, Algorithm

1.C runs in time O(n1+2/ε log(nMS)).

Proof. For convenience, write c∗ = opt and let I∗ = Opt be the optimal index set;that is,

∑i∈I∗ ci = c∗ and

∑i∈I∗ si ≤ S. Define I = i ∈ I∗ | ci > a. We have

already shown that |I| ≤ c∗/a ≤ 2cG/a = 2/ε. Therefore, in step (5) of Algorithm

1.C, the index set I will eventually be set to I . Then, the greedy strategy, as shownin the proof of Theorem 1.1, will find c(I) with the property

c(I) ≤ c∗ ≤ c(I) + a.

Since cGG

is the maximum c(I), we get

c(I) ≤ cGG≤ c∗ ≤ c(I) + a ≤ c

GG+ a.

Let IG denote the set obtained by Algorithm 1.B on the input. Let IG = i ∈ IG |ci > a. Then |IG| ≤ cG/a = 1/ε. So, we will process set IG in step (5) and getc(IG) = cG. It means c

GG≥ c

G, and so

c∗ ≤ cGG

+ a = cGG

+ εcG≤ (1 + ε)c

GG.

1.1 Open Sesame 7

Note that there are at most n2/ε index sets I of size |I| ≤ 2/ε. Therefore, therunning time of Algorithm 1.C is O(n1+2/ε log(nMS)).

By Theorem 1.2, for any fixed ε > 0, Algorithm 1.C runs in time O(n1+2/ε

log(nMS)) and hence is a polynomial-time algorithm. As ε decreases to zero, how-ever, the running time increases exponentially with respect to 1/ε. Can we slowdown the speed of increase of the running time with respect to 1/ε? The answer isyes. The following is such an approximation algorithm:

Algorithm 1.D (Polynomial Tradeoff Approximation for KNAPSACK)Input: Positive integers S, s1, s2, . . . , sn, c1, c2, . . . , cn, and an integer h > 0.

(1) For k ← 1 to n do

c′k ←⌊ckn(h+ 1)

M

⌋, where M = max

1≤i≤nci.

(2) Run Algorithm 1.A on the following instance of KNAPSACK:maximize c′1x1 + c′2x2 + · · ·+ c′nxn

subject to s1x1 + s2x2 + · · ·+ snxn ≤ S,x1, x2, . . . , xn ∈ 0, 1.

(1.2)

Let (x∗1, . . . , x∗n) be the optimal solution found by Algorithm 1.A [i.e., the

index set corresponding to the optimum value opt′ = (c′)∗ of (1.2)].

(3) Output cPT← c1x

∗1 + · · ·+ cnx

∗n.

Theorem 1.3 The solution obtained by Algorithm 1.D satisfies the relationship

opt

cPT

≤ 1 +1

h,

where opt is the optimal solution to the input instance.

Proof. For convenience, let c∗ = opt and I∗ = Opt be the optimal index set of theinput instance; that is, c∗ =

∑k∈I∗ ck. Also, let J∗ be the index set found in step

(2); that is, J∗ = k | 1 ≤ k ≤ n, x∗k = 1. Then, we have

cPT

=∑k∈J∗

ck =∑k∈J∗

ckn(h+ 1)

M· M

n(h+ 1)

≥∑k∈J∗

⌊ckn(h+ 1)

M

⌋· M

n(h+ 1)

=M

n(h+ 1)

∑k∈J∗

c′k ≥M

n(h+ 1)

∑k∈I∗

c′k

≥ M

n(h+ 1)

∑k∈I∗

(ckn(h+ 1)

M− 1

)≥ c∗ − M

h+ 1≥ c∗

(1− 1

h+ 1

).

8 Introduction

In the above, the second inequality holds because J∗ is the optimal solution tothe modified instance of KNAPSACK; and the last inequality holds because M =max1≤i≤nci ≤ c∗. Thus,

c∗

cPT

≤ 1

1− 1/(h+ 1)= 1 +

1

h.

We note that in step (2), the running time for Algorithm 1.A on the modifiedinstance is O(n3M ′ log(M ′S)), where M ′ = maxc′k | 1 ≤ k ≤ n ≤ n(h + 1).Therefore, the total running time of Algorithm 1.D is O(n4h log(nhS)), which isa polynomial function with respect to n, logS, and h = 1/ε. Thus, the tradeoffbetween running time and approximation ratio of Algorithm 1.D is better than thatof the generalized greedy algorithm.

From the above analysis, we learned that if we turn our attention from the opti-mal solutions to the approximate solutions, then we may find many new ideas andtechniques to attack the problem. Indeed, the design and analysis of approximationalgorithms are very different from that of exact (or, optimal) algorithms. It is a cavewith a mother lode of hidden treasures. Let us say “Open Sesame” and find out whatthey are.

1.2 Design Techniques for Approximation Algorithms

What makes the design and analysis of approximation algorithms so different fromthat of algorithms that search for exact solutions?4

First, they study different types of problems. Algorithms that look for exact solu-tions work only for tractable problems, but approximation algorithms apply mainlyto intractable problems. By tractable problems, we mean, in general, problems thatcan be solved exactly in polynomial time in the input size. While tractable prob-lems, such as the minimum spanning tree problem, the shortest path problem, andmaximum matching are the main focus of most textbooks for algorithms, most in-tractable problems are not discussed in these books. On the other hand, a greatnumber of problems we encounter in the research literature, such as the travelingsalesman problem, scheduling, and integer programming, are intractable. That is,no polynomial-time exact algorithms have been found for them so far. In addi-tion, through the study of computational complexity theory, people have provedthat most of these problems are unlikely to have polynomial-time exact algorithmsat all. Therefore, approximation algorithms seem to be the only resort.

Second, and more importantly, they emphasize different aspects of the perfor-mance of the algorithms. For algorithms that look for exact solutions, the most im-portant issue is the efficiency, or the running time, of the algorithms. Data structuresand design techniques are introduced mainly to improve the running time. For ap-proximation algorithms, the running time is, of course, still an important issue. It,

4We call such algorithms exact algorithms.

1.2 Design Techniques for Approximation Algorithms 9

however, has to be considered together with the performance ratio (the estimate ofhow close the approximate solutions are to the optimal solutions) of the algorithms.As we have seen in the study of the KNAPSACK problem, the tradeoff between therunning time and performance ratio is a critical issue in the analysis of approxi-mation algorithms. Many design techniques for approximation algorithms aim toimprove the performance ratio with the minimum extra running time.

To illustrate this point, let us take a closer look at approximation algorithms.First, we observe that, in general, an optimization problem may be formulated inthe following form:

minimize (or, maximize) f(x1, x2, . . . , xn)

subject to (x1, x2, . . . , xn) ∈ Ω,(1.3)

where f is a real-valued function and Ω a subset of Rn. We call the function fthe objective function and set Ω the feasible domain (or, the feasible region) of theproblem.

The design of approximation algorithms for such a problem can roughly be di-vided into two steps. In the first step, we convert the underlying intractable probleminto a tractable variation by perturbing the input values, the objective function, orthe feasible domain of the original problem. In the second step, we design an effi-cient exact algorithm for the tractable variation and, if necessary, convert its solutionback to an approximate solution for the original problem. For instance, in Algorithm1.D, we first perturb the inputs ci into smaller c′i, and thus converted the originalKNAPSACK problem into a tractable version of KNAPSACK in which the maximumparameter c′i is no greater than n(h+ 1). Then, in the second step, we use the tech-nique of dynamic programming to solve the tractable version in polynomial time,and use the optimal solution (x∗1, x

∗2, . . . , x∗n) of the tractable version of KNAPSACK

as an approximate solution to the original instance of KNAPSACK.It is thus clear that in order to design good approximation algorithms, we must

know how to perturb the original intractable problem to a tractable variation suchthat the solution to the tractable problem is closely related to that of the originalproblem. A number of techniques for such perturbation have been developed. Theperturbation may act on the objective functions, as in the greedy strategy and thelocal search method. It may involve changes to the feasible domain, as in the tech-niques of restriction and relaxation. It may sometimes also perform some operationson the inputs, as in the technique of power graphs. These techniques are very differ-ent from the techniques for the design of efficient exact algorithms, such as divideand conquer, dynamic programming, and linear programming. The study of thesedesign techniques forms an important part of the theory of approximation algo-rithms. Indeed, this book is organized according to the classification of these designtechniques. In the following, we give a brief overview of these techniques and theorganization of the book (see Figure 1.1).

In Chapter 2, we present a theory of greedy strategies, in which we demonstratehow to use the notions of independent systems and submodular potential functions

10 Introduction

Chapter 1Introduction

=

ZZZZ~

Chapter 2Greedy Strategy

ZZZ~

=

Chapter 3Restriction

?Chapter 4Partition

?Chapter 5

Guillotine Cut

Chapter 6Relaxation

?Chapter 7

Linear Programming

?Chapter 8

Primal-Dual Schemaand Local Ratio

?Chapter 9

Semidefinite Programming

Chapter 10Inapproximability

Figure 1.1: Relationships among chapters.

to analyze the performance of greedy algorithms. Due to space limits, we will omitthe related but more involved method of local search.

The technique of restriction is studied in Chapters 3–5. The basic idea of re-striction is very simple: If we narrow down the feasible domain, the solutions maybecome easier to find. There are many different ways to restrict the feasible domains,depending on the nature of the problems. We present some simple applications inChapter 3. Two of the most important techniques of restriction, partition and Guil-lotine cut, are then studied in detail in Chapters 4 and 5, respectively.

In Chapters 6–9, we study the technique of relaxation. In contrast to restriction,the technique of relaxation is to enlarge the feasible domain to include solutionswhich are considered infeasible in the original problem so that different design tech-niques can be applied. A common implementation of the relaxation technique is asfollows: First, we formulate the problem into an integer programming problem [i.e.,a problem in the form of (1.3) with Ω ⊆ Zn]. Then, we relax this integer programinto a linear program by removing the integral constraints on the variables. After wesolve this relaxed linear program, we round the real-valued solution into integers anduse them as the approximate solution to the original problem. Linear programming,

1.2 Design Techniques for Approximation Algorithms 11

solution forthe original problem

?

estimation

solution forthe restricted problem

solution forthe relaxed problem

?

estimation

solution forthe original problem

Figure 1.2: Analysis of approximation algorithms based on restriction and relax-ation.

the primal-dual method, and the local ratio method are the main techniques in thisapproach. We study these techniques in Chapters 7 and 8. In addition to the linearprogramming technique, it has recently been found that semidefinite programmingcan also be applied in such a relaxation approach. We present the theory of semidef-inite programming and its application to approximation algorithms in Chapter 9.

We remark that an important step in the analysis of approximation algorithmsis the estimation of the errors created by the perturbation of the feasible domain.For the algorithms based on the restriction and relaxation techniques, this error es-timation often uses similar methods. To analyze an algorithm designed with therestriction technique, one usually takes an optimal solution for the original problemand modifies it to meet the restriction, and then estimates the errors that occurredin the modification. For the algorithms designed with the relaxation technique, thekey part of the analysis is about rounding the solution, or estimating the errors thatoccurred in the transformation from the solution for the relaxed problem to the so-lution for the original problem. Therefore, in both cases, a key step in the analysis isthe estimation of the change of solutions from those in a larger (or, relaxed) domainto those in a smaller (or, restricted) domain (see Figure 1.2).

To explain this observation more clearly, let us consider a minimization problemminx∈Ω f(x) as defined in (1.3), where x denotes a vector (x1, x2, . . . , xn) in Rn.Assume that x∗ ∈ Ω satisfies f(x∗) = minx∈Ω f(x). Suppose we restrict the fea-sible domain to a subregion Γ of Ω and find an optimal solution y∗ for the restrictedproblem; that is, f(y∗) = minx∈Γ f(x). Then, we may analyze the performanceof y∗ as an approximate solution to the original problem in the following way (seeFigure 1.3):

(1) Consider a minimum solution x∗ of minx∈Ω f(x).

(2) Modify x∗ to obtain a feasible solution y of minx∈Γ f(x).

12 Introduction

Figure 1.3: Analysis of the restriction and relaxation approximations.

(3) Estimate the value of f(y)/f(x∗), and use it as an upper bound for the per-formance ratio for the approximate solution y∗, since y ∈ Γ implies

f(y∗)

f(x∗)≤ f(y)

f(x∗).

Similarly, consider the problem minx∈Γ f(x). Suppose we relax the feasible re-gion Γ to a bigger region Ω, and find the optimal solution x∗ for the relaxed prob-lem; that is, f(x∗) = minx∈Ω f(x). Then, we can round x∗ into a solution y ∈ Γand use it as an approximate solution to the original problem. The analysis of thisrelaxation algorithm can now be done as follows:

• Estimate the value f(y)/f(x∗), and use it as an upper bound for the perfor-mance ratio for the approximate solution y, since, for any optimal solution y∗

for the original problem, we have f(x∗) ≤ f(y∗), and hence

f(y)

f(y∗)≤ f(y)

f(x∗).

Thus, in both cases, the analysis of the performance of the approximate solution isreduced to the estimation of the ratio f(y)/f(x∗).

Notice, however, a critical difference in the above analyses. In the case of therestriction algorithms, the change from x∗ to y is part of the analysis of the algo-rithm, and we are not concerned with the time complexity of this change. On theother hand, in the case of the relaxation algorithms, this change is a step in the ap-proximation algorithm, and has to be done in polynomial time. As a consequence,while the method of rounding for the analysis of the relaxation algorithms may, ingeneral, be applied to the analysis of the restriction algorithms, the converse maynot be true; that is, the analysis techniques developed for the restriction algorithmsare not necessarily extendable to the analysis of the relaxation algorithms.

1.3 Heuristics Versus Approximation 13

1.3 Heuristics Versus Approximation

In the literature, the word “heuristics” often appears in the study of intractable prob-lems and is sometimes used interchangeably with the word “approximation.” In thisbook, however, we will use it in a different context and distinguish it from approxi-mation algorithms. The first difference between heuristics and approximation is thatapproximation algorithms usually have guaranteed (worst-case) performance ratios,while heuristic algorithms may not have such guarantees. In other words, approxi-mations are usually justified with theoretical analysis, while heuristics often appealto empirical data.

The second difference is that approximation usually applies to optimization prob-lems, while heuristics may also apply to decision problems. Let us look at an exam-ple. First, we define some terminologies about Boolean formulas. A Boolean for-mula is a formula formed by operations ∨ (OR), ∧ (AND), and ¬ (NOT) over Booleanconstants 0 (FALSE) and 1 (TRUE) and Boolean variables. For convenience, we alsouse + for OR and · for AND, and write x to denote ¬x. An assignment to a Booleanformula φ is a function mapping each Boolean variable in φ to a Boolean constant0 or 1. A truth assignment is an assignment that makes the resulting formula TRUE.We say a Boolean formula is satisfiable if it has a truth assignment. For instance,the Boolean formula

(v1v2 + v1v3v4 + v2v3)(v1v3 + v2v4)

over the variables v1, . . . , v4 is satisfiable, since the assignment τ(v1) = τ(v3) = 1and τ(v2) = τ(v4) = 0 is a truth assignment for it.

Now, consider the following problem.

SATISFIABILITY (SAT): Given a Boolean formula, determine whetherit is satisfiable.

This is not an optimization problem. Therefore, it does not make much senseto try to develop an approximation algorithm for this problem, though there are anumber of heuristics, such as the resolution method, developed for this problem.Such heuristics may work efficiently for a large subset of the input instances, butthey do not guarantee to solve all instances in polynomial time.

Although approximations and heuristics are different concepts, their ideas andtechniques can often be borrowed from each other. Theoretical analysis of approx-imation algorithms could provide interesting ideas for heuristic algorithms. In ad-dition, for some decision problem, we may first convert it into an equivalent opti-mization problem, and then adapt the approximation algorithms for the optimizationproblem to heuristic algorithms for the original decision problem. For instance, wemay use the approximation algorithms for integer programming to develop a heuris-tic algorithm for SAT as follows.

We first convert the problem SAT into an optimization problem. Let v1, v2, . . . ,vn be Boolean variables and v the vector (v1, v2, . . . , vn) in 0, 1n. Let y1, y2, . . . ,yn be real variables and y the vector (y1, y2, . . . , yn) in Rn. For each Boolean func-tion f(v), we define a real function Ff (y) recursively as follows:

14 Introduction

(1) Initially, if f(v) = vi, then set Ff (y)← yi; if f(v) = 0, then set Ff (y)← 0;and if f(v) = 1 , then set Ff (y)← 1.

(2) Inductively, if f(v) = g(v) ∨ h(v), then set Ff (y) ← Fg(y) + Fh(y) −Fg(y) ·Fh(y); if f(v) = g(v)∧h(v), then set Ff (y)← Fg(y) ·Fh(y); andif f(v) = ¬g(v), then set Ff (y)← 1− Fg(y).

The above construction converts the decision problem SAT into an equivalentoptimization problem, in the sense that a Boolean formula f(v) is satisfiable ifand only if the following 0–1 integer program has a positive maximum objectivefunction value:

maximize Ff (y)

subject to y ∈ 0, 1n.

Although this new problem is still intractable, it is nevertheless an optimizationproblem, and the approximation techniques for 0-1 integer programming are appli-cable. These approximation algorithms could then be studied and developed into aheuristic for the decision version of SAT.

Historically, heuristic algorithms have appeared much earlier than approximationalgorithms. The first documented approximation algorithm was discovered by Gra-ham [1966] for a scheduling problem, while heuristic algorithms probably existed,at least in the informal form, as early as the concept of algorithms was developed.The existence of the rich families of heuristics and their wide applications encourageus to develop them into new approximation algorithms. For instance, an importantidea for many heuristics is to link the discrete space of a combinatorial optimiza-tion problem to the continuous space of a nonlinear optimization problem throughgeometric, analytic, or algebraic techniques, and then to apply the nonlinear opti-mization algorithms to the combinatorial optimization problems. Researchers havefound that this approach often leads to very fast and effective heuristics for combi-natorial optimization problems of a large scale. However, most of these heuristics,with a few exceptions such as the interior point method for linear programming,though working well in practice, do not have a solid theoretical foundation. Theo-retical analyses for these algorithms could provide new, surprising approximationalgorithms.

1.4 Notions in Computational Complexity

Roughly speaking, the main reason for studying approximation algorithms is to findefficient, but not necessarily optimal, solutions to intractable problems. We haveinformally defined an intractable problem to be a problem which does not have apolynomial-time algorithm. From the theoretical standpoint, there are, in this in-formal definition, several important issues that have not been clearly addressed. Forinstance, why do we identify polynomial-time computability with tractability? Doespolynomial-time computability depend on the computational model that we use toimplement the algorithm? How do we determine, in general, whether a problem hasa polynomial-time algorithm? These fundamental issues have been carefully exam-

1.4 Computational Complexity 15

ined in the theory of computational complexity. We present, in this and the nextsections, a brief summary of this theory. The interested reader is referred to Du andKo [2000] for more details.

The time complexity of an algorithm refers to the running time of the algorithmas a function of the input size. As a convention, in the worst-case analysis, we takethe maximum running time over all inputs of the same size n as the time complexityof the algorithm on size n. In order to estimate the running time of an algorithm, wemust specify the computational model in which the algorithm is implemented. Sev-eral standard computational models have been carefully studied. Here, we consideronly two simple models: the pseudocode and the Turing machine.

We have already used pseudocodes to express algorithms in Section 1.1. Pseu-docodes are an informal high-level programming language, similar to standard pro-gramming languages such as Pascal, C, and Java, without complicated languageconstructs such as advanced data structures and parameter-passing schemes in pro-cedure calls. It is an abstract programming language in the sense that each variablein a procedure represents a memory location that holds an integer or a real number,without a size limit. We assume the reader is familiar with such high-level program-ming languages and understands the basic syntax and semantics of pseudocodes.The reader who is not familiar with pseudocodes is referred to any standard algo-rithm textbook.

When an algorithm is expressed in the form of a program in pseudocode, it isnatural to use the number of statements or the number of arithmetic and comparisonoperations as the basic measure for the time complexity of the algorithm. This timecomplexity measure is simple to estimate but does not reflect the exact complexity ofthe algorithm. For instance, consider the following simple procedure that computesthe function f(a,m) = am, where a and m are two positive integers:

b← 1;For k ← 1 to m do b← b · a;Output b.

It is not hard to see that, on any input (a,m), the number of operations to be executedin the above algorithm isO(m), independent of the size n of the other input numbera. However, a detailed analysis shows that the size of b increases from 1 bit to aboutnm bits in the computation of the algorithm, and yet we counted only one unit oftime for the multiplication of b and a, no matter how large b is. This does not seemto reflect the real complexity of the algorithm. A more accurate estimate of thetime complexity should take into account the size of the operands of the arithmeticoperations. For instance, the logarithmic cost measure countsO(log n) units of timefor each arithmetic or comparison operation that is executed on operands whosevalues are at most n. Thus, the time complexity of the above algorithm for am,under the logarithmic cost measure, would be O(m2 log a).

We note that even using the logarithmic cost measure does not give the time com-plexity of the algorithm completely correctly. Indeed, the logarithmic cost measureis based on the assumption that arithmetic or comparison operations on operands ofn bits can be executed inO(n) units of time (in other words, these operations can be

16 Introduction

implemented in linear time). This assumption is plausible for simple operations, butnot for more complicated operations such as multiplication and division. Indeed, nolinear-time multiplication algorithm is known. The best algorithm known today formultiplying two n-bit integers requires Ω(n log n) units of time. Therefore, the log-arithmic cost measure tends to underestimate the complexity of an algorithm withheavy multiplications.

To more accurately reflect the exact complexity of an algorithm, we usually usea primitive computational model, called the Turing machine. We refer the reader totextbooks of theory of computation, for instance, Du and Ko [2000], for the defini-tion of a Turing machine. Here, it suffices to summarize that (1) all input, output,and temporary data of the computation of a Turing machine are stored on a finitenumber of tapes, with one single character stored in one cell of the tape, and (2) eachinstruction of the Turing machine works on one cell of the tape, either changing thecharacter stored in the cell or moving its tape head to one of its neighboring cells.That is, the complexity measure of the Turing machine is a bit-operation measure,which most closely represents our intuitive notion of time complexity measure.

The instructions of Turing machines are very simple and so it makes the anal-ysis of the computation of a Turing machine easier. In particular, it allows us toprove lower bounds of a problem, which is difficult to do for more complicatedcomputational models. However, one might suspect whether we can implement so-phisticated algorithms with, for instance, advanced data structures and complicatedrecursive calls in such a simplistic machine and, even if so, whether the imple-mentation is as efficient as more general models. It turns out that Turing machines,though primitive, can simulate all known computational models efficiently in thefollowing sense: For any algorithm that can be implemented in the model in ques-tion with time complexity t(n), there is a Turing machine implementing this algo-rithm in time p(t(n)), where p is a polynomial function depending on the modelbut independent of the algorithms. In fact, a widely accepted hypothesis, called theextended Church–Turing thesis, states that a Turing machine can simulate any rea-sonable deterministic computational model within polynomial time. In other words,polynomial-time computability is a notion that is independent of the computationalmodels used to implement the algorithms.

Based on the extended Church–Turing thesis, we now formally identify the classof tractable problems with the following complexity class:

P: the class of all decision problems that are solvable in polynomial time by adeterministic Turing machine.

In other words, we say a problem is tractable if there is a Turing machine M thatsolves the problem in polynomial time in the input size [i.e., M runs in time O(nk),where n is the input size and k is a constant]. We note that the composition oftwo polynomial functions is still a polynomial function. Thus, the combination oftwo polynomial-time algorithms is still a polynomial-time algorithm. This reflectsthe intuition that the combination of two tractable algorithms should be consideredtractable.

1.5 NP-Complete Problems 17

Now, let us go back to our choice of using pseudocodes to describe algorithms.From the above discussion, we may assume (and, in fact, prove) that the logarithmiccost measure of a pseudocode procedure and the bit-operation complexity of anequivalent Turing machine program are within a polynomial factor. Therefore, inorder to demonstrate that a problem is tractable, we can simply present the algorithmin a pseudocode procedure and perform a simple time analysis of the procedure. Onthe other hand, to show that a problem is intractable, we usually use Turing machinesas the computational model.

1.5 NP-Complete Problems

In the study of computational complexity, an optimization problem is usually for-mulated into an equivalent decision problem, whose answer is either YES or NO.For instance, we can formulate the problem KNAPSACK into the following decisionproblem:

KNAPSACKD: Given 2n+ 2 integers: S,K, s1, s2, . . . , sn, c1, c2, . . . ,cn, determine whether there is a sequence (x1, x2, . . . , xn) ∈ 0, 1nsuch that

∑ni=1 sixi ≤ S and

∑ni=1 cixi ≥ K.

It is not hard to see that KNAPSACK and KNAPSACKD are equivalent, in the sensethat they are either both tractable or both intractable.

Proposition 1.4 The optimization problem KNAPSACK is polynomial-time solvableif and only if the decision problem KNAPSACKD is polynomial-time solvable.

Proof. Suppose the optimization problem KNAPSACK is polynomial-time solvable.Then, we can solve the decision problem KNAPSACKD by finding the optimal so-lution opt of the corresponding KNAPSACK instance and then answering YES if andonly if opt ≥ K.

Conversely, suppose KNAPSACKD is solvable in polynomial time by a Turingmachine M . Assume that M runs in time O(Nk), where N is the input size and kis a constant. Now, on input I = (S, s1, . . . , sn, c1, . . . , cn) to the problem KNAP-SACK, we can binary search for the maximum K such that M answers YES oninput (S,K, s1, . . . , sn, c1, . . . , cn). This maximum value K is exactly the opti-mal solution opt for input I of the problem KNAPSACK. Note that K satisfiesK ≤ M2 =

∑ni=1 ci. Thus, the above binary search needs to simulate M for at

most blogM2 + 1c = O(N) times, where N is the size of input I . So, we can solveKNAPSACK in time O(Nk+1).

From the discussion of the last section, in order to prove a problem intractable, weneed to show that (the decision version of) the problem is not in P. Unfortunately, fora great number of optimization problems, there is strong evidence, both empiricaland mathematical, suggesting that they are likely intractable, but no one is ableto find a formal proof that they are not in P. Most of these problems, however,share a common property called NP-completeness. That is, they can be solved by

18 Introduction

nondeterministic algorithms in polynomial time and, furthermore, if any of theseproblems is proved to be not in P, then all of these problems are not in P.

A nondeterministic algorithm is an algorithm that can make nondeterministicmoves. In a nondeterministic move, the algorithm can assign a value of either 0 or1 to a variable nondeterministically, so that the computation of the algorithm afterthis step branches into two separate computation paths, each using a different valuefor the variable. Suppose a nondeterministic algorithm executes nondeterministicmoves k times. Then it may generate 2k different deterministic computation paths,some of which may output YES and some of which may output NO. We say the non-deterministic algorithm accepts the input (i.e., answers YES) if at least one of thecomputation paths outputs YES; and the nondeterministic algorithm rejects the inputif all computation paths output NO. (Thus, the actions of accepting and rejecting aninput of a nondeterministic algorithm A are not symmetric: If we change each an-swer YES of a computation path to answer NO, and each NO to YES, the collectivesolution of A does not necessarily change from accepting to rejecting.) On each in-put x accepted by a nondeterministic algorithm A, the running time of A on x is thelength of the shortest computation path on x that outputs YES. The time complexityof algorithm A is defined as the function

tA(n) = the maximum running time on any x of length n that is acceptedby the algorithm A.

For instance, the following is a nondeterministic algorithm for KNAPSACK (moreprecisely, for the decision problem KNAPSACKD):

Algorithm 1.E (Nondeterministic Algorithm for KNAPSACKD)Input: Positive integers S, s1, s2, . . . , sn, c1, c2, . . . , cn, and an integer K > 0.

(1) For i← 1 to n donondeterministically select a value 0 or 1 for xi.

(2) If∑n

i=1 xisi ≤ S and∑n

i=1 xici ≥ K then output YESelse output NO.

It is clear that the above algorithm works correctly. Indeed, it contains 2n dif-ferent computation paths, each corresponding to one choice of (x1, x2, . . . , xn) ∈0, 1n. If one choice of (x1, x2, . . . , xn) satisfies the condition of step (2), then thealgorithm accepts the input instance; otherwise, it rejects. In addition, we note that,in this algorithm, all computation paths have the same running time O(n). Thus,this is a linear-time nondeterministic algorithm.

The nondeterministic Turing machine is the formalism of nondeterministic algo-rithms. Corresponding to the deterministic complexity class P, we have the follow-ing nondeterministic complexity class:

NP: the class of all decision problems that are computable by nondeterministicTuring machine in polynomial time.

We note that, in a single path of a polynomial-time nondeterministic algorithm, therecan be at most a polynomial number of nondeterministic moves. It is not hard to see

1.5 NP-Complete Problems 19

that, we can always move the nondeterministic moves to the beginning of the al-gorithm without changing its behavior. Thus, all polynomial-time nondeterministicalgorithms MN have the following common form:

Assume that the input x has n bits.(1) Nondeterministically select a string y = y1y2 · · · yp(n) ∈ 0, 1∗, where p

is a polynomial function.

(2) Run a polynomial-time deterministic algorithm MD on input (x, y).

Suppose MD answers YES on input (x, y), then we say y is a witness of theinstance x. Thus, a problem Π is in NP if there is a two-step algorithm for Π inwhich the first step nondeterministically selects a potential witness y of polynomialsize, and the second step deterministically verifies that y is indeed a witness. Wecall such an algorithm a guess-and-verify algorithm.

As another example, let us show that the problem SAT is in NP.

Algorithm 1.F (Nondeterministic Algorithm for SAT)Input: A Boolean formula φ over Boolean variables v1, v2, . . . , vn.

(1) Guess n Boolean values b1, b2, . . . , bn.

(2) Verify (deterministically) that the formula φ is TRUE under the assignmentτ(vi) = bi, for i = 1, . . . , n. If so, output YES; otherwise, output NO.

The correctness of the above algorithm is obvious. To show that SAT is in NP,we only need to check that the verification of whether a Boolean formula containingno variables is TRUE can be done in deterministic polynomial time.

We have seen that problems in NP, such as KNAPSACK and SAT, have sim-ple polynomial-time nondeterministic algorithms. However, we do not know of anyphysical devices to implement the nondeterministic moves in the algorithms. So,what is the exact relation between P and NP? This is one of the most importantopen questions in computational complexity theory. On the one hand, we do notknow how to find efficient deterministic algorithms to simulate a nondeterministicalgorithm. A straightforward simulation by the deterministic algorithm which runsthe verification step over all possible guesses would take an exponential amount oftime. On the other hand, though many people believe that there is no polynomial-time deterministic algorithm for every problem in NP, no one has yet found a formalproof for that.

Without a proof for P 6= NP, how do we demonstrate that a problem in NP islikely to be intractable? The notion of NP-completeness comes to help.

For convenience, we write, in the following, x ∈ A to denote that the answerto the input x for the decision problem A is YES (that is, we identify the decisionproblem with the set of all input instances which have the answer YES). We say adecision problem A is polynomial-time reducible to a decision problem B, denotedby A ≤P

m B, if there is a polynomial-time computable function f from instances ofA to instances of B (called the reduction function from A to B) such that x ∈ A ifand only if f(x) ∈ B. Intuitively, a reduction function f reduces the membership

20 Introduction

problem of whether x ∈ A to the membership problem of whether f(x) ∈ B. Thus,if there is a polynomial-time algorithm to solve problem B, we can combine thefunction f with this algorithm to solve problem A.

Proposition 1.5 (a) If A ≤Pm B and B ∈ P then A ∈ P.

(b) If A ≤Pm B and B ≤P

m C then A ≤Pm C.

The above two properties justify the use of the notation ≤Pm between deci-

sion problems: It is a partial ordering for the hardness of the problems (modulopolynomial-time computability).

We can now define the term NP-completeness: We say a decision problem A isNP-hard if, for any B ∈ NP, B ≤P

m A. We say A is NP-complete if A is NP-hardand, in addition, A ∈ NP. That is, an NP-complete problem A is one of the hardestproblems in NP with respect to the reduction ≤P

m. For an optimization problemA, we also say A is NP-hard (or, NP-complete) if its (polynomial-time equivalent)decision version AD is NP-hard (or, respectively, NP-complete).

It follows immediately from Proposition 1.5 that if an NP-complete problem isin P then P = NP. Thus, in view of our inability to solve the P vs. NP question, thenext best way to prove a problem intractable is to show that it is NP-complete (andso it is most likely not in P, unless P = NP).

Among all problems, SAT was the first problem proved NP-complete. It is provedby Cook [1971] who showed that for any polynomial-time nondeterministic Turingmachine, its computation on any input x can be encoded by a Boolean formula φxof polynomially bounded length such that the formula φx is satisfiable if and onlyif M accepts x. This proof is called a generic reduction, since it works directlywith the computation of a nondeterministic Turing machine. In general, it does notrequire a generic reduction to prove a new problem A to be NP-complete. Instead,By Proposition 1.5(b), we can use any problem B that is already known to be NP-complete and only need to prove that B ≤P

m A. For instance, we can prove thatKNAPSACKD is NP-complete by reducing the problem SAT to it.

Theorem 1.6 KNAPSACKD is NP-complete.

Proof. We have already seen that KNAPSACKD is in NP. We now prove thatKNAPSACKD is complete for NP. In order to do this, we introduce a subproblem3-SAT of SAT. In a Boolean formula, a variable or the negation of a variable iscalled a literal. An elementary sum of literals is called a clause. A Boolean formulais in 3-CNF (conjunctive normal form) if it is a product of a finite number of clauses,each being the sum of exactly three literals. For instance, the following is a 3-CNFformula:

(v1 + v2 + v3)(v1 + v3 + v4)(v2 + v3 + v4).

The problem 3-SAT asks whether a given 3-CNF Boolean formula is satisfiable.This problem is a restrictive form of the problem SAT, but it is also known to beNP-complete. Indeed, there is a simple way of transforming a Boolean formula φto a new 3-CNF formula ψ such that φ is satisfiable if and only if ψ is satisfiable.

1.5 NP-Complete Problems 21

We omit the proof and refer the reader to textbooks on complexity theory. In thefollowing, we present a proof for 3-SAT ≤P

m KNAPSACKD.Let φ be a 3-CNF formula which is of the form C1C2 · · ·Cm, where each Cj is a

clause with three literals. Assume that φ contains Boolean variables v1, v2, . . . , vn.We are going to define a list of 2n+2m integers c1, c2, . . . , c2n+2m, plus an integerK. All integers ci and the integer K are of value between 0 and 10n+m. Theseintegers will satisfy the following property:

φ is satisfiable ⇐⇒(∃x1, x2, . . . , x2n+2m ∈ 0, 1

) 2n+2m∑i=1

cixi = K. (1.4)

Now, let S = K, si = ci for i = 1, 2, . . . , 2n+2m. Then, it follows that the formulaφ is satisfiable if and only if the instance (S,K, s1, . . . , s2n+2m, c1, . . . , c2n+2m)to the problem KNAPSACKD has the answer YES. Therefore, this construction is areduction function for 3-SAT ≤P

m KNAPSACKD.We now describe the construction of these integers and prove that they satisfy

property (1.4). First, we note that each integer is between 0 and 10n+m, and so ithas a unique decimal representation of exactly n + m digits (with possible leadingzeroes). We will define each integer digit by digit, with the kth digit indicating thekth most significant digit. First, we define the first n digits of K to be 1 and the lastm digits to be 3. That is,

K = 11 · · · 11︸ ︷︷ ︸n

33 · · · 33︸ ︷︷ ︸m

.

Next, for each i = 1, 2, . . . , n, we define integer ci as follows: The ith digit and the(n + j)th digits, for all 1 ≤ j ≤ m such that Cj contains the literal vi, of ci are 1and all other digits are 0. For instance, if v3 occurs in C1, C5 and Cm, then

c3 = 00100 · · · 0︸ ︷︷ ︸n

100010 · · · 01︸ ︷︷ ︸m

.

Similarly, for i = 1, 2, . . . , n, integer cn+i is defined as follows: The ith digit andthe (n + j)th digits, for all 1 ≤ j ≤ m such that Cj contains the literal vi, of cn+i

are 1 and all other digits are 0.Finally, for j = 1, 2, . . . ,m, we define c2n+2j−1 = c2n+2j as follows: Their

(n + j)th digit is 1 and all other digits are 0. This completes the definition of theintegers.

Now, we need to show that these integers satisfy property (1.4). First, we observethat for any k, 1 ≤ k ≤ n+m, there are at most five integers among ct’s whose kthdigit is nonzero, and each nonzero digit must be 1. Thus, to get the sum K, we mustchoose, for each i = 1, 2, . . . , n, exactly one integer among ct’s whose ith digit is1, and, for each j = 1, 2, . . . ,m, exactly three integers whose (n + j)th digit is 1.The first part of this condition implies that we must choose, for each i = 1, 2, . . . , n,exactly one of ci or cn+i.

Now, assume that φ has a truth assignment τ on variables v1, v2, . . . , vn. Wedefine the sequence (x1, x2, . . . , x2n+2m) as follows:

22 Introduction

(1) For each i = 1, 2, . . . , n, let xn+i = 1− xi = τ(vi).

(2) For each j = 1, 2, . . . ,m, define x2n+2j−1 and x2n+2j as follows: If τ sat-isfies all three literals of Cj , then x2n+2j−1 = x2n+2j = 0; if τ satisfiesexactly two literals of Cj , then x2n+2j−1 = 1 and x2n+2j = 0; and if τsatisfies exactly one literal of Cj , then x2n+2j−1 = x2n+2j = 1.

Then, it is easy to verify that∑2n+2m

i=1 cixi = K.Next, assume that there exists a sequence (x1, x2, . . . , x2n+2m) ∈ 0, 12n+2m

such that∑2n+2m

i=1 cixi = K. Then, from our earlier observation, we see that ex-actly one of xi and xn+i has value 1. Define τ(vi) = xn+i. We claim that τ satisfieseach clause Cj , 1 ≤ j ≤ m. Since the (n + j)th digit of the sum

∑2n+2mi=1 cixi

is equal to 3, and since there are at most two integers among the last 2m integerswhose (n+ j)th digit is 1, there must be an integer k ≤ 2n such that xk = 1 and the(n+j)th digit of ck is 1. Suppose 1 ≤ k ≤ n; then, it means that τ(vk) = 0, and Cj

contains the literal vk. Thus, τ satisfies Cj . On the other hand, if n + 1 ≤ k ≤ 2n,then we know that τ(vk−n) = 1, and Cj contains the literal vk−n; and so τ alsosatisfies Cj . This completes the proof of property (1.4).

Finally, we remark that the above construction of these integers from the formulaφ is apparently polynomial-time computable. Thus, this reduction is a polynomial-time reduction.

In addition to the above two problems, thousands of problems from many seem-ingly unrelated areas have been proved to be NP-complete in the past four decades.These results demonstrate the importance and universality of the concept of NP-completeness. In the following, we list a few problems which are frequently used toprove a new problem being NP-complete.

VERTEX COVER (VC): Given an undirected graph G = (V,E) and apositive integerK, determine whether there is a setC ⊆ V of size≤ Ksuch that, for every edge u, v ∈ E, C ∩ u, v 6= ∅. (Such a set C iscalled a vertex cover of G.)

HAMILTONIAN CIRCUIT (HC): Given an undirected graph G = (V ,E), determine whether there is a simple cycle that passes through eachvertex exactly once. (Such a cycle is called a Hamiltonian Circuit.)

PARTITION: Given n positive integers a1, a2, . . . , an, determine whe-ther there is a partition of these integers into two parts which have theequal sum. (This is a subproblem of KNAPSACK.)

SET COVER (SC): Given a family C of subsets of I = 1, 2, . . . , nand a positive integer K, determine whether there is a subfamily C′ ofC of at most K subsets such that

⋃A∈C′ A = I .

For instance, from the problem HC, we can easily prove that (the decision ver-sions of) the following optimization problems are also NP-complete. We leave theirproofs as exercises.

1.6 Performance Ratios 23

TRAVELING SALESMAN PROBLEM (TSP): Given a complete graphand a distance function which gives a positive integer as the distancebetween every pair of vertices, find a Hamiltonian circuit with the min-imum total distance.

MAXIMUM HAMILTONIAN CIRCUIT (MAX-HC): Given a completegraph and a distance function, find a Hamiltonian circuit with the max-imum total distance.

MAXIMUM DIRECTED HAMILTONIAN PATH (MAX-DHP): Given acomplete directed graph and a distance function, find a Hamiltonianpath with the maximum total distance. (A Hamiltonian path is a simplepath that passes through each vertex exactly once.)

1.6 Performance Ratios

As we pointed out earlier, the two most important criteria in the study of approxi-mation algorithms are efficiency and the performance ratio. By efficiency, we meanpolynomial-time computability. By performance ratio, we mean the ratio of the ob-jective function values between the approximate and optimal solutions. More pre-cisely, for any optimization problem Π and any input instance I , let opt(I) denotethe objective function value of the optimal solution to instance I , and A(I) the ob-jective function value produced by an approximation algorithm A on instance I .Then, for a minimization problem, we define the performance ratio of an approxi-mation algorithm A to be

r(A) = supI

A(I)

opt(I),

and, for a maximization problem, we define it to be

r(A) = supI

opt(I)

A(I),

where I ranges over all possible input instances. Thus, for any approximation algo-rithm A, r(A) ≥ 1, and, in general, the smaller the performance ratio is, the betterthe approximation algorithm is.

For instance, consider the maximization problem KNAPSACK again. Let opt(I)be the maximum value of the objective function on input instance I , and c

G(I) and

cGG

(I) the objective function values obtained by Algorithms 1.B and 1.C, respec-tively, on instance I . Then, by Theorems 1.1 and 1.2, the performance ratios of thesetwo algorithms (denoted by A1B and A1C) are

r(A1B) = supI

opt(I)

cG

(I)≤ 2,

and

r(A1C) = supI

opt(I)

cGG

(I)≤ 1 + ε.

24 Introduction

That is, both of these algorithms achieve a constant approximation ratio, but Algo-rithm 1.C has a better ratio.

As another example, consider the famous TRAVELING SALESMAN PROBLEM(TSP) defined in the last section. We assume that the distance between any twovertices is positive. In addition, we assume that the given distance function d satisfiesthe triangle inequality (abbr. ∆-inequality), that is,

d(a, b) + d(b, c) ≥ d(a, c)

for any three vertices a, b and c. Then, there is a simple approximation algorithm forTSP that finds a tour (i.e., a Hamiltonian circuit) with the total distance within twiceof the optimum. This algorithm uses two basic linear-time algorithms on graphs:

Minimum Spanning Tree Algorithm: Given a connected graph Gwith a distance function d on all edges, this algorithm finds a mini-mum spanning tree T of the graphG. (T is a minimum spanning tree ofG if T is a connected subgraph of G with the minimum total distance.)

Euler Tour Algorithm: Given a connected graph G in which each ver-tex has an even degree, this algorithm finds an Euler tour, i.e., a cyclethat passes through each edge in G exactly once.

Algorithm 1.G (Approximation Algorithm for TSP with ∆-Inequality)

Input: A complete graph G = (V,E), where V = 1, 2, . . . , n, and a distancefunction d : V × V → N which satisfies the triangle inequality.

(1) Find a minimum spanning tree T of G.

(2) Change each edge e in T to two (parallel) edges between the same pair ofvertices. Call the resulting graph H .

(3) Find an Euler tour P of H .

(4) Output the Hamiltonian circuit Q that is obtained by visiting each vertex oncein the order of their first occurrence in P . (That is,Q is the shortcut of P whichskips a vertex if it has already been visited. See Figure 1.4.)

We first note that, after step (2), each vertex in graph H has an even degree andhence the Euler Tour Algorithm can find an Euler tour of H in linear time. Thus,Algorithm 1.G is well defined. Next, we verify that its performance ratio is boundedby two. This is easy to see from the following three observations:

(a) The total distance of the minimum spanning tree T must be less than that ofany Hamiltonian circuit C, since we can obtain a spanning tree by removingan edge from C.

(b) The total distance of P is exactly twice of that of T , and so at most twice ofthat of the optimal solution.

1.6 Performance Ratios 25

Figure 1.4: Algorithm 1.G: (a) the minimum spanning tree, (b) the Euler tour,and (c) the shortcut.

(c) By the triangle inequality, the total distance of the shortcut Q is no greaterthan that of tour P .

Christofides [1976] introduced a new idea into this approximation algorithm andimproved the performance ratio to 3/2. This new idea requires another basic graphalgorithm:

Minimum Perfect Matching Algorithm: Given a complete graph Gof an even number of vertices and a distance function d on edges, thisalgorithm finds a perfect matching with the minimum total distance. (Amatching of a graph is a subset M of the edges such that each vertexoccurs in at most one edge in M . A perfect matching of a graph is amatching M with each vertex occurring in exactly one edge in M .)

Algorithm 1.H (Christofides’s Algorithm for TSP with ∆-Inequality)Input: A complete graph G = (V,E), where V = 1, 2, . . . , n, and a distance

function d : V × V → N which satisfies the triangle inequality.(1) Find a minimum spanning tree T = (V,ET ) of G.

(2) Let V ′ be the set of all vertices in T of odd degrees;Let G′ = (V ′, E′) be the subgraph of G induced by vertex set V ′;Find a minimum perfect matching M for G′;Add the edges in M to tree T (with possible parallel edges between

two vertices) to form a new graph H ′.(See Figure 1.5(b).)

(3) Find an Euler tour P ′ of H ′.

(4) Output the shortcut Q of the tour P ′ as in Step (4) of Algorithm 1.G.

It is clear that after adding the matching M to tree T , each vertex in graph H ′

has an even degree. Thus, step (3) of Algorithm 1.H is well defined. Now, we note

26 Introduction

Figure 1.5: Christofides’s approximation: (a) The minimum spanning tree, (b) theminimum matching (shown in broken lines) and the Euler tour, and (c) the shortcut.

that the total distance of the matching M is at most one half of that of a minimumHamiltonian circuit C ′ in G′, since we can remove alternating edges from C ′ toobtain a perfect matching. Also, by the triangle inequality, the total distance of theminimum Hamiltonian circuit in G′ is no greater than that of the minimum Hamil-tonian circuit in G. Therefore, the total distance of the tour P ′, as well as that of Q,is at most 3/2 of the optimal solution. That is, the performance ratio of Algorithm1.H is bounded by 3/2.

Actually, the performance ratio of Christofides’s approximation can be shown tobe exactly 3/2. Consider the graph G of Figure 1.6. Graph G has 2n + 1 verticesv0, v1, . . . , v2n on the Euclidean space R2, with the distance d(vi, vi+1) = 1 fori = 0, 1, . . . , 2n − 1, and d(vi, vi+2) = 1 + a for i = 0, 1, . . . , 2n − 2, where0 < a < 1/2. It is clear that the minimum spanning tree T of G is the path fromv0 to v2n containing all edges of distance 1. There are only two vertices, v0 andv2n, having odd degrees in tree T . Thus, the traveling salesman tour produced byChristofides’s algorithm is the cycle (v0, v1, v2, . . . , v2n, v0), whose total distanceis 2n+n(1 + a) = 3n+na. Moreover, it is easy to see that the minimum travelingsalesman tour consists of all horizontal edges plus the two outside non-horizontaledges, whose total distance is (2n− 1)(1 + a) + 2 = 2n+ 1 + (2n− 1)a. So, if welet A1H denote Christofides’s algorithm, we get, in this instance I ,

A1H(I)

opt(I)=

3n+ na

2n+ 1 + (2n− 1)a,

which approaches 3/2 as a goes to 0 and n goes to infinity. It follows that r(A1H) =3/2.

Theorem 1.7 For the subproblem of TSP with the triangle inequality, as well asthe subproblem of TSP on Euclidean space, the Christofides’s approximation A1H

has the performance ratio r(A1H) = 3/2.

For simplicity, we say an approximation algorithm A is an α-approximation ifr(A) ≤ α for some constant α ≥ 1. Thus, we say Christofides’s algorithm is a

1.6 Performance Ratios 27

Figure 1.6: A worst case of Christofides’s approximation

(3/2)-approximation for TSP with the triangle inequality, but not an α-approxima-tion for any α < 3/2.

An approximation algorithm with a constant performance ratio is also called abounded approximation or a linear approximation. An optimization problem Π issaid to have a polynomial-time approximation scheme (PTAS) if for any k > 0,there exists a polynomial-time approximation algorithmAk for Π with performanceratio r(Ak) ≤ 1 + 1/k. Furthermore, if the running time of the algorithm Ak in theapproximation scheme is a polynomial function in n+1/k, where n is the input size,then the scheme is called a fully polynomial-time approximation scheme (FPTAS).For instance, the generalized greedy algorithm (Algorithm 1.C) is a PTAS, and thepolynomial tradeoff approximation (Algorithm 1.D) is a FPTAS for KNAPSACK.

In this book, our main concern is to find efficient approximations to intractableproblems with the best performance ratios. However, some optimization problemsare so hard that they don’t even have any polynomial-time bounded approximations.In these cases, we also need to prove that such approximations do not exist. Sincemost optimization problems are NP-complete, and hence have polynomial-time op-timal algorithms if P = NP. So, when we try to prove that a bounded approximationdoes not exist, we must assume that P 6= NP. Very often, we simply prove that theproblem of finding a bounded approximation (or, an α-approximation for some fixedconstant α) itself is NP-hard. The following is a simple example. We will present amore systematic study of this type of inapproximability results in Chapter 10.

Theorem 1.8 If P 6= NP, then there is no polynomial-time approximation algorithmfor TSP (without the restriction of the triangle inequality) with a constant perfor-mance ratio.

Proof. For any fixed integer K > 1, we will construct a reduction from the problemHC to the problem of finding aK-approximation for TSP.5 That is, we will constructa mapping from each instance G of the problem HC to an instance (H, d) of TSP,such that the question of whether G has a Hamiltonian circuit can be determinedfrom any traveling salesman tour for (H, d) whose total distance is within K timesof the length of the shortest tour.

For any graph G = (V,E), with |V | = n, let H be the complete graph oververtex set V . Define the distance between two vertices u, v ∈ V as follows:

5Note that TSP is not a decision problem. So, the reduction here has a more general form than thatdefined in Section 1.5.

28 Introduction

d(u, v) =

1, if u, v ∈ E,n(K + 1), otherwise.

Now, assume that C is a traveling salesman tour of the instance (H, d) whosetotal distance is at mostK times of the length of the shortest tour. If the total distanceof C is less than n(K + 1), then we know that all edges in C are of distance oneand so they are all in E. Thus, C is a Hamiltonian circuit of G. On the other hand,if the total distance of C is greater than or equal to n(K + 1), this implies that theminimum traveling salesman tour has total distance at least n(K+ 1)/K and hencegreater than n. It implies that the minimum traveling salesman tour must contain anedge not in E. Thus, G has no Hamiltonian circuit.

Thus, if there is a polynomial-time K-approximation for TSP, we can then use itto solve the problem HC, which is NP-complete. It follows that P = NP.

Exercises

1.1 Prove that Algorithm 1.A always finds the optimal solution for KNAPSACK.More precisely, prove by induction that if there is a subset A ⊆ 1, . . . , i such that∑

k∈A ck = j and∑

k∈A sk ≤ S, then the value a(i, j) obtained at the end of Step(3) of Algorithm 1.A satisfies a(i, j) 6= nil and a(i, j) has the minimum total cost∑

k∈a(i,j) sk among such sets A.

1.2 Formulate the following logic puzzles into satisfiability instances and solvethem:

(a) Three men named Lewis, Miller, and Nelson fill the positions of accountant,cashier, and clerk in a department store. If Nelson is the cashier, Miller isthe clerk. If Nelson is the clerk, Miller is the accountant. If Miller is not thecashier, Lewis is the clerk. If Lewis is the accountant, Nelson is the clerk.What is each man’s job?

(b) Messrs. Spinnaker, Buoy, Luff, Gybe, and Windward are yacht owners.Each has a daughter, and each has named his yacht after the daughter ofone of the others. Mr. Spinnaker’s yacht, the Iris, is named after Mr. Buoy’sdaughter. Mr. Buoy’s own yacht is the Daffodil; Mr. Windward’s yacht is theJonquil; Mr. Gybe’s, the Anthea. Daffodil is the daughter of the owner of theyacht which is named after Mr. Luff’s daughter. Mr. Windward’s daughteris named Lalage. Who is Jonquil’s father?

1.3 For any Boolean function f , Ff (y) is defined as in Section 1.3. Prove thatfor y ∈ 0, 1n, 0 ≤ Ff (y) ≤ 1.

1.4 For a 3-CNF formula φ = C1C2 · · ·Cm over Boolean variables x1, x2, · · ·,xn, let x be the vector (x1, x2, . . . , xn) in 0, 1n. For each variable xj , 1 ≤ j ≤ n,define a corresponding real variable yj , and let y be the vector (y1, y2, · · · , yn) in

Exercises 29

Rn. Define a function f1 : Rn → R as follows: First, for each pair (i, j), with1 ≤ i ≤ m and 1 ≤ j ≤ n, define a literal function

qij(yj) =

(yj − 1)2, if xj is in clause Ci,

(yj + 1)2, if xj is in clause Ci,

1, neither xj nor xj is in Ci,

and, for each 1 ≤ i ≤ m, define a clause function ci(y) =∏n

j=1 qij(yj). Finally,define f1 to be the sum of the clause functions: f1(y) =

∑mi=1 ci(y).

Define a correspondence between x and y as follows:

xj =

1, if yj = 1,

0, if yj = −1,

undefined, otherwise.

Then, it is clear that φ is satisfiable if and only if the minimum value of f1(y)is 0. Now, define f(y) = f1(y) +

∑nj=1(y2

j − 1)2, and consider the followingminimization problem:

minimize f(y).

Show that the objective function f(y) satisfies the following properties:

(a) There exists y such that f(y) = 0 if and only if there exists y such thatf(y) < 1.

(b) At every minimum point y∗, f(y∗) is strictly convex.

1.5 Consider the greedy algorithm for KNAPSACK that selects the most valuableitem first. That is, in Algorithm 1.B, replace the ordering c1/s1 ≥ c2/s2 ≥ · · · ≥cn/sn by c1 ≥ c2 ≥ · · · ≥ cn. Show that this greedy algorithm is not a linearapproximation.

1.6 Give an example to show that the performance ratio of Algorithm 1.G forTSP with the triangle inequality cannot be any constant smaller than two.

1.7 When the distance function in TSP is allowed to be asymmetric, i.e., possiblyd(u, v) 6= d(v, u), the problem is called DIRECTED TSP. Give an example to showthat Christofides’s approximation (Algorithm 1.H) does not work for DIRECTEDTSP with triangle inequality.

1.8 (a) Suppose there exists an algorithm that can compute the maximum valueopt of the objective function for KNAPSACK. Can you use this algorithmas a subroutine to design an algorithm computing an optimal solution forKNAPSACK (i.e., the 0-1 vector (x∗1, x

∗2, . . . , x

∗n) such that

∑ni=1 cix

∗i =

opt) in polynomial time, provided that the time spent by the subroutine isnot counted?

30 Introduction

(b) Suppose there exists an algorithm that can compute the distance of the short-est tour for TSP. Can you use this algorithm as a subroutine to design analgorithm computing an optimal solution for TSP (i.e., the shortest tour)in polynomial time, provided that the time spent by the subroutine is notcounted?

(c) Suppose there exists an algorithm that can compute a value within a factorα from the distance of the shortest tour for TSP, where α is a constant. Canyou use this algorithm as a subroutine to design an algorithm computing anoptimal solution for TSP in polynomial-time, provided that the time spentby the subroutine is not counted?

1.9 Show that for any ε > 0, there exists a polynomial-time (2 + ε)-approximation for MAX-HC and there exists a polynomial-time 2-approximationfor MAX-DHP. [Hint: Use the polynomial-time Maximum Matching Algorithm.]

1.10 Consider the following problem:

MINIMUM VERTEX COVER (MIN-VC): Given an undirected graphG,find a vertex cover of the minimum size.

(a) Design a polynomial-time 2-approximation for the problem [Hint: Use thepolynomial-time Maximum Matching Algorithm.]

(b) Show that MIN-VC in bipartite graphs can be solved in polynomial time.

1.11 A subset S of vertices in a graph G = (V,E) is independent if no edgesexist between any two vertices in S.

(a) Show that I is a maximum independent set of graphG = (V,E) if and onlyif V − I is a minimum vertex cover of G.

(b) Give an example to show that if C is a vertex cover within a factor of 2 fromthe minimum, then V −C is still an independent set, but may not be withina factor of 2 from the maximum.

1.12 Find a polynomial-time 2-approximation for the following problem:

STEINER MINIMUM TREE (SMT): Given a graph G = (V,E) with adistance function on E, and a subset S ⊆ V , compute a shortest treeinterconnecting the vertices in S.

1.13 There are n jobs J1, J2, . . . , Jn and m identical machines. Each job Ji,1 ≤ i ≤ n, needs to be processed in a machine without interruption for a timeperiod pi. Consider the problem of finding a scheduling to finish all jobs with them machines in the minimum time. Graham [1966] proposed a simple algorithmfor this problem: Put n jobs in an arbitrary order; whenever a machine becomesavailable, assign it the next job. Show that Graham’s algorithm is a polynomial-time2-approximation.

Exercises 31

1.14 There are n students in a late-night study group. The time has come toorder pizzas. Each student has his own list of preferred toppings (e.g. mushroom,pepperoni, onions, garlic, sausage, etc.), and each pizza may have only one topping.Answer the following questions:

(a) If each student wants to eat at least one half of a pizza with the topping in hispreferred list, what is the complexity of computing the minimum number ofpizzas to order to make everyone happy?

(b) If everyone wants to eat at least one third of a pizza with the topping in hispreferred list, what is the complexity of computing the minimum number ofpizzas to order to make everyone happy?

1.15 Assume that C is a collection of subsets of a set X . We say a set Y ⊆ Xhits a set C ∈ C if Y ∩C 6= ∅. A set Y ⊆ X is a hitting set for C if Y hits every setC ∈ C. Show that the following problems are NP-hard:

(a) MINIMUM HITTING SET (MIN-HS): Given a collection C of subsets of aset X , find a minimum hitting set Y for C.

(b) Given a collection C of subsets of a set X , find a subset Y of X of theminimum size such that all sets Y ∩ C for C ∈ C are distinct.

(c) Given two collections C andD of subsets of X and a positive integer d, finda subset A ⊆ X of size |A| ≤ d that minimizes the total number of subsetsin C not hit by A and subsets in D hit by A.

1.16 Show that the following problems are NP-hard:

(a) Given a graph G = (V,E) and a positive integer m, find the minimumsubset A ⊆ V such that A covers at least m edges and the complement ofA has no isolated vertices.

(b) Given a 2-connected graphG = (V,E) and a setA ⊆ V , find the minimumsubset B ⊆ V such that A ∪B induces a 2-connected subgraph.

1.17 Show that the following problem is NP-complete:

Given two disjoint setsX and Y , and a collection C of subsets ofX∪Y ,determine whether C can be partitioned into two disjoint subcollectionscovering X and Y respectively.

1.18 Let k > 0. A collection C of subsets of a set X is a k-set cover if C can bepartitioned into k disjoint subcollections each being a set cover for X .

(a) Consider the following problem:

k-SET COVER (k-SC): Given a collection C of subsets of a setX , determine whether it is a k-set cover.

Show that the problem 2-SC is NP-complete.

32 Introduction

(b) Show that the following problem is not polynomial-time 2-approximableunless P = NP:

Given a collection C of subsets of a setX , compute the minimumk such that C is a k-set cover.

1.19 For each 3-CNF formula F , we define a graphG(F ) as follows: The vertexset of G(F ) consists of all clauses and all literals in F . An edge exists in G(F )between a clause C and a literal x if and only if x belongs to C, and an edge existsbetween two literals x and y if and only if x = y. A 3-CNF formula is called aplanar formula if G(F ) is a planar graph. Show that the following problems areNP-complete:

(a) NOT-ALL-EQUAL 3-SAT: Given a 3-CNF F , determine whether F has anassignment which assigns, for each clause C, value 1 to a literal in C andvalue 0 to another literal in C.

(b) ONE-IN-THREE 3-SAT: Given a 3-CNF F , determine whether F has anassignment which, for each clause C, assigns value 1 to exactly one literalin C.

(c) PLANAR 3-SAT: Given a planar 3-CNF formula F , determine whether F issatisfiable.

1.20 A subset D of vertices in a graph G = (V,E) is called a dominating set ifevery vertex v ∈ V is either in D or is adjacent to a vertex in D.

(a) Show that the problem of computing the minimum dominating set for agiven graph is NP-hard.

(b) Show that the problem of determining whether there exist two disjoint dom-inating sets for a given graph is polynomial-time solvable.

(c) Show that the problem of determining whether there exist three disjointdominating sets for a given graph is NP-complete. [Hint: Use NOT-ALL-EQUAL 3SAT.]

(d) Show that the problem of computing the maximum number of disjoint dom-inating sets for a given graph is not (3/2)-approximable in polynomial time,unless P = NP.

1.21 A graph is said to be k-colorable if its vertices can be partitioned into kdisjoint independent sets.

(a) Show that the problem of deciding whether a given graph is 2-colorable ornot is polynomial-time solvable.

(b) Show that the problem of deciding whether a given graph is 3-colorable ornot is NP-complete.

(c) Show that the problem of computing, for a given graph G, the minimum ksuch that G is k-colorable is not (3/2)-approximable, unless P = NP.

Historical Notes 33

1.22 A subset C of vertices of a graph G = (V,E) is a clique if the subgraphof G induced by C is a complete graph. Study the computational complexity of thefollowing problems:

(a) For a given graph, compute the maximum number of disjoint vertex covers.

(b) For a given graph, compute the minimum number of disjoint cliques suchthat their union contains all vertices.

Historical Notes

Graham [1966] initiated the study of approximations using the performance ratio toevaluate the approximation algorithms. However, the importance of this work wasnot fully understood until Cook [1971] and Karp [1972] established the notion ofNP-completeness and its ubiquitous existence in combinatorial optimization. Withthe theory of NP-completeness as its foundation, the study of approximation algo-rithms took off quickly in 1970s. Garey and Johnson [1979] gave an account of thedevelopment in this early period.

The PTAS for KNAPSACK belongs to Sahni [1975]. The first FPTAS for KNAP-SACK was discovered by Ibarra and Kim [1975]. Later, many different FPTASs, in-cluding Algorithm 1.D of Section 1.1, have been found for KNAPSACK. Christofides[1976] found a polynomial-time (3/2)-approximation for TSP with the triangle in-equality. So far, nobody has found a better one in terms of the performance ratio.