Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite...
-
date post
19-Dec-2015 -
Category
Documents
-
view
228 -
download
1
Transcript of Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite...
![Page 1: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/1.jpg)
Semidefinite Programming
Lecture 22: Apr 2
![Page 2: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/2.jpg)
Semidefinite Programming
What is semidefinite programming?• A relaxation of quadratic programming.• A special case of convex programing.• A generalization of linear programming.• Can be optimized in polynomial time.
What is it good for?
• Shannon capacity
• Perfect graphs
•Approximation algorithms
• Image segmentation and
clustering
• Constraint satisfaction problems
• Number theory, quantum computation,
etc…
![Page 3: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/3.jpg)
Maximum Cut
(Maximum Cut) Given an undirected graph, with an edge weight w(e) on each edge e, find a partition (S,V-S) of V so as to maximum the total weight of edges in this cut, i.e. edges that have one endpoint in S and one endpoint in V-S.
![Page 4: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/4.jpg)
Maximum Cut
When is computing maximum cut easy?
When we are given a bipartite graph.
The maximum cut problem can also be interpreted asthe problem of finding a maximum bipartite subgraph.
There is a simple greedy algorithm with approximation ratio ½.
Similar to vertex cover.
![Page 5: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/5.jpg)
Quadratic Program for MaxCut
The two sides of the partition.
if they are on opposite sides.
if they are on the same side.
![Page 6: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/6.jpg)
Quadratic Program for MaxCut
This is unlikely to be solved in polynomial time, otherwise P=NP.
This quadratic program is called strict quadratic program,
because every term is of degree 0 or degree 2.
![Page 7: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/7.jpg)
Vector Program for MaxCut
![Page 8: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/8.jpg)
Vector Program for MaxCut
This is a relaxation of the strict quadratic program (why?)
Vector program: linear inequalities over inner products.
Vector program = semidefinite program.
Can be “solved” in polynomial time (ellipsoid, interior point).
![Page 9: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/9.jpg)
Geometric Interpretation
Think of as an n-dimensional vector.
Contribute more to the objectiveif the angle is bigger.
![Page 10: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/10.jpg)
Demonstration
Rubber band method.
László Lovász
![Page 11: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/11.jpg)
Algorithm
(Max-Cut Algorithm)
1. Solve the vector program. Let be an optimal solution.
2. Pick r to be a uniformly distributed vector on the unit sphere .
3. Let
![Page 12: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/12.jpg)
Analysis
Claim:
![Page 13: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/13.jpg)
Analysis
Suppose and has an edge.
Contribution to semidefinite program:
Contribution to the solution:
Approximation Ratio:
![Page 14: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/14.jpg)
Let W be the random variable denoting the weight of edges in the cut.
Analysis
Proof: Linearity of expection.
Claim:
![Page 15: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/15.jpg)
(Max-Cut Algorithm)
1. Solve the vector program. Let be an optimal solution.
2. Pick r to be a uniformly distributed vector on the unit sphere .
3. Let
Algorithm
Repeat a few times to get a good approximation with high probability.
This algorithm performs extremely well in practice.
Try to find a tight example.
![Page 16: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/16.jpg)
Remarks
Hard to imagine a combinatorial algorithm with the same performance.
Assuming the “unique games conjecture”,
this algorithm is the best possible!
That is, it is NP-hard to find a better approximation algorithm!
![Page 17: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/17.jpg)
Constraint Satisfaction Problems
(Max-2-SAT)
Given a formula in which each clause contains two literals,
find a truth assignment that satisfies the maximum number of clauses.
e.g.
An easy algorithm with approximation ratio ½.
An LP-based algorithm with approximation ratio ¾.
An SDP-based algorithm with approximation ratio 0.87856.
![Page 18: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/18.jpg)
Vector Program for MAX-2-SAT
(Max-2-SAT)
Given a formula in which each clause contains two literals,
find a truth assignment that satisfies the maximum number of clauses.
Additional variable (trick):
A variable is set to be true if:
A variable is set to be false if:
![Page 19: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/19.jpg)
Vector Program for MAX-2-SAT
Denote v(C) to be the value of a clause C, which is defined as follows.
Consider a clause containing 2 literals, e.g. . Its value is:
![Page 20: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/20.jpg)
Vector Program for MAX-2-SAT
Objective:
where a(ij) and b(ij) is the sum of coefficients.
![Page 21: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/21.jpg)
(MAX-2-SAT Algorithm)
1. Solve the vector program. Let be an optimal solution.
2. Pick r to be a uniformly distributed vector on the unit sphere .
3. Let be the “true” variables.
Algorithm
![Page 22: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/22.jpg)
Analysis
Term-by-term analysis.
Contribution to semidefinite program:
Contribution to the solution:
Approximation Ratio:
First consider the second term.
![Page 23: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/23.jpg)
Analysis
Term-by-term analysis.
Contribution to semidefinite program:
Contribution to the solution:
Approximation Ratio:
Consider the first term.
![Page 24: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/24.jpg)
(MAX-2-SAT Algorithm)
1. Solve the vector program. Let be an optimal solution.
2. Pick r to be a uniformly distributed vector on the unit sphere .
3. Let be the “true” variables.
Algorithm
This is a 0.87856-approximation algorithm for MAX-2-SAT.
Can be improved to 0.931!
![Page 25: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/25.jpg)
More SDP in approximation algorithms
• Sparsest cut: O(√log n)• Constraint satisfaction problems• Correlation clustering• Graph colouring
A very powerful tool in the design of approximation
algorithms.
Useful to know geometry and algebra.
Next two lectures: SDP and perfect graphs.
![Page 26: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/26.jpg)
Summary: Topics
1. Classical problem: TSP, Steiner trees
2. Covering problem: vertex cover, set cover
3. Packing problem: knapsack, bin packing
4. Graph partitioning problem: multiway cut, multi-cut
5. Job scheduling: makespan, general assignment
6. Network design: Steiner network, degree constrained spanning trees
7. Constraint satisfaction: maximum cut, max 2-SAT
![Page 27: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/27.jpg)
Summary: Techniques
1. Combinatorial arguments: TSP, Steiner trees, multiway cut
2. Greedy algorithm and Randomized rounding: set cover
3. Dynamic programming: knapsack, bin packing
4. Region Growing: multi-cut
5. Iterative relaxation: scheduling, network design problems
6. Primal-dual method: vertex cover, multi-cut in tree
7. Semidefinite programming: maximum cut, constraint satisfaction
![Page 28: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/28.jpg)
Concluding Remarks
Learn to design better heuristics.
e.g. iterative rounding, SDP performs extremely well in practice.
Use LP or SDP as a good way to estimate the optimal value.
e.g. help to test the performance of a heuristic.
Relax the search space to a convex set.
![Page 29: Semidefinite Programming Lecture 22: Apr 2. Semidefinite Programming What is semidefinite programming? A relaxation of quadratic programming. A special.](https://reader036.fdocuments.net/reader036/viewer/2022062714/56649d3e5503460f94a169cb/html5/thumbnails/29.jpg)
Remaining Schedule
Apr 10-11: NO CLASSES
Apr 17-18: Last week of lecture (perfect graphs, SDP)
Apr 23-24: Project presentation (15 minutes)
May 1: Project report deadline
May 8: Homework 3 deadline