Nonlinear Discrete Optimization - Cornell...

Shmuel Onn

Technion – Israel Institute of Technologyhttp://ie.technion.ac.il/~onn

Nonlinear Discrete Optimization

Based on several papers joint with several co-authors including Berstein, De Loera, Hemmecke, Lee, Rothblum, Weismantel, Wynn

(Update on Lecture Series given at CRM Montréal)

Billerafest 2008 - conference in honor of Lou Billera's 65th birthday

Shmuel Onn

Framework for Nonlinear Discrete Optimization

The set of feasible points is a subset S of Zn suitably presented, e.g.

Shmuel Onn



Shmuel Onn

- by a system of inequalities



Shmuel Onn

- by a system of inequalities(often linear inequalities, giving “integer programming”)



Shmuel Onn

- by a suitable oracle (membership, linear optimization, etc.)

- by a system of inequalities(often linear inequalities, giving “integer programming”)



Shmuel Onn


(often S is 0,1-valued, giving “combinatorial optimization”)

(often linear inequalities, giving “integer programming”)




The objective function is parameterized as f(w1x, . . ., wdx), where w1x, . . ., wdx are linear forms and f is a real valued function on Rd.

Shmuel Onn







min or max f(w1x, . . ., wdx) : x in S .


The problem can be interpreted as multi-objective discrete optimization, where the goal is to optimize the “balancing” by f of d linear criteria:

Shmuel Onn







min or max f(w1x, . . ., wdx) : x in S .


The problem can be interpreted as multi-objective discrete optimization, where the goal is to optimize the “balancing” by f of d linear criteria:

Shmuel Onn






It is generally intractable even for fixed d=1 and f the identity on R(e.g., it may be NP-hard or even require exponential oracle-time).

Some Applications and Examples

Where Our Theory Provides

Polynomial Time Algorithms

Shmuel Onn

- Vector Partitioning and Clustering:

- Shaped partition problems (SIAM Opt.)

- Partition problems with convex objectives (Math. OR)

Partition m items evaluated by k criteria to p players, to maximize socialutility that is function of the sums of vectors of items each player gets.

The nonlinear function on k x p matrices is f(X) = ∑ Xij3

Example: Consider m=6 items, k=2 criteria, p=3 players

The criteria-item matrix is: items

criteria

The social utility of π is f(Aπ) = 244432

The matrix of a partition such as π = (34, 56, 12) is:players

criteria

Each player should get 2 items

Vector Partitioning

Shmuel Onn

All 90 partitions πof items 1, …,6 To 3 players where each player gets 2 items

π = (34, 56, 12)

players

criteria

f(Aπ) = 244432

The optimal partition is:

with optimal utility:

Shmuel Onn

Minimum Variance Clustering

Given m points v1, …, vm in Rk, group them into p (balanced) clusters so as to minimize the sum of cluster variances .

P=3, k=3, m large

Shmuel Onn

Shmuel Onn

- Matroids and Their Applications:

Shmuel Onn

- Spanning trees, polymatroids, intersections of matroids:


Shmuel Onn

(3 -2)

(-1 2)

(1 0)(2 -1)(-2 3)

(0 1)

Example - spanning trees:

Consider n=6, the graph G=K4 , d=2,

weights w, and the Euclidean norm

(squared) f(x) = |x|2 = x12 + x2

2



Shmuel Onn

(3 -2)

(-1 2)

(1 0)(2 -1)(-2 3)

(0 1)

Example - spanning trees:

Consider n=6, the graph G=K4 , d=2,

weights w, and the Euclidean norm

(squared) f(x) = |x|2 = x12 + x2

2

The optimal tree x has

w(x) = (0 1) + (-1 2) + (-2 3) = (-3 6)

and objective value f(w(x)) = |-3 6|2 = 45



Shmuel Onn


- Systems of polynomial equations:simultaneous computation of universal Gröbner basesfor all ideals on the Hilbert Scheme


Gröbner Polyhedra

Shmuel Onn


- Experimental design and learning:finding optimal multivariate polynomial modelthat fits experiment-results or learning-queries



Shmuel Onn

- Convex matroid optimization (SIAM Disc. Math.)- Convex combinatorial optimization (Disc. Comp. Geom.)- Cutting corners (Adv. App. Math.)- The Hilbert zonotope and universal Gröbner bases (Adv. App. Math.)- Nonlinear matroid optimization and experimental design (SIAM Disc. Math.)- Nonlinear optimization over a weighted independence system (submitted)


- Experimental design and learning:finding optimal multivariate polynomial modelthat fits experiment-results or learning-queries



Shmuel Onn

- Multiway Tables and Their Applications:

Shmuel Onn

- Privacy and confidentiality in statistical data bases


Shmuel Onn


- Congestion avoiding (multiway) transportation


Shmuel Onn


- Error correcting codes



Shmuel Onn




- Scheme for (nonlinear) optimization over any integer program


Shmuel Onn

- The complexity of 3-way tables (SIAM Comp.)- Markov bases of 3-way tables (J. Symb. Comp.)- All linear and integer programs are slim 3-way programs (SIAM Opt.)- N-fold integer programming (Disc. Opt. in memory of Dantzig) - Graver complexity of integer programming (Annals Combin.)- Nonlinear bipartite matching (Disc. Opt.)- Convex integer maximization (J. Pure App. Algebra)- Convex integer minimization (submitted)




- Scheme for (nonlinear) optimization over any integer program


Some Geometric Methods:

Convex Discrete Maximization

Consider the convex hull P = conv S of the feasible set S in Zn.When we can control the edge-directions of P, we can reduce

convex to linear maximization in strongly polynomial time.

Shmuel Onn




Shmuel Onn


- Cubes: unit vectors 1i (e.g. 0-1 quadratic programming)



Shmuel Onn



- Matroid polytopes: pairs 1i - 1j (e.g. spanning trees



Shmuel Onn



- Matroid polytopes: pairs 1i - 1j (e.g. spanning trees

- Transportations: nxp circuit matrices (e.g. partitioning, clustering)

Theorem:Theorem: Fix any d. Then for any S in Zn endowed with a set E that

covers all edge-directions of conv S, and any convex f presented

by a comparison oracle, the convex discrete maximization problem

max f(w1x, . . ., wdx) : x in S

reduces to strongly polynomially linear counterparts over S.

Shmuel Onn


Lemma: If E = e1, …, em covers all edge-directions of a polytope P

then the zonotope Z = [-1, 1] e1 + … + [-1, 1] em is a refinement of P.

Proof: preliminaries on zonotopes

(Minkowsky, Grunbaum , …, )

Shmuel Onn

EE

ee11ee22

ee33

P


Shmuel Onn



EE

ee11ee22

ee33

P Z


Shmuel Onn



EE

ee11ee22

ee33

Paa55

aa44

aa66

aa22

aa33

aa11

Z


Shmuel Onn



EE

ee11ee22

ee33

aa55

aa44

aa66

aa22

aa33

aa11

Z

aa11

aa55

aa44aa33

P aa22

aa66


Shmuel Onn



Lemma: In Rd, the zonotope Z can be constructed from E = e1, …, em along with a vector ai in the cone of every vertex in O(md-1) operations.

EE

ee11ee22

ee33

aa55

aa44

aa66

aa22

aa33

aa11

Z

aa11

aa55

aa44aa33

P aa22

aa66


(Edelsbrunner, Gritzmann, Orourk, Seidel, Sharir, Sturmfels, …)Shmuel Onn



Proof: the algorithm Input: S in Zn given by linear optimization oracle, set E of edge-directionsof P=conv S, d x n matrix w, and convex f on Rd given by comparison oracle

Shmuel Onn

E RnP


Shmuel Onn

wEE

E Rn

Rd

ww

aa44

aa33

aa55

aa11

aa66Z

aa22

wP

1. Construct the zonotope Z generated by theprojection wE, and find ai in each normal cone

projection

E RnP


Shmuel Onn

Rn

Rd

w

aa44

aa33

aa55

aa11

aa66Z

aa22

P

bbii=wTaaii


2. Lift each ai in Rd to bi = wT ai in Rn and solve linear optimization with objective bi over S

aaiiwP


Shmuel Onn

Rn

Rd

w

aa44

aa33

aa55

aa11

aa66Z

aa22

P

bbii=wTaaii

vi

wvi

wP aaii


3. Obtain the vertex vi of Pand the vertex wvi of wP



Shmuel Onn

Rn

Rd

w

aa44

aa33

aa55

aa11

aa66Z

aa22

P

bbii=wTaaii

vi

3. Obtain the vertex vi of Pand the vertex wvi of wP


4. Output any viattaining maximumvalue f(w vi) usingcomparison oracle

wvi

wP aaii



Shmuel Onn

Shmuel Onn

Theorem:Theorem: For any fixed d, there is a strongly polynomial time algorithm that, given any S in 0,1n presented by membership oracle and endowed with set E covering all edge-directions of conv S, and convex f, solves


Strongly PolynomialConvex Combinatorial Maximization

Shmuel Onn




- Convex combinatorial optimization (Disc. Comp. Geom.)

Shmuel Onn




- Nonlinear bipartite matching (Disc. Opt.)

Natural case with exponentially many edge-directions –permutation matrices and Birkhoff polytope - is treated in

Consider maximizing wx over S, with E all edge-directions of conv S:

Proof: membership augmentation linear optimization

Shmuel Onn


Lemma: Membership AugmentationProof: x in SS can be improved if and only if there is an edge direction e in E such that w e > 0 and x + e = y for some y in SS.


Shmuel Onn



Lemma: Augmentation linear OptimizationProof: Schulz-Weismantel-Ziegler and GrÖtschel–Lovászusing scaling ideas going back to Edmonds-Karp.


Shmuel Onn



Lemma: Augmentation linear OptimizationProof: Schulz-Weismantel-Ziegler and GrÖtschel–Lovászusing scaling ideas going back to Edmonds-Karp.

Lemma: Polynomial time Strongly polynomial timeProof: Frank-Tárdos show that using Diophantine approximationcan replace w by w’ of bit size depending polynomially only on n.


Shmuel Onn

Some Algebraic Methods:

Nonlinear Integer Programming

min or max f(w1x, . . ., wdx) : x ≥ 0, Bx = b, x integer

Nonlinear Integer Programming

with w1x, . . ., wdx linear forms on Rn and f real valued function on Rd.

Shmuel Onn

We now concentrate on the nonlinear integer programming problem:

N-Fold Systems

Shmuel Onn

Let A be (r+s) x t matrix with submatrices A1, A2 of first r and last s rows.

Shmuel Onn

N-Fold Systems

Define the n-fold product of A to be the following (r+ns) x nt matrix,

A(n) =

n

Shmuel Onn


N-Fold Systems


A(n) =

n

Shmuel Onn


N-Fold Systems

We have the following four theorems on n-fold integer programming:


A(n) =

n

Theorem 1:Theorem 1: For any fixed matrix A, linear integer programmingover n-fold products of A can be done in polynomial time:

max wx: A(n)x = a, x ≥ 0, x integer

Shmuel Onn


N-Fold Systems



A(n) =

n

Shmuel Onn


N-Fold Systems


max f(w1x, . . ., wdx) : A(n)x = a, x ≥ 0, x integer

Theorem 2:Theorem 2: For any fixed d and matrix A, convex integer maximizationover n-fold products of A can be done in polynomial time:


A(n) =

n

Shmuel Onn


N-Fold Systems


min f1(x1) + . . . + fnt(xnt) : A(n)x = a, x ≥ 0, x integer

Theorem 3:Theorem 3: For fixed matrix A, separable convex integer minimizationover n-fold products of A can be done in polynomial time:


A(n) =

n

Shmuel Onn


N-Fold Systems


Theorem 4:Theorem 4: For fixed matrix A, integer point lp-nearest to a given xover n-fold products of A can be determined in polynomial time:

min |x - x|p : A(n)x = a, x ≥ 0, x integer


A(n) =

n

Shmuel Onn


N-Fold Systems

- N-fold integer programming (Disc. Opt. in memory of Dantzig) - Convex integer maximization via Graver bases (J. Pure App. Algebra)- Convex integer minimization (submitted)

Shmuel Onn

Universality of N-Fold Systems

Define the n-product of A as the following variant of the n-fold operator:

Shmuel Onn

A[n] =

n



Shmuel Onn

A[n] =

n


Consider m-products of the 1 x 3 matrix (1 1 1). For instance,


Shmuel Onn

A[n] =

n


(1 1 1)[3] =

Consider m-products of the 1 x 3 matrix (1 1 1). For instance,


Shmuel Onn

A[n] =


x integer : x ≥ 0, (1 1 1)[m][n]x = a

Universality Theorem: Universality Theorem: Any bounded set x integer : x ≥ 0, Bx = b is inpolynomial-time-computable coordinate-embedding-bijection with some


Shmuel Onn

A[n] =


x integer : x ≥ 0, (1 1 1)[m][n]x = a


- All linear and integer programs are slim 3-way programs (SIAM Opt.)


Shmuel Onn

A[n] =


x integer : x ≥ 0, (1 1 1)[m][n]x = a


Scheme for Nonlinear Integer Programming:any program: max f(w1x, . . ., wdx) : x ≥ 0, Bx = b, x integer

n-fold program: max f(w’1x, . . ., w’dx) : x ≥ 0, (1 1 1)[m][n]x = a, x integercan be lifted to

The Computational Complexity of

Nonlinear Integer Programming:

The Graver Complexity of K3,m

Shmuel Onn

Graver Bases

Shmuel Onn

Graver Bases

Vector x conforms to y if lie in same orthant and |xi| ≤ |yi| and for all i.

Shmuel Onn

Graver Bases

The Graver basis of an integer matrix A is the finite set G(A) of conformal-minimal nonzero integer vectors x satisfying Ax = 0.


Shmuel Onn

Graver Bases


Example: Consider A=(1 2 1).


Shmuel Onn

Graver Bases


Example: Consider A=(1 2 1). Then G(A) consists of:


Shmuel Onn

Graver Bases


circuits: ±(2 -1 0)



Shmuel Onn

Graver Bases


circuits: ±(2 -1 0), ±(1 0 -1)



Shmuel Onn

Graver Bases


circuits: ±(2 -1 0), ±(1 0 -1), ±(0 1 -2)



Shmuel Onn

Graver Bases


circuits: ±(2 -1 0), ±(1 0 -1), ±(0 1 -2)


non-circuits: ±(1 -1 1)


Shmuel Onn

Graver Bases of N-ProductsConsider n-products of A:

A[n]=

Shmuel Onn

Graver Bases of N-ProductsConsider n-products of A:

The type of x = (x1, . . . , xn) is the number of its nonzero blocks.

A[n]=

Shmuel Onn

Graver Bases of N-Products

(Aoki-Takemura, Santos-Sturmfels, Hosten-Sullivant)

Consider n-products of A:

Lemma: Every integer matrix A has a finite Graver complexity g(A) such that any element in the Graver basis G(A[n]) for any n has type ≤ g(A).


A[n]=

Shmuel Onn


Theorem: Graver bases of A[n] and A(n) are computable in polynomial time.




A[n]=

Shmuel Onn


Theorem: Graver bases of A[n] and A(n) are computable in polynomial time.




A[n]=

Practicality: the complexity is high and dominated by ng(A), but promising scheme is to consider Graver elements of gradually increasing type.

Shmuel Onn

The Graver Complexity of a Graph

Shmuel Onn


Definition: The Graver complexity of a graph or a digraph G is the Graver complexity g(G):=g(A) of its vertex-edge incidence matrix A.

Shmuel Onn


Let g(m) := g(K3,m) = g((1 1 1)[m]).


Shmuel Onn


Let g(m) := g(K3,m) = g((1 1 1)[m]).

g(3) = g

For instance,

= 9


Shmuel Onn


Recall: g(m) := g(K3,m) = g((1 1 1)[m])

max f(w1x, . . ., wdx) : x ≥ 0, (1 1 1)[m][n]x = a, x integer

Theorem:Theorem: Consider the following universal nonlinear integer program:

Shmuel Onn


Recall: g(m) := g(K3,m) = g((1 1 1)[m])


1. 1. For any fixed m can solve it in polynomial time ng(m)


Shmuel Onn


Recall: g(m) := g(K3,m) = g((1 1 1)[m])



2. 2. For variable m it is universal and NP-hard


Shmuel Onn


Recall: g(m) := g(K3,m) = g((1 1 1)[m])



2. 2. For variable m it is universal and NP-hard

So ifSo if PP≠≠NP NP thenthen g(m) must grow with m. How fast ?


Shmuel Onn


Theorem:Theorem: We have Ω( 2m ) = g(m) = Ο( m46m ).

Recall: g(m) := g(K3,m) = g((1 1 1)[m])

Shmuel Onn



Also:Also: g(2)=3, g(3)=9, g(4) ≥ 27, g(5) ≥ 61.

Recall: g(m) := g(K3,m) = g((1 1 1)[m])

Shmuel Onn



Also:Also: g(2)=3, g(3)=9, g(4) ≥ 27, g(5) ≥ 61.

- Graver complexity of integer programming (Annals Combin.)

Recall: g(m) := g(K3,m) = g((1 1 1)[m])

Proofs of Theorems 1-4

about

Nonlinear N-Fold Integer Programming

Proof of Theorem 1 - Linear Optimization

Shmuel Onn


Shmuel Onn

- Compute the Graver basis of A(n) efficiently.


Shmuel Onn

- Use integer caratheodory theorem and convergence property to showthat G(A(n)) allows to augment feasible to optimal point efficiently.



Shmuel Onn


- Use an auxiliary n-fold program to find an initial feasible point.

- Use integer caratheodory theorem and convergence property to showthat G(A(n)) allows to augment feasible to optimal point efficiently.

Proof of Theorem 2 – Convex Maximization

Shmuel Onn


Shmuel Onn



Shmuel Onn

- Simulate linear optimization oracle using Theorem 1.



Shmuel Onn

- Simulate linear optimization oracle using Theorem 1.


- Reduce convex to linear maximization using the Geometric Theoremwith the Graver basis G(A(n)) providing a set of edge-directions of

conv x : A(n)x = a, x ≥ 0, x integer

Proof of Theorem 3 – Separable Convex Minimization

Shmuel Onn


Shmuel Onn

- Compute Graver basis of n-fold product of auxiliary larger matrix.


Shmuel Onn


- Use integer caratheodory, convergence property, and superadditivity, and use the Graver basis to augment feasible to optimal point.


Shmuel Onn


- Use an auxiliary n-fold program to find an initial feasible point.

- Use integer caratheodory, convergence property, and superadditivity, and use the Graver basis to augment feasible to optimal point.

Proof of Theorem 4 – lp-Nearest Point

Shmuel Onn


Shmuel Onn

- For positive integer p this is the special case of Theorem 3 with

(|x - x|p)p = |x1 – x1|p + . . . + |xnt – xnt|p


Shmuel Onn

- For positive integer p this is the special case of Theorem 3 with

(|x - x|p)p = |x1 – x1|p + . . . + |xnt – xnt|p

- For p infinity this reduces to the case of finite q for suitably large q.

Multiway Tables and Applications

Multiway Tables and Margins

Shmuel Onn

A k-way table is an m1 X . . . X mk array of nonnegative integers.Multiway Tables and Margins

Shmuel Onn

A k-way table is an m1 X . . . X mk array of nonnegative integers.A margin of a table is the sum of all entries in some flatof the table, so can be a line-sum, plane-sum, and so on.


Shmuel Onn

0 2 2 2 1 0

Example: 2-way table of size 2 X 3:



Shmuel Onn

0 2 2 2 1 0 3

Example: 2-way table of size 2 X 3 with line-sums:



Shmuel Onn

0 2 2 2 1 0 3

2




Shmuel Onn

0 2 2 2 1 0

43

2 3 2




Shmuel Onn

Example: 3-way table of size 6 X 4 X 3:



Shmuel Onn

24

Example: 3-way table of size 6 X 4 X 3 with a plane-sum:

03

503 320

14

12



Shmuel Onn

03

503 320

14

12

8

Example: 3-way table of size 6 X 4 X 3 with a line-sum:



Shmuel Onn

Casting “Long” Multiway Tables as N-fold Programs

Shmuel Onn

The margin equations for m1 X . . . X mk X n tables form an n-fold system defined by a suitable (r+s) x t matrix A, where A1 controls the equations of margins involving summation over layers, whereas A2 controls the equations of margins involving summation within a single layer at a time.


Shmuel Onn


A(n) =

n

x = (x1, . . . , xn),

A1(x1 + . . .+ xn) = a0, A2xk = ak, k=1, . . . n

a = (a0,a1, . . . , an)A(n) x = a,


Shmuel Onn


Example:

The line-sum equations for 3 x 3 x n tables are defined by the(9+6) x 9 matrix A where A1 is the 9 x 9 identity matrix and


Shmuel Onn

A2 = (1 1 1)[3] =

Corollary: nonlinear optimization over m1 X . . . X mk X n tableswith hierarchical margin constraints can be done in polynomial time.

Long Tables are Efficiently Treatable

Shmuel Onn

Privacy and Confidentiality in Statistical Data Bases:Entry Uniqueness Problem

Shmuel Onn

Shmuel Onn


Agencies allow web-access to information on their data basesbut are concerned about confidentiality of individuals.

Shmuel Onn



Common strategy: disclose margins but not table entries.

Shmuel Onn




If the value of an entry is the same in all tables with the disclosed margins then that entry is vulnerable; otherwise it is secure.

Shmuel Onn




If the value of an entry is the same in all tables with the disclosed margins then that entry is vulnerable; otherwise it is secure.

Corollary: For m1 X . . . X mk X n tables can check entry uniquenessin polynomial time and refrain from margin disclosure if vulnerable.

Shmuel Onn

Congestion Avoiding Transportation

Shmuel Onn


We wish to find minimum cost transportation or routing subject to demand and supply constraints and channel capacity constraints.

Shmuel Onn


f(x) = ∑ij fij(xij) = ∑ij cij|xij|aij

The classical approach assumes fixed channel cost per unit flow.But due to channel congestion when subject to heavy

traffic or communication load, the delay and cost are better approximated by a nonlinear function of the flow such as


Shmuel Onn


In the high dimensional extension, wish to find an optimal multiwaytable satisfying margin constraints and upper bounds on the entries.





Shmuel Onn


Corollary: We can find congestion avoiding transportation over m1 X . . . X mk X n tables in polynomial time.





In the high dimensional extension, wish to find an optimal multiwaytable satisfying margin constraints and upper bounds on the entries.

Shmuel Onn

Error Correcting Codes

Shmuel Onn

We wish to transmit a message X on a noisy channel. The message is augmented with some “check-sums” to allow for error correction.


Shmuel Onn


Multiway tables provide an appealing way of organizing the check sum protocol. The sender transmits the message as a multiway table augmented with slack entries summing up to pre-agreed margins.


Shmuel Onn



The receiver gets a distorted table X and reconstructs the messageas that table X with the pre-agreed margins that is lp-nearest to X.


Shmuel Onn


Corollary: We can error-correct messages of format m1 X . . . X mk X n in polynomial time.


The receiver gets a distorted table X and reconstructs the messageas that table X with the pre-agreed margins that is lp-nearest to X.


Bibliographyavailable online at http://ie.technion.ac.il/~onn

Shmuel Onn

- Convex discrete optimization (Montréal Lecture Notes)- Shaped partition problems (SIAM Opt.) - Cutting corners (Adv. App. Math.)- Partition problems with convex objectives (Math. OR)- Convex matroid optimization (SIAM Disc. Math.)- The Hilbert zonotope and universal Gröbner bases (Adv. App. Math.)- The complexity of 3-way tables (SIAM Comp.)- Convex combinatorial optimization (Disc. Comp. Geom.)- Markov bases of 3-way tables (J. Symb. Comp.)- All linear and integer programs are slim 3-way programs (SIAM Opt.)- N-fold integer programming (Disc. Opt. in memory of Dantzig) - Graver complexity of integer programming (Annals Combin.)- Nonlinear bipartite matching (Disc. Opt.) - Convex integer maximization via Graver bases (J. Pure App. Algebra)- Nonlinear matroid optimization and experimental design (SIAM Disc. Math.)- Convex integer minimization (submitted)- Nonlinear optimization over a weighted independence system (submitted)

Nonlinear Discrete Optimization - Cornell...

Documents

Transcript of Nonlinear Discrete Optimization - Cornell...