Constructive Algorithms for Discrepancy Minimization
description
Transcript of Constructive Algorithms for Discrepancy Minimization
1/26
Constructive Algorithms for Discrepancy Minimization
Nikhil Bansal (IBM)
2/26
Combinatorial Discrepancy
Universe: U= [1,…,n] Subsets: S1,S2,…,Sm
Color elements red/blue so each set is colored as evenly as possible.
S1
S2
S3
S4
3/26
ApplicationsCS: Computational Geometry, Comb. Optimization, Monte-Carlo simulation, Machine learning, Complexity, Pseudo-Randomness, …
Math: Dynamical Systems, Combinatorics, Mathematical Finance, Number Theory, Ramsey Theory, Algebra, Measure Theory, …
4/26
General Set System
Universe: U= [1,…,n] Subsets: S1,S2,…,Sm
Find : [n] ! {-1,+1} toMinimize |(S)|1 = maxS | i 2 S (i) |
For simplicity consider m=n henceforth.
5/26
Best Known AlgorithmRandom: Color each element i independently as x(i) = +1 or -1 with probability ½ each.
Thm: Discrepancy = O (n log n)1/2
Pf: For each set, expect O(n1/2) discrepancyStandard tail bounds: Pr[ | i 2 S x(i) | ¸ c n1/2 ] ¼ e-c2
Union bound + Choose c ¼ (log n)1/2
Analysis tight: Random actually incurs (n log n)1/2).
6/26
Better Colorings Exist![Spencer 85]: (Six standard deviations suffice) Always exists coloring with discrepancy · 6n1/2
(In general for arbitrary m, discrepancy = O(n1/2 log(m/n)1/2)Tight: For m=n, cannot beat 0.5 n1/2 (Hadamard Matrix, “orthogonal” sets)
Inherently non-constructive proof (pigeonhole principle on exponentially large universe)
Challenge: Can we find it algorithmically ?Certain algorithms do not work [Spencer]
Conjecture [Alon-Spencer]: May not be possible.
7/26
Approximating DiscrepancyQuestion: If a set system has low discrepancy (say << n1/2) Can we find a good discrepancy coloring ?
[Charikar, Newman, Nikolov 11]: Even 0 vs. O (n1/2) is NP-Hard
(Matousek): What if system has low Hereditary discrepancy? herdisc (U,S) = maxU’ ½ U disc (U’, S|U’)
Robust measure of discrepancyWidely used: TU set systems, Geometry, …
S1
S2
…
S’1
S’2
…
1 2 … n 1’ 2’ … n’
8/26
Our Results
Thm 1: Can get Spencer’s bound constructively. That is, O(n1/2) discrepancy for m=n sets.
Thm 2: For any set system, can findDiscrepancy · O(log (mn)) Hereditary discrepancy.
General Technique: Constructive bounds for geometric problems, Beck Fiala setting, k-permutation problem, …
9/26
Relaxations: LPs and SDPs
Not clear how to use.
Linear Program is useless. Can color each element ½ red and ½ blue. Discrepancy of each set = 0!
SDPs (vector coloring) | i 2 S vi |2 · n 8 S |vi|2 = 1 Intended solution vi = (+1,0,…,0) or (-1,0,…,0).Trivially feasible: vi = ei (all vi’s orthogonal)
Yet, SDPs will be the major tool.
10/26
Key Point
The SDP gap example does not work if discrepancy << n1/2
As we will see, SDPs are very useful in that regime.
But seems useless for Spencer’s problem
Idea: “Tighter” bounds for some sets. |i 2 S vi |2 · 2 n | i 2 S’ vi|2 · n/log n |vi|2 = 1
Why can one do this: Entropy Method.
Tighter bound for S’
11/26
Talk Outline
Introduction
The Method Low Hereditary discrepancy -> Good coloring
Additional IdeasSpencer’s O(n1/2) bound
12/26
Algorithm (at high level)
Cube: {-1,+1}n
Analysis: Few steps to reach a vertex (walk has high variance) Disc(Si) does a random walk (with low variance)
start
finish
Algorithm: “Sticky” random walk Each step generated by rounding a suitable SDP Move in various dimensions correlated, e.g. t
1 + t2 ¼ 0
Each dimension: An ElementEach vertex: A Coloring
13/26
An SDP Hereditary disc. ) the following SDP is feasible
SDP: Low discrepancy: |i 2 Sj
vi |2 · 2
|vi|2 = 1
Rounding: Pick random Gaussian g = (g1,g2,…,gn) each coordinate gi is iid N(0,1)
For each i, consider i = g ¢ vi
Obtain vi 2 Rn
14/26
Properties of RoundingLemma: If g 2 Rn is random Gaussian. For any v 2 Rn, g ¢ v is distributed as N(0, |v|2)Pf: N(0,a2) + N(0,b2) = N(0,a2+b2) g¢ v = i v(i) gi » N(0, i v(i)2)
1. Each i » N(0,)
2. For each set S, i 2 S i = g ¢ (i2 S vi) » N(0, · 2)
(std deviation ·)
SDP:|vi|2 = 1|i2 S
vi|2 ·2
Recall: i = g ¢ vi
’s mimics a low discrepancy coloring (but is not {-1,+1})
15/26
Algorithm Overview Construct coloring iteratively.Initially: Start with coloring x0 = (0,0,0, …,0) at t = 0.At Time t: Update coloring as xt = xt-1 + (t
1,…,tn)
( tiny: 1/n suffices)
x(i)
xt(i) = (1i + 2
i + … + ti)
Color of element i: Does random walk over time with step size ¼
Fixed if reaches -1 or +1.
time
-1
+1
Set S: xt(S) = i 2 S xt(i) does a random walk w/ step N(0,· 2)
16/26
Analysis Consider time T = O(1/2)
Claim 1: With prob. ½, at least n/2 elements reach -1 or +1.Pf: Each element doing random walk with size ¼ Recall: Random walk with step 1, is ¼ O(t1/2) away in t steps. A Trouble: Walks for various elements are correlated
Consider basic walk x(t+1) = x(t) 1 with prob ½Define Energy (t) = x(t)2 if never reached n1/2 = (t-1) + 1 otherwise E[(t)] = ½ (x(t-1)+1)2 + ½ (x(t-1)-1)2 = x(t-1)2 + 1 = (t-1)+1
So, E[(10 n) ] = 10 n. Moreover, (10 n) · 11 n So only a small fraction of walks can have < n.
17/26
AnalysisConsider time T = O(1/2)
Claim 2: Each set has O() discrepancy in expectation.Pf: For each S, xt(S) doing random walk with step size ¼
Define a round as T = O(1/2) time steps. Claim 1: ) Everything colored in O(log n) rounds.
Claim 2: ) Expected discrepancy of a set at end = O( log n)
+ By Chernoff Bounds, discrepancy = O( log mn) whp over all sets.
18/26
Recap At each step of walk, formulate SDP on unfixed variables. Use some (existential) property to argue SDP is feasible Rounding SDP solution -> Step of walk
Properties of walk: High Variance -> Quick convergence Low variance for discrepancy on sets -> Low discrepancy
19/26
Refinements
Spencer’s six std deviations result: Goal: Obtain O(n1/2) discrepancy for any set system on m = O(n) sets.
Random coloring has n1/2 (log n)1/2 discrepancy
Previous approach seems useless: Expected discrepancy for a set O(n1/2), but some random walks will deviate by up to (log n)1/2 factor
Need an additional idea to prevent this.
20/26
Spencer’s O(n1/2) result
Partial Coloring Lemma: For any system with m sets, there exists a coloring on ¸ n/2 elements with discrepancy O(n1/2 log1/2 (2m/n))
[For m=n, disc = O(n1/2)]
Algorithm for total coloring: Repeatedly apply partial coloring lemma Total discrepancy O( n1/2 log1/2 2 ) [Phase 1]+ O( (n/2)1/2 log1/2 4 ) [Phase 2] + O((n/4)1/2 log1/2 8 ) [Phase 3]+ … = O(n1/2)
21/26
Proving Partial Coloring LemmaBeautiful Counting argument (entropy method + pigeonhole)Idea: Too many colorings (2n), but few “discrepancy profiles”
Key Lemma: There exist k=24n/5 colorings X1,…,Xk such that every two Xi, Xj are “similar” for every set S1,…,Sn.
Some X1,X2 differ on ¸ n/2 positionsConsider X = (X1 – X2)/2
Pf: X(S) = (X1(S) – X2(S))/2 2 [-10 n1/2 , 10 n1/2]
X1 = ( 1,-1, 1 , …,1,-1,-1)X2 = (-1,-1,-1, …,1, 1, 1) X = ( 1, 0, 1 , …,0,-1,-1)
22/26
A useful generalization
There exists a partial coloring with non-uniform discrepancy bounds S for set S
Provided S = ( n1/2) in some average sense
23/26
An SDP Suppose there exists partial coloring X:1. On ¸ n/2 elements2. Each set S has |X(S)| · S
SDP: Low discrepancy: |i 2 Sj
vi |2 · S2
Many colors: i |vi|2 ¸ n/2
|vi|2 · 1
Pick random Gaussian g = (g1,g2,…,gn) each coordinate gi is iid N(0,1)
For each i, consider i = g ¢ vi
Obtain vi 2 Rn
24/26
Algorithm
Initially write SDP with S = c n1/2
Each set S does random walk and expects to reach discrepancy of O(S) = O(n1/2)
Some sets will become problematic. Reduce their S on the fly.Not many problematic sets, and entropy penalty low.
0 20n1/2 30n1/2 35n1/2 …
Danger 1 Danger 2 Danger 3 …
25/26
Concluding RemarksProbable right answer: O( (log m)1/2) for her. Disc.
Can be derandomized [Bansal-Spencer] (add new constraints to SDP)
Gives derandomization for prob. inequalities stronger than Chernoff bounds.(Exponential moment technique for derandomizing Chernoff loses (log n)1/2)
Other non-constructive problems: Lattices (Minkowski Thm) Fixed-Point Based (Nash, Sperner’s Lemma) Topological (Hypergraph matching), …
Discrepancy problems: Beck Fiala Conjecture (more generally Komlos conjecture) Erdos Discrepancy problem 3-permutation conjecture, …
26/26
Thank You!