Random Low Degree Polynomials Are Hard To...

IntroductionProof

Concluding Remarks and Open Problems

Random Low Degree Polynomials Are Hard ToApproximate

Ido Ben-EliezerRani Hod

Shachar Lovett

August 4, 2009

I. Ben-Eliezer, R. Hod and S. Lovett Random Low Degree Polynomials Are Hard To Approximate

IntroductionProof

Outline

1 Introduction

2 Proof

3 Concluding Remarks and Open Problems

IntroductionProof

Outline

1 Introduction

2 Proof

IntroductionProof

Outline

1 Introduction

2 Proof

IntroductionProof

Outline

1 Introduction

2 Proof

IntroductionProof

Introduction - polynomials

We study functions f : Fn2 → F2. Every such function can be

represented as a (multilinear) polynomial. (e.g., x1 + . . .+ xn

is the XOR function, x1 · · · xn is the AND function).

A monomial is a product of some variables, its degree is thenumber of distinct variables (the degree of x1x3x7x19 is 4).

The degree of a polynomial is the maximal degree among allits monomials.

Random polynomial of degree d - the coefficients of allmonomials of degree at most d are chosen uniformly andindependently from {0, 1}.

IntroductionProof

Correlation - definition

Definition

Two functions f , g : Fn2 → F2 are ε-correlated if

Prx∈Fn2

[f (x) = g(x)] ≥ 1 + ε

The bias of a function f is E [(−1)f ].

If the sum of two functions has small bias then they are notcorrelated.

IntroductionProof

Definition

Prx∈Fn2

[f (x) = g(x)] ≥ 1 + ε

IntroductionProof

Definition

Prx∈Fn2

[f (x) = g(x)] ≥ 1 + ε

IntroductionProof

Our goal

We are interested in functions that have a low correlation withthe set of all polynomials of degree at most d . How Complexmust such a function be?

We show a far tail bound on the bias of a random polynomialof degree d + 1 has a very small bias with very high probability.

By using the union bound we conclude that a typicalpolynomial of degree d + 1 cannot be approximated bypolynomials of degree d .

IntroductionProof

Our goal

IntroductionProof

Our goal

IntroductionProof

Our goal – From broader perspective

A very fundamental problem in Complexity Theory is to findobjects that cannot be approximated by objects of lowercomplexity.

For a function f : Fn2 → F2, one of the most natural

complexity measures is its degree when it is represented as a(multilinear) polynomial.

Polynomials are strongly related to circuits. Constant depthcircuits with certain types of gates can be approximated bylow degree polynomials.

IntroductionProof

Random functions cannot be approximated

It is easy to verify, using a simple union bound, that for anyconstant δ < 1 and for d < δn, a random function has anexponentially small correlation with degree d polynomials.

A random function has high complexity. In particular, itsdegree is close to n whp.

We show that random polynomials cannot be approximated byslightly less complex functions whp.

IntroductionProof

Our main result

Theorem

There exist a constant 0 < δmax < 1 and constants c, c ′ > 0 suchthat the following holds. Let f be a random n-variate polynomialof degree d ≤ δmaxn. Then,

Pr[∃p ∈ RM(n, d − 1) : corr (f , p) ≥ 2−cn/d

−c ′( n≤d)

IntroductionProof

The bias of random polynomials

Our theorem follows from the following lemma.

Lemma (Main Lemma)

Let ε > 0 and let f be a random degree d polynomial ford ≤ (1− ε)n. Then,

Pr[|bias(f )| > 2−c1n/d

]≤ 2

−c2( n≤d).

IntroductionProof

The bias of random polynomials

Our theorem follows from the following lemma.

Lemma (Main Lemma)

Let ε > 0 and let f be a random degree d polynomial ford ≤ (1− ε)n. Then,

Pr[|bias(f )| > 2−c1n/d

]≤ 2

−c2( n≤d).

IntroductionProof

Polynomials as codewords

The Reed-Muller code RM(n, d) is the defined as the set ofall polynomials of degree at most d over Fn

This is a linear code with minimal weight 2n−d . That is, thebias of every non-zero polynomial (codeword) is at most1− 21−d . We prove that the vast majority of the codewordshave bias very close to 0.

In other words, our main lemma actually shows a new boundon the weight distribution of Reed-Muller code.

IntroductionProof

Related work

Explicit constructions of functions that cannot beapproximated by degree-d polynomials are known ford ≤ Θ(log n) (e.g., [VW08]).

The main lemma shows the existence of a large linear space ofpolynomials with very small bias. [ABK08] used such aconstruction to provide a lower bound on the support size ofdistributions that fool low degree polynomials. Using ourlemma, we can improve this result.

IntroductionProof

Related work

Explicit constructions of functions that cannot beapproximated by degree-d polynomials are known ford ≤ Θ(log n) (e.g., [VW08]).

The main lemma shows the existence of a large linear space ofpolynomials with very small bias. [ABK08] used such aconstruction to provide a lower bound on the support size ofdistributions that fool low degree polynomials. Using ourlemma, we can improve this result.

IntroductionProof

Outline

1 Introduction

2 Proof

IntroductionProof

Proof outline

We consider the bias of a random polynomial, and show howit is related to the probability that a randomly generatedvector is in RM(n, d)⊥ (defined as the set of vectors in F2n

that are orthogonal to every polynomial of degree at most d).

We use algebraic techniques to estimate the probability of thelatter event.

We use Markov inequality on a high moment of the bias.

IntroductionProof

Proof outline

IntroductionProof

Proof outline

IntroductionProof

The moments of the bias of a polynomial

The following lemma shows how to compute the moments ofthe bias of a random polynomial.

Fix t ∈ N and let p ∈ RM(n, d) and x1, . . . , xt ∈ Fn2 be chosen

independently and equiprobably. Then,

E[bias(p)t

]= Pr [∀q ∈ RM(n, d), q(x1) + . . .+ q(xt) = 0] .

IntroductionProof

The moments of the bias of a polynomial

The following lemma shows how to compute the moments ofthe bias of a random polynomial.

Fix t ∈ N and let p ∈ RM(n, d) and x1, . . . , xt ∈ Fn2 be chosen

independently and equiprobably. Then,

E[bias(p)t

]= Pr [∀q ∈ RM(n, d), q(x1) + . . .+ q(xt) = 0] .

IntroductionProof

The d-rank of a set of vectors

Definition

Given a point x ∈ Fn2, its d-evaluation is

eval(x) =

(∏i∈I

)I⊆[n],|I |≤d

Given a set of (random) points x1, . . . , xt ∈ Fn2 define the matrix

M(x1, . . . , xt) to be the d-evaluation of x1, . . . , xt . The d-rank ofthis set of points is the rank of this matrix.

IntroductionProof

The d-rank of a set of vectors - Cont.

Note that p(x1) + · · ·+ p(xt) = 0 for every polynomial ofdegree d if and only if the sum of the rows of M(x1, . . . , xt)is 0.

We partition the set of variables V to two sets: V ′ of sizen(1− 1

d ) and V ′′ of size nd .

Consider the event E where columns that correspond tomonomials with exactly one variable from V ′′ sum to zero.

IntroductionProof

Denote by S the set of monomials with at most d − 1variables from V ′ (without variables from V ′′).

Denote by S∗ the set of monomials of degree 1 with variablefrom V ′′.

The event E occurs if and only if every column fromM(x1, . . . , xt) that corresponds to a monomial from S isorthogonal to every column that corresponds to a monomialfrom S∗ (which are uniformly chosen vectors).

IntroductionProof

Consider the submatrix M ′ which contains only columns fromS .

We have to bound the probability that M ′ has low rank.

Equivalent to the probability that x1, . . . , xt restricted to V ′

has low (d − 1)-rank.

IntroductionProof

A lower bound on the d-rank

Let x1, . . . , xR be R = 2r distinct points in Fn2. Consider the linear

space of degree d polynomials restricted to these points; that is,the space

{(p(x1), . . . , p(xR)) : p ∈ RM(n, d)} .

The linear dimension of this space is at least( r≤d

This lemma was proved independently in [KS05].

IntroductionProof

A lower bound on the d-rank

Let x1, . . . , xR be R = 2r distinct points in Fn2. Consider the linear

space of degree d polynomials restricted to these points; that is,the space

{(p(x1), . . . , p(xR)) : p ∈ RM(n, d)} .

The linear dimension of this space is at least( r≤d

This lemma was proved independently in [KS05].

IntroductionProof

Therefore, if a set of points have small d-rank then they havesmall linear rank.

Bounding the probability that the points have low rank is easy.

Some calculations show that with probability 1− 2−c(nd) the

columns have rank at least β( nd−1

IntroductionProof

Recap (2)

We consider the t-moment of the bias, for t = Θ(( nd−1

By Markov inequality, we conclude that

Pr[|bias(f )| ≥ 2−c2n/d ] ≤ 2−c3(nd).

IntroductionProof

Recap (2)

We consider the t-moment of the bias, for t = Θ(( nd−1

By Markov inequality, we conclude that

Pr[|bias(f )| ≥ 2−c2n/d ] ≤ 2−c3(nd).

IntroductionProof

Outline

1 Introduction

2 Proof

IntroductionProof

The main lemma is tight

Proposition

Fix ε > 0 and let f be a random degree d polynomialfor d ≤ (1/2− ε)n. There are constants c ′1, c

′2 such that

Pr[|bias(f )| > 2−c ′1n/d

]≥ 2

−c ′2(n≤d).

IntroductionProof

Concluding remarks and open questions

Recently, [KL09] gave further bounds on the weightdistribution of Reed-Muller codes. Still, determining the exactweight distribution of such codes is a major open question.

Find other families of low complexity functions that typicallycannot be approximated by low degree polynomials.

Provide an explicit function that cannot be approximated bydegree-d polynomials, for d � log n.

IntroductionProof

Questions?

Random Low Degree Polynomials Are Hard To...

Documents

Transcript of Random Low Degree Polynomials Are Hard To...