Locally Testable Codes and Expanders

Tali Kaufman

Joint work with Irit Dinur

Codes as CSPs • A constraint satisfaction problem (CSP) is a sequence

of constraints over variables: f1 (x1,x2,x5), f2 (x5,x1,x15), …

For example, max-3SAT, max-3col, …

• CSP describes a code if every two satisfying assignments are far apart (in Hamming distance). The minimum distance between a pair of sat assignments is the distance of the code.

• Today : CSPs that yield codes. • Challenge: CSPs that yield large and robust codes.

Robust Codes

• CSP describes a code if every two satisfying assignments are far apart (in Hamming distance).

• CSP describes a robust code if a vector that is far from the code falsifies many constraints.

• Robust codes are called LTCs (strong relation to PCP).

• Want to understand: What makes a code robust?

Structure

• One can associate a “constraint hyper-graph” with a CSP by placing a vertex per variable, and a hyper-edge per clause.

• Two “separate” aspects of a CSP are – The graph structure.– The constraint type : what functions fi do we allow.

Today we focus on the structure.

Constraint (Hyper graph) f1 (x1,x2,x3), f2 (x3,x4,x5), …

x1

x3

x4

x5

x2

x6

Constraint graphf1 (x1,x2,x3), f2 (x3,x4,x5), …

We replace every hyper-edge with a clique between its vertices, so that we deal with graphs instead of hyper-graphs.

x1

x3

x4

x5

x2

x6

x1

x3

x4

x5

x2

x6

Locally Testable Codes (LTCs)An LTC is given by a constraint graph G on n vertices.

The valid codewords C(G) {0,1}⊆ n are all assignments to [n] that satisfy all constraints.

Two required properties of the constraint graph:

1. Distance: Every two codewords differ on Ω(n) coordinates. The minimum distance is the code distance.

2. Robustness: Assignment that is γ-far (in Hamming distance) from the code, falsifies at least ργ fraction of the constraints (for some fixed ρ > 0 ).

Our question(s) for today

What is the relation between the structure of the constraint graph and the code obtained.

What graphs lead to codes?[Sipser-Spielman]

What graphs lead to robust codes (LTCs)?

Motivation• How can we construct LTCs? • Understand whether good LTCs exist.• Construct new, simpler LTCs, PCPs.• Understand the structure of general CSPs; CSPs

that are hard to approximate.

What graphs lead to codes

[Sipser-Spielman]: Expanders!

Expanders are graphs without small cuts.

G=(V,E) is an r-expander if for every S⊆V, |S| ≤ |V|/2 E(S,V-S) > r |S|

S V-SIn good expanders small setsexpands even more.

Expanders lead to codes - The Idea

Constraint graph: good expander.Constraints: local view; all zeros, or many non zeros.

Satisfying assignment must have a distance.

x4

x5

x9

x7

x2

x15

x1

x3

x10

x6

x8

x12

x11

x14

x13

Non zero variables zero neighbors

Satisfying assignment can not have low support: - Small set of non-zeros: by expansion its zero neighborhood is large.- Contradicts the requirement that each local constraint assigned many non zeros.

x4

x5

x9

x7

x2

x15

x1

x3

x10

x6

x8

x12

x11

x14

x13

Non zero variables zero neighbors

Expanders lead to codes - The Idea

Other issue:

How to ensure large code?

What graphs lead to robust codes?[Ben-Sasson, Harsha, Raskhodnikova]: Very good expanders yield codes that are NOT robust/LTC.

Idea: • Remove one constraint from the constraint graph. • The resulted graph is still an expander.• All satisfying assignments of the new graph are far apart. • Some sat assignments of the new graph do not sat the

original graph.

Conclusion: there is a vector far from code that falsifies only one constraint from the constraint graph.

What graphs lead to robust codes?[BHR]: Very good expanders yield codes that are not robust/LTC.There is a vector far from code that falsifies only one constraint from the constraint graph.

But: Maybe can test this code using *other* short constraint. I.e. constraints that are implied by original constraints.

Main point: Due to the high expansion, there are no short constraints that are implied by the original short constraints.

Example: f1 (x1,x2,x3): x1 + x2 + x3 = 0f2 (x4,x5,x6): x4 + x5 + x6 = 0f3 (x7,x8,x9): x7 + x8 + x9 = 0f4 (x3,x6,x9): x3 + x6 + x9 = 0f5 (x2,x5,x8): x2 + x5 + x8 = 0

Imply:f6 (x1,x4,x7): x1 + x4 + x7 = 0

Our Conclusions so farHigh expansion of the constraint graph implies code (distance).

High expansion implies NON robustness.

What can we say about the structure of the constraint graphs of LTCs?

Structure theorem for LTCsC is ρ-LTC : w γ-away from C rejected with probability > ρ γ

Theorem [Dinur-K]: The constraint graph of an LTC is a small set expander; all sets up to linear size expands.

Theorem: In ρ-LTC with r. distance Δ and with constraint graph G=(V,E), every S ⊆ V ; |S|≤ 0.75Δ|V| expands:

E(S,V-S) > ρd/3 |S| d = |E|/|V|.

LTC decomposes on sparse cutsC is ρ-LTC : vector γ-away from C rejected with probability > ρ γ

Decomposition on sparse cuts: Let |S| = γ|V|. If E(S,V-S) is a sparse cut, i.e., E(S,V-S) < ρd/3|S|= ργ/3|E| then

C =~ γ/3 CS x CV-S = {xSyV-S | xS C|S , yV-S C|V-S}

• Note C ⊆ CS x CV-S

• Every c CS x CV-S is γ/3-close to some codeword from C.

• Main point: Such c can be rejected only by edges (constraints) that cross the cut (S,V-S) and since the cut is sparse, there are too few of those edges.

Sparse cuts occur only on large setsC is ρ-LTC : vector γ-away from C rejected with probability > ρ γ

E(S,V-S) is a sparse cut: |S| = γ|V|= γ n , E(S,V-S) < ρd/3 |S|= ργ/3 |E|

C =~ γ/3 CS x CV-S

• Pick a ≠ a’ in CS with distance(a,a’) > |S|/3 = γ/3 n :

• Consider aS bV-S C ; a’S bV-S CS x CV-S

• Distance of a’b from C is < γ/3n since C=~ γ/3 CS x CV-S

• The closest codeword of C to a’S bV-S is w ≠ aS bV-S

• By triangle-inequality distance (a’b,ab) = distance (a,a’) > (Δ - γ/3) n

• In particular |S|= γ n > (Δ - γ/3) n ; |S| > 3/4 Δn

a’b

ab C

> γ/3 n w C

< γ/3 n by LTC

> Δn

Structure theorem for LTCs

We have shown: The constraint graph of an LTC is a small set expander.

Our main theorem shows that the constraint graph can be decomposed into constant-many (regular) expanders with few edges in between.

Main Theorem [Dinur-K]

• Let C be an LTC with constraint graph G=(V,E). Then V can be partitioned into V1,…Vt, such that

C =~ C1 x C2 x … x Ct

and such that each G(Vi) is an expander, with few edges between expanders. Moreover, each Ci is an LTC.

• t is at most 1/r.distance• Expansion depends on testability params

Summary and Questions•Expansion of a constraint graph implies distance.•Very high expansion of the constraint graph implies non-robustness.•The constraint graph of a robust code (LTC) is a small set expander for linear size sets. It can be decomposed into constant many (regular) expanders with few edges in between.•Q: What “amount” of expansion implies LTC?•Q: What structure of the constraint graph implies LTC?•Recent work (Dinur-K, and Ben-Sasson-Viderman) shows that the constraint graph cannot be dense for LTCs with good rate.