Kunal Talwar
MSR SVC
The Price of Privacy and the Limits of LP decoding
[Dwork, McSherry, Talwar, STOC 2007]
Compressed Sensing:If x 2 RN is k-sparseTake M ~ Ck log N/k random Gaussian measurements
Then L1 minimization recovers x.
For what k does this make sense (i.e M < N)?
How small can C be?
Teaser
Database of information about individualsE.g. Medical history, Census data, Customer
info.Need to guarantee confidentiality of individual
entriesWant to make deductions about the database;
learn large scale trends.E.g. Learn that drug V increases likelihood of
heart diseaseDo not leak info about individual patients
Setting
Curator
Analyst
Simple Model (easily justifiable)Database: n-bit binary vector xQuery: vector aTrue answer: Dot product axResponse is ax+e = True Answer + Noise
Blatant Non-Privacy: Attacker learns n−o(n) bits of x.
Theorem: If all responses are within o(√n) of the true answer, then the algorithm is blatantly non-private even against a polynomial time adversary asking O(n log2n) random questions.
Dinur and Nissim [2003]
Privacy has a PriceThere is no safe way to avoid increasing the
noise as the number of queries increases
Applies to Non-Interactive SettingAny non-interactive solution permitting
answers that are “too accurate” to “too many” questions is vulnerable to the DiNi attack.
This work : what if most responses have small error, but some can be arbitrarily off?
Implications
Real vector x 2 Rn
Matrix A 2 Rmxn
with i.i.d. Gaussian entriesTransmit codeword Ax 2 R
m
Channel corrupts message. Receive y=Ax +eDecoder must reconstruct x, assuming e has
small support small support: at most m entries of e are non-
zero.
Error correcting codes: Model
Channel
Encoder Decoder
The Decoding problem
min support(e')such that
y=Ax'+e'
x' 2 Rn
solving this would give the original message x.
min |e'|1
such that
y=Ax'+e'
x' 2 Rn
this is a linear
program; solvable in poly time.
Theorem [Donoho/ Candes-Rudelson-Tao-Vershynin]For an error rate < 1/2000, LP decoding succeeds in recovering x (for m=4n).
This talk: How large an error rate can LP decoding tolerate?
LP decoding works
Let * = 0.2390318914495168038956510438285657…
Theorem 1: For any <*, there exists c such that if A has i.i.d. Gaussian entries, and ifA has m = cn rowsFor k=m, every support k vector ek satisfies|e – ek| <
then LP decoding reconstructs x’ where |x’-x|2 is O( ∕ √n).
Theorem 2: For any >*, LP decoding can be made to fail, even if m grows arbitrarily.
Results
In the privacy setting: Suppose, for <*, the curatoranswers (1- ) fraction of questions within error o(√n)answers fraction of the questions arbitrarily.
Then the curator is blatantly non-private.
Theorem 3: Similar LP decoding results hold when the entries of A are randomly chosen from §1.
Attack works in non-interactive setting as well. Also leads to error correcting codes over finite
alphabets.
Results
Theorem 1: For any <*, there exists c such that if B has i.i.d. Gaussian entries, and ifB has M = (1 – c) N rowsFor k=m, for any vector x 2 R
N
then given Ax, LP decoding reconstructs x’ where
In Compressed sensing lingo
jx ¡ x0j2 · CpN
infxk :jxk j0 · k jx ¡ xk j1
Let * = 0.2390318914495168038956510438285657…
Theorem 1 (=0): For any <*, there exists c such that if A has i.i.d. Gaussian entries with m=cn rows, and if the error vector e has support at most m, then LP decoding accurately reconstructs x.
Proof sketch…
Rest of Talk
Scale and translation invariance
LP decoding is scale and translation invariant
Thus, without loss of generality, transmit x = 0
Thus receive y = Ax+e = e
If reconstruct z 0, then |z|2 = 1
Call such a z bad for A.
Ax
Ax’
y
Proof Outline
Proof:Any fixed z is very unlikely to be bad for A:
Pr[z bad] · exp(-cm)
Net argument to extend to Rn:Pr[9 bad z] · exp(-c’m)
Thus, with high probability, A is such that LP decoding never fails.
z bad: |Az – e|1 < |A0 – e|1
) |Az – e|1 < |e|1
Let e have support T.Without loss of generality,
e|T = Az|T
Thus z bad:|Az|T
c < |Az|T
) |Az|T > ½|Az|1
Suppose z is bad…
e1 e2 e3 ....
em
000....0
a1za2za3z....
am
z
T
0 y=e Az
0Tc
A i.i.d. Gaussian ) Each entry of Az is an i.i.d. Gaussian
Let W = Az; its entries W1,…Wm are i.i.d. Gaussians
z bad ) i 2 T |Wi| > ½i |Wi|
Recall: |T| · m
Define S(W) to be sum of magnitudes of the top fraction of entries of W
Thus z bad ) S(W) > ½ S1(W)
Few Gaussians with a lot of mass!
Suppose z is bad…
0
T
Let us look at E[S]
Let w* be such that
Let * = Pr[|W| ¸ w*]Then E[S*] = ½ E[S1]
Moreover, for any < *, E[S] · (½ – ) E[S1]
Defining *
E[S*] =½ E[S1]
E[S]
w*
S depends on many independent Gaussians.
Gaussian Isoperimetric inequality implies:
With high probability, S(W) close to E[S].
S1 similarly concentrated.
Thus Pr[z is bad] · exp(-cm)
Concentration of measure
E[S*] =½ E[S1]
E[S]
Now E[S] > ( ½ + ) E[S1]
Similar measure concentration argument shows that any z is bad with high probability.
Thus LP decoding fails w.h.p. beyond *
Donoho/CRTV experiments used random error model.
Beyond *
E[S*] =½ E[S1]
E[S]
Compressed Sensing:If x 2 RN is k-sparseTake M ~ Ck log N/k random Gaussian measurements
Then L1 minimization recovers x.
For what k does this make sense (i.e M < N)? How small can C be?
Teaser
k < * N ≈ 0.239 N
C > (* log 1/ * )–1 ≈ 2.02
Tight threshold for Gaussian LP decodingTo preserve privacy: lots of error in lots of
answers.
Similar results hold for +1/-1 queries.
Inefficient attacks can go much further:Correct (½-) fraction of wild errors.Correct (1-) fraction of wild errors in the list
decoding sense.Efficient Versions of these attacks?
Dwork-Yekhanin: (½-) using AG codes.
Summary
Top Related