Post on 25-Feb-2021
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Differentially Private Aggregation of DistributedTime-series with Transformation and Encryption
Part 4 — Differentially Private Aggregation 2
presenter: Le Chen
Nanyang Technological University
lechen0213@gmail.com
October 27, 2013
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Introduction
Outline
Introduction
L2 Sensitivity
Discrete Fourier Transform
Conclusion & Discussion
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Introduction
Reference
Cynthia Dwork.Differential Privacy.Invited talk at ICALP, Venice, Italy, July 10-14, 2006.Automata, Languages and Programming, Lecture Notes inComputer Science Volume 4052, 2006, pp 1-12.
Wikipedia.<http:
//en.wikipedia.org/wiki/Differential_privacy>
Elaine Shi, T-H. Hubert Chan, Eleanor Rieffel, Richard Chowand Dawn Song.Privacy-Preserving Aggregation of Time-Series Data.In Network and Distributed System Security Symposium(NDSS), 2011.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Introduction
Reference
T-H. Hubert Chan, Elaine Shi, and Dawn SongPrivacy-Preserving Stream Aggregation with Fault Tolerance.16th International Conference, FC 2012, Kralendijk, Bonaire,Februray 27-March 2, 2012. Financial Cryptography and DataSecurity, Lecture Notes in Computer Science Volume 7397,2012, pp 200-214.
Vibhor Rastogi, Suman NathDifferentially Private Aggregation of Distributed Time-serieswith Transformation and Encryption.ACM SIGMOD 2010, New York, NY. Proceedings of the 2010ACM SIGMOD International Conference on Management ofData, pages 735-746.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Motivation
Motivation of Aggregation
Figure: System Model
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Motivation
Notations
I I = I1 ∪ I2 · · · ∪ IU is a set of vectors.
I Ii = [Ii1, Ii2, · · · , Iin] is a n × 1 vector, where Iij ∈ {0, 1}represents whether the average weight of user is over 200 lb inmonth j .
I nbrs(I ): the data set obtained from adding/removing oneuser’s data from I .
I Qj(I ) outputs ”the number of users over 200 lb” in month j ,
i.e.∑U
n=1 Iikj .
I Q(I ) = {Q1(I ),Q2(I ), · · ·Qn(I )}.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Motivation
LPA (Laplace Perturbation Algorithm)
I By adding LAP(λ) noise to the final results, LPA(Q, λ) isε-differentially private for λ = ∆(Q)/ε.
I Q̃ = Q(I ) + LAPn(λ).
I DLPA: Each user u adds its noise nu to xu.
I Q̃ =∑
u(xu + nu) = Q(I ) +∑
u nu.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Motivation
Basic Distributed Protocol
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Motivation
Distributed Decryption
I Distributed keys: The private key λ is shared by U users asλ =
∑u λu.
I Each user u computes his decryption share cu = cλu .
I The aggregator combines c ′ =∏U
u=1 cu.
I The final result t = L(c ′ mod N2)L(gλ mod N2)
mod N.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Motivation
Protocol for Computing Exact Sum
I Encrypt-Sum(xu, ru): the encryption of∑Uu=1(xu + ru) = Q +
∑Uu=1 ru.
I Decrypt-Sum(c , ru): each user u computes a decryption sharec ′u = cλug−ruλ.
I Final result: the aggregator aggregates c ′ =∏U
u=1 c ′u and gets
Q = L(c ′ mod N2)L(gλ mod N2)
mod N.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Motivation
Protocol for Computing Noisy Sum
I Let Yi ∼ N(0, λ) for i ∈ {1, 2, 3, 4} be four Gaussian randomvariables. Then Z = Y 2
1 + Y 22 − Y 2
3 − Y 24 is a Lap(2λ2)
random variable.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Motivation
Protocol for Computing Noisy Sum
where a =∑
u au, Enc(a2) is public, and∑
u bu = 0.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
L2 Sensitivity
L2 Sensitivity
I If Q is a query sequence, Q(I ) and Q(I ′) are each vectors.
I L1 distance metric, denoted as |Q(I )− Q(I ′)|1, that measuresthe Manhattan distance
∑j |Qj(I )− Qj(I ′)| between these
vectors.
I L2 distance metric, denoted as |Q(I )− Q(I ′)|2, that measures
the Euclidean distance√∑
j(Qj(I )− Qj(I ′))2.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
L2 Sensitivity
Sensitivity
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
L2 Sensitivity
Sensitivity Example
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Discrete Fourier Transform
DFT (Discrete Fourier Transform)
I The DFT of an n-dimensional sequence X is a lineartransform giving another n-dimensional sequence.
I DFT (X ): the j th element DFT (X )j =∑n
i=1 e2π√−1
njiXi .
I DFT (X ): the j th element of the Inverse DFT is
IDFT (X )j = 1n
∑ni=1 e−
2π√−1
njiXi .
I Furthermore, IDFT (DFT (X )) = X .
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Discrete Fourier Transform
DFT k(X )
I Denote DFT k(X ) as the first k elements of DFT (X ).
I The elements of DFT k(X ) are called the Fourier coefficientsof the k lowest frequencies and they compactly represent thehigh-level trends in X .
I An approximation X ′ to X can be obtained from DFT k(X ) asfollows:
I PADn(DFT k(X )): appends n − k 0 to DFT k(X ),I compute X ′ = IDFT (PADn(DFT k(X ))).
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Discrete Fourier Transform
DFT k(X )
I Obviously X ′ may be different from X as ignoring the lastn − k Fourier coefficients may introduce some error.
I Denote RE kj (X ), short for reconstruction error at the j th
position, to be the value |X ′j − Xj |.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Discrete Fourier Transform
Reconstruction Error
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Discrete Fourier Transform
Reconstruction
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Fourier Perturbation Algorithm
FPA (Fourier Perturbation Algorithm)
I The parameter λ in FPAk needs to be adjusted in order to getε-differential privacy.
I Since FPAk perturbs the sequence F k , λ has to be calibratedaccording to the L1 sensitivity, ∆1(F k), of F k .
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Fourier Perturbation Algorithm
Proof
I (i) holds since ∆2(F k) ≤ ∆2(Q), as the n Fourier coefficientshave the same L2 norm as Q, while F k ignores the last n − kFourier coefficients,
I and ∆1(F k) ≤√
k∆2(F k), due to a standard inequalitybetween the L1 and L2 norms of a sequence.
I Let xj = |Qj(I )− Qj(I ′)|, then ∆1(Q) =∑
j |Qj(I )− Qj(I ′)|=
∑j xj , ∆2(Q) =
√∑j(Qj(I )− Qj(I ′))2 =
√∑j x2
j . Since
the inequality ∑j xj
n≤
√∑j x2
j
n
holds, we can see that ∆1(F k) ≤√
k∆2(F k).
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Fourier Perturbation Algorithm
Proof
I (ii) follows since for λ =√
k∆2(Q)/ε ≥ ∆1(F k)/ε,
I F̃ k = LPA(F k , λ) computed in step 2 is ε-differential private,and Q̃ in step 3 is obtained using F̃ k only.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Fourier Perturbation Algorithm
Accuracy
I The theorem shows that the error by FPAk for each query isk/ε+ RE k
i (Q(I )), but the LPA yields an error of n/ε.
I Since the reconstruction error, RE ki (Q(I )), is often small even
for k << n, we expect the error in FPAk to be much smallerthan in LPA.
I This hypothesis is confirmed in our experiments that showthat FPAk gives orders of magnitude improvement over LPAin terms of error.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Fourier Perturbation Algorithm
Choosing the Right k
I So far we have assumed that k is known to us.
I Since errori (FPAk) is k/ε+ RE ki (Q(I )), a good value of k is
important in obtaining a good trade-off between theperturbation error, k/ε, and the reconstruction error,RE k
i (Q(I )).
I If k is too big, the perturbation error becomes too big, while ifk is too small the reconstruction error becomes too high.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Fourier Perturbation Algorithm
Choosing the Right k
I We can often choose k based on prior assumptions aboutQ(I ).
I For instance, if Q(I ) is such that the Fourier coefficientscorresponding to Q(I ) decrease exponentially fast, then only aconstant number (say k = 10) of Fourier coefficients need tobe retained during perturbation.
I Our experiments show that this naive method is applicable inmany practical scenarios as Fourier coefficients of manyreal-word sequences decrease very rapidly.
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption
Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion
Conclusion & Discussion
Conclusion & Discussion
I Last week, we introduced an aggregation protocol supportsdistributed differential privacy and distributed decryption.
I This week we talked about using (DFT) Discrete FourierTransform in LPA (Laplace Perturbation Algorithm) to getFPA (Fourier Perturbation Algorithm), and discussed the errorof DFT and chosen of k.
I FPA can achieve the same differential privacy level with lesserror, the noise actually reduces by a factor of n/k .
presenter: Le Chen Nanyang Technological University lechen0213@gmail.com
Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption