Randomness Extraction: A Survey
David Zuckerman
University of Texas at Austin
Randomness in Computer Science
• Many uses of randomness in CS.– Randomized algorithms– Cryptography– Distributed computing
• But: high-quality randomness expensive.• Can low-quality (weak) randomness suffice?
Models for Weak Randomness
• Independent bits with same, unknown bias– [von Neumann ’51]
• Semirandom sources [Santha-Vazirani ‘84]– δ < Pr[Xi|X1=x1,…,Xi-1=xi-1] < 1-δ– Block sources [Chor-Goldreich ‘85]
• Bit-fixing sources [CFGHRS ‘85,…]– k uniform bits; others set by adversary.
General Weak Random Source [Z ‘90]
• Random variable X on {0,1}n.• General model: min-entropy
• Flat source:– Uniform on A,
|A| ≥ 2k.|A| ³ 2k
{0,1}n
General Weak Random Source [Z ‘90]
• Can arise in different ways:– Physical source of randomness.– Cryptography: condition on adversary’s
information, e.g. bounded storage model.
– Pseudorandom generators (for space s machines): condition on TM configuration.
Goal: Extract Randomness
Ext n bits m bits
statistical error
Problem: Impossible, even for k=n-1, m=1, ε<1/2.
Impossibility Proof
• Suppose f:{0,1}n {0,1} satisfies sources X ∀with H∞(X) ≥ n-1, f(X) ≈ U.
f-1(0)f-1(1)
Take X=f-1(0)
Randomness Extractor: short seed[Nisan-Z ‘93,…, Guruswami-Umans-Vadhan ‘07]
Ext n bits m =.99k bits
statistical error
d=O(log (n/ε)) random bit seed Y
Strong extractor: (Ext(X,Y),Y) ≈ Uniform
Outline
• Seeded Extractors– Basic applications– Alternate view with applications– Sketch of two constructions
• Seedless Extractors for Structured Sources– Algebraic sources: independent, affine, …
• Applications in cryptography
– Complexity-theoretic sources• Crypto-tailored Extractors
Simulating Randomized Algorithms• Randomized algorithm R using m random bits.• Assume only random bits X have H∞(X)≥k>m.
– No high-quality randomness available.
• Given Ext for H∞(X)≥k– seed length d, output length m.
• Simulation with factor 2d blowup:– Run R with random string Ext(x,y1),…,Ext(x,y2d).– Take majority vote or median.
Use in Privacy Amplification[Bennett, Brassard, Robert 1985]
• Goal: convert weak shared secret X to uniform secret.• Unbounded passive adversary.
public
Pick Y
Shared secret = Ext(X,Y). Correct by strong extractor definition.
PRGs for Space-Bounded Machines
• Basic PRG: G(x,y) = (x,Ext(x,y)) [Nisan-Z]• Condition on configuration v after read x.• Whp • Hence whp Ext(X,Y) close to uniform.
• G:{0,1}O(s) {0,1}poly(s) fools space s TMs [NisanZ]• Sometimes can avoid union bound!
– O(log n log log n) bit seed fools read-once polylog-width “regular” BPs [BRRY ‘10,BV ‘10]
– O(log n) bit seed fools read-once O(1)-width permutation BPs [KNP].
PRGs from Shrinkage
• Hardness vs. Randomness paradigm:– Lower bounds give PRGs [Nisan-Wigderson,…].
• But: need superpolynomial lower bounds.• Known: polynomial lower bounds for restricted
models.– E.g., formulas Ω(n3/polylog n) [Andreev, Hastad].
• [Impagliazzo, Meka, Z 2012]: polynomial lower bounds proved via shrinkage give PRGs.– E.g., seed length s1/3+o(1) fools size s formulas.
Graph-Theoretic View: “Expansion”
(1-)M K=2k
D=2d
N=2n
M=2m
Can use this to constructexpanders beatingeigenvalue bound [WZ]
x y Ext(x,y)
output uniform
K-Expanding Graphs
K
N
K
|A|≥K |Γ(A)|>N-K
Goal: minimize degree DD>N/K
Random graphs:D=O((N/K) log (N/K))
2nd Eigenvalue: D≥(N/K)2/2
Extractors: D=N1+o(1)/K [Wigderson-Z ‘93]
Useful for sorting, networks
Extractors K-Expanding Graphs
(1-)M K
N
M
(1-)M
KK-Expanding Graph:V=[N]E=Paths of length 2 in Ext
Alternate View
S
BADS
D=2d
N=2n M=2m
x
Other direction:ErrorS ≤ |BADS|2-k + ε
Averaging Sampler via Alternate View [Z ‘96]
• Goal: Estimate mean μ of– Black box access to f.
Algorithm: Pick x randomly in {0,1}n. Sample f at Γ(x) = {x1,…,xD}.
Output μf.
Pr[error] = |BADf|/2n.
Can use 1.01m random bits with Pr[error]=2-Ω(m).
Extractor Perspective Helps
• Proposition: Sampler using O(m) random bits implies sampler using 1.01m random bits.
• Equivalent Statement: Extractor outputting Ω(k) bits implies extractor outputting .99k bits.
• Ext(x,(y1,y2)) = Ext(x,y1)Ext(x,y2) [Wigderson-Z]– Conditioned on Ext(X,y1) of length m, still ≈k-m bits
of entropy in X.
Extractor Codes via Alt-View[Ta-Shma-Z 2001]
• • List recovery – generalizes list decoding.
S=(S1,…,SD), agreement = |{i|xi in Si}|
|{Codewords with agreement ≥(μ(S) + ε)D}|≤ |BADS|.
Extractor codes with efficient decoding give hardcore bits Ext(x,y) wrt 1-way (f(x),y).
Codes Extractors [Tre,TZS, SU, GUV].
Max Clique and Chromatic Number• [FGLSS,…,Hastad]: Max Clique
inapproximable to n1-, any >0, assuming NP ZPP.
• [LY,…,FK]: Same for Chromatic Number.
• Derandomize with linear degree extractors:Thm [Z]: Both inapproximable to n1-, any >0,
assuming NP P.
Constructions of Strong ExtractorsRestrictions Degree
D=2dOutput Length m
Existence None (n-k)/ε2 k – 2lg(1/ε)
Leftover Hash Lemma [ILL]
None 2n k – 2lg(1/ε)
GUV 2007 None (n/ε)O(1) (1-α)k
GUV 2007 None nO(log(k/ε)) k – 2lg(1/ε)-O(1)
DKSS 2009 ε≥1/logcn nO(1) (1-1/logcn)k
Z 2006 k=Ω(n)ε=Ω(1)
O(n) (1-α)k
Pseudorandom Generators
• Cryptographically secure PRGs:– Run in time less than adversary.– Exist iff one-way functions exist [HILL].
• PRGs for derandomization:– Can take slightly more time than adversary.– Exist iff “hard” functions exist [Nisan-Wigderson ...]
PRGpseudorandomrandom seed
PRGs from Hard Functions[Nisan-Wigderson 1988 …]
PRGcomp. error εrandom seed
hard function
NW-Style PRGs Give Extractors[Trevisan 1999]
• View x as hard function f:{0,1}lg n {0,1}– Most functions hard
• Set Ext(x,y) = NW-PRG(f,y)• Better: Ext(x,y) = NW-PRG(Code(f),y)
Ext n bits
statistical error
seed
Linear Degree Extractor [Z] (Sketch)
Condense:
Extract:
.9
uniform
+ lg n+O(1) random bits
+ O(1) random bits
Condensing via Incidence Graph
• 1-Bit Somewhere Condenser:– Input: edge– Output: random endpoint
• Condenses rate to rate (1+), some > 0.• Proof uses bound on incidences [BKT]+ probabilistic lemma.• Combine with technique of [Raz] to get actual condenser.
linespoints = Fq
2
L
P (L,P) an edge iff P on L
|P|3/2 edges
High Entropy Extractor
• Chernoff bound for random walks on expanders [Gillman,Kahale]
• Implies Sampler• Implies Extractor.
Seeded Extractor Techniques/History
• Hashing based: Z ’90-91, Nisan-Z ‘93, Wigderson-Z ‘93, Srinivasan-Z ’94, Z ‘96, Ta-Shma ‘96, Raz-Reingold-Vadhan ‘99, Reingold-Shaltiel-Wigderson ‘00,
• NW-PRG based: Trevisan ’99, Raz-Reingold-Vadhan ‘99, Impagliazzo-Shaltiel-Wigderson ‘99-00, Ta-Shma-Umans-Z ‘01
• Algebraic/coding theory based: Ta-Shma-Z-Safra ’01, Shaltiel-Umans ‘01, Lu-Reingold-Vadhan-Wigderson ‘03, Gurusmami-Umans-Vadhan ‘07, Ta-Shma-Umans ’12
• Additive combinatorics based: Barak-Kindler-Shaltiel-Sudakov-Wigderson ’05, Raz ‘05, Z ’07, Dvir-Wigderson ‘08, Dvir-Kopparty-Sharaf-Sudan ‘09
Seedless (Deterministic) Extractors for Structured Sources
• Probabilistic Method: If ≤ sources of min-entropy k:
Can deterministically extract m=(1-α)k bits with error 2-αk/3.
• Algebraic sources:– Bit-fixing, affine.
• Independent sources.• Complexity-theoretic sources:
– AC0 sources, small-space sources.
Oblivious Bit-Fixing Sources
• Example: ?0010?111??11.– ? = uniform on {0,1}.– (n-k) bits fixed by adversary; k uniform bits.– Parity extracts 1 bit.
• For k≥logc n, can extract k-o(k) bits [GRS, Rao].• Application: Exposure Resilient Cryptography.
– Adversary learns many bits of secret key.– Can still do cryptography.
Affine Extractors
• X = random element from affine subspace.• Generalizes bit-fixing sources.• Extractor for min-entropy αn, any α>0
[Bourgain].• 1-bit disperser for min-entropy exp(log.9 n)
[Shaltiel].• Large fields: any k>0 [Gabizon-Raz].
Independent Sources
n bits n bits
Ext
m =Ω(k) bits statistical error
Classical: entropy rate > 1/2
• Lindsey Lemma: H∞ (X) + H∞ (Y) > n+t implies
X.Y ≈ U, error 2-t/2.
Independent Sources# sources k=H∞(X) Restrictions
Existence 2 k ≥ 2log n None
Bourgain 2 k ≥ .499n None
BRSW 2 k ≥ nα Disperser
Li 3 k ≥ n1/2+α None
Rao-Z 3 k ≥ nα Uneven lengths
Li O(1) k ≥ log3 n None
Cryptography with Weak Sources
• Players have independent weak sources.• Allow Byzantine faults.• For 2 players, impossible [DOPS].• For more players, possible!
Network Extractor Protocol [Goldwasser-Sudan-Vaikunthanatan05, Dodis-
Oliveira03]
010101010
01001011011011
11010
100100101
10100
010100101
10110
011110101
11001
01010101
01001
001010101
01001
010111101
10101
Input: x1,…,xp 2 {0,1}n from independent weak random sources
Output: z1,…,zp 2 {0,1}m private nearly-uniformrandom strings (for honest parties)
Byzantine faults:can send arbitrary messages
Network Extractor Protocols
• After running network extractor protocol, run standard protocol, e.g., Byzantine Agreement.
• Naïve idea to design protocol:– A few players broadcast sources.– Remaining players apply independent-source
extractor to those sources and own source.– Problem: what if only malicious players
broadcast?
Network Extractor Constructions
• Information-theoretic setting [Kalai-Li-Rao-Z]:– For k ≥ exp(logα n), can still tolerate linear number
of faults in BA and leader election, any α>0.• Computational setting [Kalai-Li-Rao]:
– Under certain crypto assumptions, for k = αn, secure multiparty computation if ≥ 2 honest players.
– Under certain crypto assumptions, 2-source extractors for k = αn, any α>0.
Complexity-Theoretic Sources
• X=f(U), complexity(f) small.• Deterministic extraction possible under
assumptions [Trevisan-Vadhan ‘00].• No assumptions:
– NC0 [De-Watson ‘11, Viola ‘11]– AC0 [Viola ‘11]– Proofs reduce to low-weight affine extractors [Rao
‘09].
Small Space Sources• Space s source: min-entropy k source
generated by width 2s branching program.
n+1 layers
1 1 0 1 0 0
1/, 0
1-1/, 0 1,10.1,0
0.8,1
0.1,0
0.3,0
0.5,10.1,1
0.1,0
1
width 2s
Bit Fixing Sources can be modelled by Space 0 sources
? 1 ? ? 0 1
0.5,1 0.5,1 0.5,1
0.5,0 0.5,0 0.5,0
1,1 1,0 1,1
Extractors for Small Space Sources
• For k ≥ αn, any α>0, space αβn, β>0 sufficiently small, can extract k-o(k) bits [Kamp-Rao-Vadhan-Z ‘06].
• Proof reduces to variants of independent sources by conditioning on intermediate states.
Crypto-Tailored Extractors
• Fuzzy extractors– Noise tolerant [Dodis-Ostrovsky-Reyzin-Smith ‘04]
• Correlation extractors– [Ishai-Kushilevitz-Ostrovsky-Sahai ‘09].
• Non-malleable extractors [Dodis-Wichs ’09]
Privacy Amplification With Active Adversary
• Problem: Active adversary could change Y to Y’.
public
Pick Y
Shared secret = Ext(X,Y).
Active Adversary
• Can arbitrarily insert, delete, modify, and reorder messages.
• E.g., can run several rounds with one party before resuming execution with other party.
Non-Malleable Extractor[Dodis-Wichs 2009]
• Strong extractor: (Ext(X,Y),Y) ≈ (U,Y).• nmExt is a non-malleable extractor if for arbitrary
A:{0,1}d {0,1}d with y’ = A(y) ≠ y.(nmExt(X,Y),nmExt(X,Y’),Y) ≈ (U,nmExt(X,Y’),Y)
• Can’t ignore a bit of the seed.• Existence: k > log log n + c, d = log n + O(1),
m = (k-log d)/2.01.• Gives privacy amplification with active adversary in
2 rounds with optimal entropy loss.
Explicit Non-Malleable Extractor
• Even k=n-1, m=1 nontrivial.– E.g., Ext(x,y) = x.y. X=0??...?, y’=A(y) flips first bit,
x.y’= x.y.
• Dodis-Li-Wooley-Z 2011: H∞ (X) > n/2.• Cohen-Raz-Segev 2012: Seed length O(log n).• Li 2012: H∞ (X) > .499n.
– Connection with 2-source extractors.
A Simple 1-Bit Construction [Li]
• Sidon set: set S with all s+t, s,t in S, distinct.• Example: S={(x,x3)|x in F2n/2}.
• Thm [Li]: f(x,y) = x.y, y uniform from S, nonmalleable extractor for H∞ (X) > n/2.
• Proof: H∞ (Y) = n/2, so X.Y ≈ U (Lindsey’s lemma).
• Suffices to show X.Y+X.A(Y) ≈ U (XOR lemma).• X.Y+X.A(Y) = X.(Y+A(Y)). • H∞ (Y+A(Y)) = H∞ (Y) = n/2.
Conclusions
• Interesting mathematics used in constructions: additive combinatorics, coding theory, random walks on expander graphs, hashing, …
Crypto
Expanders Coding Theory
Extractors
PRGs Inapproximability
Open Questions
• Seeded Extractors– O(n) degree for all min-entropy.– O(log n) seed to extract k - 2log(1/ε) – O(1).
• Seedless Extractors– 2-source extractors for min-entropy αn, any α>0. – Affine extractors for min-entropy nα.– Other general models.
• Crypto-Tailored Extractors– Non-malleable extractors for min-entropy αn.
• Other Applications & Connections.
Top Related