Influence and Noise
description
Transcript of Influence and Noise
1
Influence and Noise
Gil KalaiHebrew University of Jerusalem,
Yale UniversityMicrosoft R&D, Israel
ICS2011, Beijing January 2011
2
Part I: Influence
3
Cause
When does event A cause event B?
Example (Kira Radinsky’s automatic system for making deductions based on internet searches):
Earthquake causes TsunamiCabbage grew causes Linux release
4
Causality
When does event A cause event B? Central problem in philosophy, law,
economics, physics, statistics, CS… (Examples are often somber) Two people try to assassin a third person
who plan a trip to the desert. One puts poison in his jar, the other empties it.
A person throws a baby from a tall building, another is waiting with a sharp sword.
5
Influence
The word “influence” (dating back, according to Merriam-Webster dictionary, to the 14th century) is close to the word “fluid”. The original definition of influence is: “an ethereal fluid held to flow from the stars and to affect the actions of humans.” The modern meaning (according to Wictionary) is: ”The power to affect, control or manipulate something or someone.”
6
Influence
What is the influence an event A has on another event B?
Can be regarded as an approach to causality and also as a generalization.
7
Influence
(Of a variable on a function of many variables) The “amount” that changing the value of a variable will change the value of the function.
8
Boolean Functions
We consider a BOOLEAN FUNCTION f :{-1,1}n {-1,1} f(x1 ,x2,...,xn)
It is convenient to regard {-1,1}n as a probability space with the uniform probability distribution.
9
Influence
We consider a BOOLEAN FUNCTION f :{-1,1}n {-1,1}
The influence of the kth variable xk on f, denoted by Ik(f) is the probability that flipping the value of the kth variable will flip the value of f.
I(f) is the sum of all individual influences.
10
Examples
1) Dictatorship f(x1 ,x2,...,xn) =x1
Ik(f) = 0 for k>1 I1(f)=1
2) Majority f(x1 ,x2,...,xn) =1
iff x1 + x2+...+xn > 0
Ik(f) behaves like n-1/2 for every k.
11
Critical Percolation
12
Examples (cont.)
3) The crossing event for percolations For percolation, every hexagon corresponds
to a variable. xi =-1 if the hexagon is white and xi =1 if it is grey. f=1 if there is a left to right grey crossing.
Ik(f) behaves like n-3/8 for every k but few.
13
Examples (cont.)
4) Recursive majority of threes Ik(f) behaves like n-log3 for every k.
5) Ben-Or Linial TRIBE example
Divide the variables to tribes of size logn-loglogn+loglog2
f=1 iff for some tribe all variables equal to 1Ik(f) behaves like Klogn/n for some constant K.
14
KKL theorem: There always exists an influential variable
Theorem (Kahn, Kalai, Linial 1988)Let f be a Boolean function and suppose that
Prob(f=1)=sThen there is a variable k such that
Ik(f) > C s (1-s) log n / n
This result was a conjecture by Ben-Or and Linial
15
KKL theorem
16
Fourier
Given a Boolean function f :{-1,1}n {-1,1}, the Fourier expansion of f is simply writing f(x) as a sum of multilinear (square free) monomials.
17
Fourier SpectrumFor every set S of variables we have the associated Fourier coefficient. The sum of squares of Fourier coefficients is 1.This defines a probability distribution called the “Fourier spectrum” (or “Fourier distribution”). The probability that k belongs to S, when S is distributed according to the Fourier spectrum is the influence of variable k on f.
18
Fourier
The study of Boolean functions based on their Fourier expansion is fruitful.
It can be regarded as a very special case of spectral methods in graph theory.
19
HypercontractivityA very useful technical tool:
The ratio between p-norms of low degree polynomials is bounded.
(Khintchine , Nelson, Bonami, Gross, Beckner… )
20
A glance at further advances and open problems
Extensions to general Bernoulli product spaces.Notions of influence for continuous product spaces; larger alphabets; graph products.Symmetry and influenceUnderstanding influence of sets of variables; Power-laws for influence
21
Part II
Choice, power, rationality and manipulation
22
Individual and collective rationality and judgments
Boolean functions can model how individual preferences between two alternatives aggregate.
We can consider aggregation of individual preferences between m alternatives.(Social welfare functions.)
We can consider aggregation of judgments on r different binary questions, when there are certain consistency requirements. (Judgment aggregation.)
23
Elections: measures of power
Influence= Banzhaf power index
Shapley-Shubik power index:
Integral over p of the influence of the kth player with respect to the Bernoulli probability with parameter p.(Sum to 1.)
24
Voting Paradoxes
Condorcet’s paradox:Arrow’s paradox:“Doctrinal paradox”:
25
Manipulation
A social choice function is a function from the profile of individual order relations to the set of alternatives.
Manipulation: reporting an incorrect preference relation will improve the outcome.
26
A measure for manipulation
The Gibbard-Satterthwaite theorem asserts for a non dictatorial choice function with at least 3 possible outcomes there are preferences that leads to manipulation.
The manipulation power (Friedgut, Kalai and Nisan) of an individual k for a social choice function f, denoted by Mk(f) is the probability that x’k is a profitable manipulation for voter k when the profile of preferences x1 x2 ,…, xn and x’k are chosen uniformly at random.
27
Conclusion of the Algorithmic GT/Econ Part
The notions discussed in this lecture (measures for influence, power, manipulation, noise sensitivity…) may be of interest to other GT/econ models. For example, the model of exchange economy.
Off-topic comment: Why is it rational and important to give incentives to difficult technical works. Added in proof: There is a work supporting this thought by Kleinberg and Oren.
28
Part III
Randomness
29
Collective coin flipping We need to create a random bit using a
protocol based on random bits contributed by n processors. Some of the processors are malicious.
A simple suggestion: Choice based on a Boolean function where each processor contributes a single bit.
(An often asked question: why not choose among these bits at random?)
30
Randomness as a computational resource
“So, yes, I know that the theory folks consider derandomization an open problem, but from my perspective, it is a solved problem for all practical purposes.” AnonCSProf on Shtetl-Optimized
31
What is Randomness? What is randomness and what is
probability are fundamental question in many areas. Computational complexity offers a deep understanding to randomness. Its asymptotic nature makes it of little use in statistics.
Q: What allows much simpler answers in statistics?
A: The interactive proof system known as ``statistical hypothesis testing''.
32
What is the source of randomness?
Is human uncertainty the only source of randomness?
What is the explanation of the apparent randomness of high-level phenomena in nature? For example the distribution of females vs. males in a population (I am referring to randomness in terms of the unpredictability and not in the sense of it necessarily having to be evenly distributed). 1. Is it accepted that these phenomena are not really random, meaning that given enough information one could predict it? If so isn't that the case for all random phenomena? 2. If there is true randomness and the outcome cannot be predicted - what is the origin of that randomness? (is it a result of the randomness in the micro world - quantum phenomena etc...)
33
Sharp threshold phenomena: Determinism
from randomness Law of large numbers: Large stochastic
systems behave deterministically.
Sharp threshold phenomena: Choose the value of the variables to be 1 with probability p, independently. The value of f will rapidly change from 0 to 1.
34
Sharp threshold phenomena:
Influence is a sort of derivative. Large total influence corresponds to small threshold interval.
Theorem (Friedgut Kalai, 96): Symmetry implies sharp threshold
Theorem (Kalai, 04): Sharp threshold is equivalent to diminishing maximum Shapley-Shubik power index.
The economics term: complete aggregation
of information.
35
Influence without independence
(Haggstrom, Kalai, Mossel; Graham, Grimmett.)
Influence version A: The probability that changing the value of a variable will change the value of the function.
Influence version B: The (normalized) correlation between the value of the variable and the value of the function.
36
Part IV
Noise
37
Noise Sensitivity
We consider a BOOLEAN FUNCTION
f :{-1,1}n {-1,1}f(x1 ,x2,...,xn)
Given x1 ,x2,...,xn we define y1 ,y2,...,yn as follows:
xi = yi with probability 1-t
xi = -yi with probability t
38
Noise Sensitivity
Let C(f;t) be the correlation between f(x1 , x2,...,xn) and f(y1,y2,...,yn)
A sequence of Boolean function (fn ) is noise-sensitive if for every t>0, C(fn,t) tends to zero with n.
39
Noise Stability
Lets fix a small s=0.0001(say).
A Boolean function is noise stable to noise level t if the probability that
f(x1 , x2,...,xn) is different from f(y1,y2,...,yn) is smaller than s.
40
Noise sensitivity, and non-classical stochastic
processes; black noise
Closely related notions to “noise sensitivity” were studied by Tsirelson and Vershik . In their terminology “noise sensitivity” translates to “non Fock processes”, “black noise”, and “non-classical stochastic processes”. Their motivation is closer to mathematical quantum physics.
41
BKS theorem
Theorem (Benjamini, Kalai, Schramm 1999)
Monotone balanced Boolean functions are noise sensitive unless they have substantial correlation with some weighted majority functions.
42
Percolation is Noise sensitive
Corollary [BKS, 1999]: The crossing event for critical planar percolation model is noise- sensitive
Theorem (Schramm and Smirnov, 2010): Percolation is a 2-dimensional black noise.
43
Percolation is Noise sensitive
Imagine two separate pictures of n by n hexagonal models for percolation. A hexagon is grey with probability ½. If the grey and white hexagons are independent in the two pictures the probability for crossing in both is ¼.If for each hexagon the correlation between its colors in the two pictures is 0.99, still the probability for crossing in both pictures is very close to ¼ as n grows! If you put one drawing on top of the other you will hardly notice a difference!
44
Other cases of noise sensitivity
First Passage Percolation (Benjamini, Kalai, Schramm)
The recursive Majority on threes example by Ben-Or and Linial (BKS)
Eigenvalues of random Gaussian matrices (Essentially follows from the work of Tracy-Widom) Here, we leave the Boolean setting.
Examples related to random walks (required replacing the discrete cube by trees) and more...
45
Majority is noise stable
Sheppard Theorem: (1899): Suppose that there is a probability t for a mistake in counting each vote.
The probability that the outcome of the election are reversed is: arccos(1-t)/π +o(1)
When t is small this behaves like t1/2
46
Majority is noise stable (cont.)
Weighted majority functions are also noise stable (BKS, Peres)
Is there a more stable voting rule? Sure! dictatorship
47
Majority is stablest! Theorem: Mossel, O’Donnell and
Oleszkiewicz : (2005)Let (fn ) be a sequence of Boolean functions
with diminishing maximum influence. I.e., limn->∞ maxk Ik(fn) -> 0
Then the probability that the outcome of the election are reversed when for every vote there is a probability t it is flipped is at least
(1-o(1)) arccos(1-t)/π
48
Majority is stablest: Two applications
1. The probabilities of cyclic outcomes for voting rules with diminishing influences are minimized for the majority voting rule.
2. Improving the Goemans-Williamson 0.878567 approximation algorithm is hard, unique-game-hard.
49
Is the universe noise sensitive?
Are the basic models of high energy physics noise stable?
If this is indeed the case, does it reflect some law of physics? Otherwise, will noise sensitivity allow additional modeling power?
50
Part V: Computational complexity
Are the notions discussed here related to computational complexity? (We already mentioned relation to PCP; there are some interesting connection to randomized decision trees.)
Diversion: CC and modeling
52
Computational complexity and Influence
(… Ajtai, Furst, Saxe, Sipser, Yao, Hastad, Boppana, Linial,Mansour,Nisan…)
The total influence of Boolean functions
that can be described by depth D size M Boolean circuits is at most
log MD-1
53
Computational complexity and Noise stability
Conjecture 1: Let f be a monotone Boolean function described by monotone threshold circuits of size M and depth D. Then f is stable to (1/t)-noise where
t=log M100D
Conjecture 2: For some η >0, every balanced monotone functions in TC0 have correlation at least η with a function in monotone TC0.
54
Complexity of sampling the Fourier spectrum
Suppose that f is a Boolean function in P. Can we approximately sample according to its Fourier spectrum?
This is unknown and it might be hard.
But... It is in BQP. (Namely, it is known to be easy for quantum computers. )
55
Computation with noise Fault tolerant (quantum) computation.
Are quantum computers possible? (This is a main research interest for me in recent years.)
The hope regarding FTQC: No matter what the quantum computer computes or
simulates, nearly all of the noise will be a mixture of states that are not codewords in the error correcting code, but which are correctable to states in the code.
The concern: The process for creating a quantum error correcting
code will necessarily lead to a mixture of the desired codeword with undesired codewords.
56
Polymath3A recent endeavor: you are most welcome to join
Details can be found on my blog http://gilkalai.wordpress.com/2010/02/10/noise-stability-and-threshold-circuits/
1) The AC0 analog 2) Positivity Vs Monotonicity 3) Natural Proof Obstructions
57
谢谢Thank you!
谢谢
רבה !תודה
It is great to be in Beijing!