Phase transition in Discrete Mathematics, Randomized and...

Selecting the topic Why does the randomization help? Approximating the volume Finding the diameter Derandomization

Phase transition in Discrete Mathematics,Randomized and Deterministic algorithms

Miklós Simonovits

Rényi Institute

Fritz Jóska ,2013 május 23


Friends


Contents

„Phase transition” in Discrete Mathematics

The meaning of phase transition „here”

Random walks, the Polya story,

Random objects investigated for their own sake

using the ergodicity


Selecting the topic

Joska was always interested in Physics and AnalysisJóska started as a PhysisistHaving finished the university he started working on

Rudemo Entropy (?)Rényi, A.

On the dimension and entropy of probability distributions. ActaMath. Acad. Sci. Hungar. 10 1959 193–215

Rudemo, Mats

Dimension and entropy for a class of stochastic processes.(Russian summary) Magyar Tud. Akad. Mat. Kutató Int. Közl. 91964 73–88.On the dimension and entropy of probability distributions. ActaMath. Acad. Sci. Hungar. 10 1959 193–215


Happy Birthday, Jóska

Fritz, J.

Entropy of point processes. Studia Sci. Math. Hungar. 4 1969389–399.

Fritz, J.

An approach to the entropy of point processes. Collection of ar-ticles dedicated to the memory of Alfréd Rényi, II. Period. Math.Hungar. 3 (1973), 73–83.


Happy Birthday, Jóska


Meeting Jóska

Reiman, Olypiad, playing the piano


Physics

Both Jóska and I loved Physics very much in theHighschool.

Jóska started as a physisist, however, after a year hedecided that he will understand physics much better as amathematician.

Brave step „downgrading”


Rényi and Jóska

Jóska spent a long period of his time, in the Rényi Institute.Rényi invited him to the institute, Rényi gave him the firstresearch topic at the Institute


Phase transition for Pastur


Phase transition for us

Sudden changes in the behaviour of a system

TheoremKomlós-Szemerédi Iw we throw down edges at random, assoon as the min degree reaches 2, the graph will beHamiltonian.

The meaning of this theorem


Using randomized constructions

Theorem (Erdos: Ramsey, log n)Let c = 3. Most of the graphs on n vertices do not have c log ncomplete subgraphs,neither (induced) c log n-empty graphs.

Theorem (Shannon: typical codewords)≈ If the chanel has capacity c, then one can send over a littleless information with high security but if one tries to send overmore information, that will be lost. random codewords

Theorem (Pinsker: Expander)There are graphs with linear edge density having the expanderproperty


The three approaches:

• Erdos-Rényi:On the strength of connectednessasymmetry of graphs

• Gilbert

• Ehud Friedgut: sharp thresholdOn what does it depend if we have a slow or a

sudden phase-transition?


Supersaturated graphs

.

.......


Universal path

Aleliunas, Karp, Lipton, Lovász, Rackoff,Random walks, universal traversal sequences, and the complex-ity of maze problems. 20th Annual Symposium on Foundationsof Computer Science, pp. 218–223, IEEE, New York, 1979.

From the introduction: "It is well known that the reachability prob-lem for directed graphs is logspace-complete for the complex-ity class NSPACE(logn), and thus holds the key to the openquestion of whether DSPACE(logn)=NSPACE(logn). Here asusual DSPACE(logn) is the class of languages that are ac-cepted in logn space by deterministic Turing machines, whileNSPACE(logn) is the class of languages that are accepted inlogn space by nondeterministic ones. . . .

The reachability problem for undirected graphs has also beenconsidered [N. D. Jones, Y. E. Lien and W. T. Laasev, Math.Systems Theory 10 (1976), no. 1, 1â17; MR0443429 (56


QuickSort

Wikipedia, shortened:

Quicksort is a sorting algorithm. On average, it makes O(n log n)comparisons to sort n items. It is also known as partition-exchange sort. In the worst case, it makes O(n2) comparisons,though this behavior is rare.Quicksort is often faster in practice than other O(n log n) al-gorithms .


Finding the median

Seems to be trivial, but highly non-trivialThe randomized algorithm is „completely trivial”

• Take a random subset of n2/3 elements

• Sort them

• Choose two of them defining an interval containing themedian „very probably”

• compare each element with them: expected number ofcomparisions is ≈ 3

2n + o(n).

• find the median


What is the problem?

We wish to calculate the volume of high-dimensionalbodies efficiently.

Efficiently = in polynomially many steps.

Is it possible deterministically?

Is it possible by randomized algorithms?


The problem of estimating the volume

Given ε > 0,Find an algorithm which outputs a ζ with

(1 − ε)ζ < vol (K ) < (1 + ε)ζ

Includes several combinatorial problems

E.g. full extensions of partial orderings

No exact solution is possible

Khachiyan: even to write down the result takes exponentialtime.

Permanent problem = counting the 1-factors in a bipartite graph

= hardest problem (Valiant) / This was the first case which wasapproximated by randomized a algorithm


Does the randomization really help?

If P = NP (???) then not as we would like to have it, however,then we do not need this „help”

OtherwiseDyer, M.; Frieze, A.: Computing the volume of convex bodies: acase where randomness provably helps, (1991)

It does help in the volume-estimation forhigh dimensional convex bodies given by oracles

Important:

High dimensionalconvexgiven by oracles


What is an oracle?

„Black box:” We ask questions and it provides some answersabout K

The oracles vary according to which questions can be askedand what types of answers do we get


The model: Oracle

NO / YES

x

SEPARATION ORACLE

Convex body ⇔ separation oracleAnswer: x ∈ K ,x 6∈ K , and a separating plane


Why to use oracles?

Several arguments for it:

There are combinatorial problems where the attached convexbody can be described only by oracles.Matching Polytope, . . . After Khachiyan

Object oriented programming

Mathematical Logic: P = NP? can be asked in various modelswith oracles

In some of them this holds, in others it does not.


What types of oracles?

Weak separation oracleStrong separation oracleWeak Membership oracleStrong Membership oracle. . . Maximizing oracle

They are equivalent in some sense . . .

error!with ε

x

NO / YES

Safe outside

Uncertain cases

Safe inside

M. Grötschel, L. Lovász and A. Schrijver:Geometric Algorithms and Combinatorial Optimization,Springer-Verlag.


Which Oracle: Weak/Strong membership

NO / YES

MEMBERSHIP ORACLE


Weak/Strong separation oracle

x

with error!εNO / YESStrong: no errors

Weak:Within an ε distance from ∂Kit may make errors


Well guaranteed oracle

��

��

WELL GUARANTEED BODY

We need the guarantee, otherwise could answer „NO” adinfinity.


Why not Monte Carlo?

THE INSCRIBED BALL IS TOO SMALL

The ball inscribed into the n-dimensional cube is exponentiallysmall:

vol (Balln)vol (Cuben)

< qn for some 0 < c < 1

We need exponentially many points to get just one HIT.


Approximating the volume

Multiphase Monte Carlo Markov Chain

Andrei Broder used a randomized algorithm to approxi-mate the number of 1-factors – turns out to be incorrect

Jerrum-Sinclair: Approximating the number of 1-factors ina bipartite graph

1. Dyer-Kannan-Frieze O∗(n26)

2. Lovász-Sim. O∗(n16)

3. Applegate-Kannan O∗(n10)

4. Dyer-Frieze O∗(n8)

5. Kannan-Lovász-Simonovits O∗(n5)

6. Lovász-Vempala: O∗(n4)


Related topics:

Sampling

A Frieze, R Kannan, N Polson - The Annals of Applied Probabil-ity, 1994 - Sampling from log-concave distributions

A function is log-concave if its logarithm is concave.The important probability distributions are log-concave. e.g.

f (x) = c exp(xAx)

Integration

of log-concave functions


Negative results

1. Elekes

2. Bárány-Füredi

3. Khachiyan


Is the diameter of the same difficulty?

No!Volume can be approximated,Diameter cannot be approximated even if we can use random-ization


How to estimate the volume of a high dimensional convex body?

Based on:Kannan, Lovász, Simonovits:Random walks and an O∗(n5) volume algorithm, RandomStructures and Algorithms 1 (1997), 1–50.


MAIN THEOREM

Given a convex K ⊆ Rn, by a weak separation oracle. There

exists a randomized algorithm which for arbitrary ε, η > 0,returns a ζ > 0-t, such that

(1 − ε)ζ < vol (K ) < (1 + ε)ζ

with probability ≥ 1 − η.


MAIN THEOREM

Given a convex K ⊆ Rn, by a weak separation oracle. There

exists a randomized algorithm which for arbitrary ε, η > 0,returns a ζ > 0-t, such that

(1 − ε)ζ < vol (K ) < (1 + ε)ζ

with probability ≥ 1 − η. The algorithm uses

O(

n5

ε2 (ln1ε)3(ln

1η) ln5 n

)

= O∗(n5)

oracle queries.



The task is:

• estimate/approximate the volume of a high dimensionalconvex body K ⊆ R

n

• generate uniform distribution on K

• Integrate on K



The task is:

• estimate/approximate the volume of a high dimensionalconvex body K ⊆ R

n

• generate uniform distribution on K

• Integrate on K

The method of fine division does not work in high dimension!Neither does the (simple) Monte Carlo!

The above three questions are equivalent (in some sense), [∫

ffor some functions]


What is wrong with the fine subdivison?

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

As the dimension increases, we need too many points!


What is Monte Carlo?

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

We can generate uniform distribution on a high dimensionalbox. Then we count, how many of them are in our domain.


Brute force: Löwner-John Ellipsoid

If we consider the ellipsoid Eof smallest volume containing Kand shrink it by a factor 1

n :

1nE ⊆ K ⊆ E

This yields an estimate, of relative error nn.But is it algorithmic?


Elekes Theorem

If an algorithm approximates the volume for every K within afactor C=constant, then it uses exponentially many questions

If it uses P = polynomially many questions then in the worstcase it makes

2n/2√

P

relative error.


Does randomization help?

Prehistory:Counting 1-factors in (dense) random graphsA. Broder proposed to use:

• random walks,

• multiphase Monte Carlo

• Product estimator

Jerrum-Sinclair: Yes, for this problem it helps!

ρ(P,Pi) ≤ ρ(P,P0) ·(

1 − Φ2

2

)i

.


How to measure the speed?

Book: Grötschel, Lovász and Schrijver (1988)Real, exact arithmetics

• The number of oracle questions

• Random bits

• Arithmetic steps

The error-probability, η, is unimportant:Jerrum, Valiant and Vazirani (1986): log 1

η


Two basic phases:

Rounding = sandwiching

WE DO NOT LIKE THE PENCILS!

The volume of the rounded body: product estimator

vol (K ) = vol (K0) ·m∏

i=1

vol (Ki)

vol (Ki−1).


Outline: Rounding, Estimating

K, given by a separation Oracle

Bringing into isotropic position

Estimating the volumein isotropic position

truncationsratios of "subsequent"

Approximate the volume

The lazy walk

Analyzing the speedy walk

Sampling forisotropy


Rounding?

B(0, 1) ⊆ K ⊆ B(0, nk ).

B(0, n√

n) is achievable by ellipsoid method in O∗(n4) steps.

Approximate Rounding


What is the multiphase Monte Carlo?

Ki := K ∩(

1 +1n

)i

B(0, 1)


What to estimate?

The volume-ratio of the conse-cutive truncated convex bodies!

1 ≤ vol (Ki)

vol (Ki−1)≤ 3.

vol (K0) is known,

vol (K ) = vol (K0) ·m∏

i=1

vol (Ki)

vol (Ki−1).


What is a rounded Convex body?

��

��

��

��

ROUNDING = SANDWICHING

We wish to find a linear transformation A such that the radii ofthe circumscribed AK and inscribed balls is small: nc , perhapsn, perhaps

√n?


How to generate uniform distribution?

• Use a random walk (Markov chain) whose stationarydistribution is uniform

• Converges to its stationary distribution fast:

Rapid mixing(= in polynomial time in n):

ρ(P,Pi) ≤ ρ(P,P0) ·(

1 − Φ2

2

)i

.

Φ = conductance should not be too small, ≥ 1nc


Rapid mixing and isoperimetric inequalities

For convex bodies there are no bottlenecks: isoperimetricinequalities

We developed a tool: Localization Lemma to proveisoperimetric inequalities


Rapid mixing and uniform distribution?

FailureStart

Failure 1 2

34

07

11End

RAPID MIXING PRODUCES THE STATIONARY DISTRIBUTION

Questions: Is that uniform?And do we get it fast?


Random walk on the truncated grid

RANDOM WALK ON GRID


Ball-walk

BALL WALK

(a) Choose a random point in the ball x i + δB(0, 1).(b) Try to jump there: If that is outside of K , stay put.(c) Lazy walk / speedy walk


Def: Isotropic position

Let K ⊆ Rn,

b(K ) = baricenter.K is in isotropic position if

b(K ) = 0 and for every i , j , 1 ≤ i ≤ j ≤ n,

1vol (K )

∫

Kxixj =

{

1, if i = j ,

0, if i 6= j ,

Milman and Pajor (1987)Then

1vol (K )

∫

K‖x‖2 dx = n,

The majority of ⇒ K is in B(0,√

nε ).


Def: Near-isotropic position

K ϑ-near isotropic, (0 < ϑ ≤ 1), if

‖b(K )‖ ≤ ϑ,

and for every v ∈ Rn,

(1 − ϑ)‖v‖2 ≤ 1vol (K )

∫

K−b(K )(vTx)2 dx ≤ (1 + ϑ)‖v‖2.


Bringing into isotropic position

Given: 0 < η, ϑ < 1,There exists a randomized algorithm finding an affine A forwhich AK is ϑ-near-isotropic with probability ≥ 1 − η.# Oracle queries:

O(n5 ln(ϑη) ln n).


Continuation

=⇒ with probability ≥ 1 − η ,

vol (AK \ 2√

2n log(1/ε)B) < εvol (AK ).

# Oracle queries

O(

n5 ln1η

ln n)

.


What was hidden?

• Lazy random walk

• Speedy random walk

• Dangerous corners

• Conductance, Spectral gap

• average conductance

• Further technical difficulties

• Improved partial results

Last Breakthrough


Dangerous Corners

K

x

x’

x"

It may happen that we get into a corner and it takesexponentially many steps to get out of it. Fortunately, this ishighly unprobable.


Improvements

Lovász-Vempala: There exists a randomized algorithm . . . using

O∗(n4)

oracle queries.

Lovász-Vempala: Simulated Annealing in Convex Bodiesand an O∗(n4) Volume Algorithm, 2003


Diameter

Bound for polynomially many querries

Relative error√

nlog n

Boxing: deterministic, with relative error√

nBoxing into random direction: relative error

√

n/ log nderandomization


Hit-and-Run

x

Seems to be fast in practice.


What about the diameter?

The diameter cannot be approximated, not even by randomizedalgorithms

ROTATED DOUBLE CONE

the two body cannot be distinguished by polynomially manyqueries.


The positive result

There exist a deterministic poly algorithm for which: If K ⊆ Rn

is given by a convex, (well guaranteed) then the algorithmyields a ζ(K ) for which

ζ(K ) ≤ diam (K ) ≤ c√

nlog n

· ζ(K ).


Bárány and Füredi

To approximate diam (K ) (from below) by a deterministic

algorithm with smaller than c√

nlog n , relative error needs more

than polynomial oracle-queries.


The negative result

If the randomized algorithm provides for every K ⊆ Rn an

estimate ζ(K ) ≤ diam (K ) in poly steps, then there exists aconvex K0 ⊂ R

n, amelyre

diam (K0)

ζ(K0)≥ c ·

√

nlog n

.


The negative result, cont:

If the randomized algorithm provides for every K ⊆ Rn an

estimate ζ(K ) ≤ diam (K ), and

ζ(K ) ≤ diam (K ) ≤ C · ζ(K ),

Then there exists a convex K0 ⊂ Rn, for which uses

exponentially many oracle queries.


Other metrics

Brieden-Kannan-Klee-Lovász-Simonovits,Various results for Lp norms.


Derandomization

Typical girth, Erdos LemmaErdos-Pósa Theorem

Regularity Lemma and randomness

Cayley graphs may behave in a randomlike way

Vera Sós: Deterministic and random and what is inbetween


Erdos-Rényi: connectedness


Margulis/Lubotzky-Phillips-Sarnak


Helen a vizbe


Evfolyamtalalkozon


Happy Birthday, Jóska, once more!

Phase transition in Discrete Mathematics, Randomized and...

Documents

Transcript of Phase transition in Discrete Mathematics, Randomized and...