Random objects, and objects of low complexity

Random objects, and objects of low complexity

André Nies

The University of Auckland

August 5

André Nies (The University of Auckland) Randomness and complexity August 5 1 / 38

Part I

Introduction to randomness and complexity


To see where the English word randomness comes from, we look in


Etymology of “randomness”Sir Thomas Malory, Le Morte D’Arthur, Book I.10 (1485):

The old French noun “raundon” is derived from randir, “to gallop”. Ithas been used in English since the 14th Century.Metaphorically, “raundon” also meant ‘impetuousity’.


Randomness in other languagesFrench chance(lucky chance) German ZufallRussian sluchainostPolish losowosc (‘Los’ means ‘fate’)Georgian shemtkhvevitobaSpanish aleatoridad (Latin ‘alea’ means ‘dice’)Hebrew mikriyut (‘mikre’ - occurrence, incident)Greek τυχαιoσ (tuchaios) (identical in ancient Greek)

Mandarin

· 1 ·

(Sui ji).

Humans have an intuitive concept of randomness. This is evidencedby the fact that there is a word for it in different cultural contexts, andthat the word stems in different languages are usually unrelated. Theconnotations of the concept can vary depending on the culturalcontext.


Clarifying the intuitive notion of randomness

I Random event: the change from an initial situtation to one ofseveral possible end situations. There is no cause for this particularend situation.

I If one repeats the experiment, the same initial situation can lead toa different end situation.

I Examples:

I radioactive decay of a particleI tossing a coin.


Quantum random number generator


Random continuous functions (Mathieu Hoyrup)

0 1

0 1

I These pictures are actually random walks with time intervals oflength 1/n, where n = 4000. Such random walks look more andmore like random continuous functions as n gets large.

I The slope at each step is ±√

n.I We toss a coin to decide whether to go up or down.


Two more examples of random functions

0 10 1


Random continuous functions and the Dow-Jones

0 1


Random continuous functions and exchange rates

0 1


Complexity

I The complexity measures the information contained in an object.

I A system consists of parts and relationships between them.

I The interdisciplinary area of complex systems connects statisticalphysics, life sciences, nonlinear dynamics, probability theory andalgorithmic information theory.Reference: G. Nicolis and C. Nicolis, Foundations of Complex Systems,World Scientific 2007.


Visual representation of major paths in the internet (2006)


Network of neurons


Interaction of randomness and complexity: evolution

Evolution: random changes and selection lead to complex systems

Examples

I human genome

I natural languages

I internet?


(Pseudo)randomness vs. low complexity : Green/TaoI Szemerédi’s Theorem states that a set A of natural numbers with

positive density (i.e., 0 < lim supn |A ∩ 0, . . . ,n|/n) containsarbitrarily long arithmetic progressions.

I Terence Tao’s 2006 ICM paper surveys how each of the threeknown proofs of the theorem proceeds by splitting an object into apseudorandom and a structured (low complexity) part.This is surprising as the proofs of Szemerédi’s Theorem are fromrather different areas:

I graph theory (Szemerédi himself, 1975),I ergodic Theory (Furstenberg, 1977), andI Fourier Analysis (Gowers, 2001).

I Green and Tao (Ann of Math., to appear) use these ideas to showthat the set of primes (a set of density 0) contains arbitrarily longarithmetic progressions.


Part II

Studying randomness via computability


Basics of computability theory 1A Turing machine in action looks like this:

The finite control holds a Turing program. A function F : N→ N iscalled computable if there is a Turing program which, with n in binaryon the input tape, outputs F (n) in binary.

n // Turing program // F (n)

N can be replaced by domains that are effectively encoded by naturalnumbers, such as the rationals Q.

A real r ∈ R is computable if there is a computable sequence (qn)n∈Nof rational numbers such that |r − qn| ≤ 2−n for each n.

Examples of computable reals are π,e,√

2, . . .André Nies (The University of Auckland) Randomness and complexity August 5 19 / 38

Randomness in probability theory

Imagine we toss a fair coin repeatedly. In probability theory this ismodelled as follows.

I We have a sequence (Xn)n∈N of 0,1-valued random variables on aprobability space (M,B,P).

I The Xn are independent. We have P[Xn = 0] = 1/2 for each n.

I Each w in the space determines a sequence (Xn(w))n∈N of cointosses.

I To say that a property holds for a “random” sequence means thatthe property holds with probability 1. Thus, the set of exceptions isa null set, i.e. has measure 0.

I An example of such a property is the law of large numbers.


The probability spaces

In the following, the probability space will be either

I Cantor space 0,1N with the product topology, and the productmeasure, where 0,1 is equipped with the measure such thatboth 0,1 have probability 1/2, or

I the unit interval [0,1] of reals, with Lebesgue measure.

The two spaces are equivalent via the binary representation of reals.The intuition on randomness is different in each case, though.

I For Cantor space we think of a sequence of coin tosses.I For the unit interval, we pick an “arbitrary” real in one go.


Algorithmic randomness notions

The idea in algorithmic randomnessz is random⇐⇒ z avoids each algorithmic null set.

I We have to specify what we mean by an algorithmic null set.

I For instance, having more than 3/4 zeros on arbitrarily long initialsegments will be an algorithmic null set in the sense of Martin-Löf.


Algorithmic null sets in the sense of Martin-Löf (1966)An open set U ⊆ [0,1] is called effective if it is the effective union ofopen interval with rational endpoints. We require the followingconditions on a sequence (Un)n∈N of open sets:

the Un are effective opensets, uniformly in n, and

Un has measure at most 2−n

for each n.

0 1

...

U0U1U2U3U4

DefinitionA sequence (Un)n∈N as above is called a Martin-Löf test.The real r is Martin-Löf random if r 6∈

⋂Un for each such test.


Functions of bounded variation are differentiableoutside a null set

A function f : [0,1]→ R is of bounded variation if it doesn’t “wiggle” toomuch:

∞ > supn∑

i=1

|f (ti+1)− f (ti)|,

where the sup is taken over all collections t1 ≤ t2 ≤ . . . ≤ tn in [0,1].

Theorem (Classical Analysis)Let f : [0,1]→ R be of bounded variation. Then

f ′(r) exists for each r outside a null set (depending on f ).


Complexity of the exception set

Theorem (Demuth 1975/Brattka, Miller, Nies ta)Let r ∈ [0,1]. Thenr is ML-random⇐⇒

f ′(r) exists, for each function f of bounded variation suchthat f (q) is a computable real, uniformly for each rational q.

I The implication “⇒” is an effective version of the classical theorem.I The implication “⇐” has no classical counterpart. To prove it, one

builds a computable function f of bounded variation that is onlydifferentiable at ML-random reals.


Excerpt from Demuth’s 1975 paper

и, следовательно,

в) если с РЧ и 8 ПЧ такие, что

с б # ^ & 1, С ^ е # ^ ) & 1' т ' . I ГГс ) I 2: I ^ 1 с I ,

то 1 л ^ ( ! ^ п ^ & 5 « ^ А ^ а ^ ) 1Лемма 6, Пусть ф функция, которая не может не быть

функцией слабо ограниченной вариации, а С р»п# множество

НЧ такие, что сегменты, е б ^ ^ , ; X е С ; не перекрываются и

Чх(\?Ы)\>0эЭ1(1еС8<Эл<&^)^*<:Эъ(&^Аз>» •

Тогда У | ( | с П ^ В 1 1 1 . В ^ ( ^ б С & | е Ф ^ Ь Я * * ™ , ^ Р >

и для всяких НЧ /т, и Л существуют последовательность ра-

циональных сегментов ^ $ Й » ^ ^ , неинфинитное р .п . множест-

во НЧ 3) и неубывающая последовательность НЧ ^^^^, т а "

кие, что

^ ^ ^ Ч / ^ 1 ^ 1 < 4 г ) & ^С!еП8<11а^СХ!С&|!

с ^ Х ^ & п З ^ л С & е Р ^ б ^ .

Доказательство» Пусть ^ алгорифм, построенный со-

гласно лемме 5, а гт, НЧ. Существуют последовательность от-

личных друг от друга пар НЧ < е ^ а т ^ ^ и последователь-

ность рациональных сегментов < Х^1Ьи, т « к и е , что

У^ гСЗт,С6^а^х^аа>аСЗ^А<Х!СС (^4Чс с^)&

Ьи^^т,^ а % г * < ^ ЗХСХ е С С ^^С<^ ^ ^ асСа л (^б ^ 4^Ь

5901

Main

Lemma


Part III

Studying computability via randomness


Basics of computability theory 2

A function ψ : N→ N is partial computable if there is a Turing programwhich, with n on the input tape, outputs ψ(n) if ψ(n) is defined, andloops forever otherwise.

n // Turing program // ψ(n) if ψ(n) is defined

n // Turing program / if ψ(n) is undefined

We say that A ⊆ N is computably enumerable (c.e.) if A is the domainof a partial computable function. Equivalently, one can effectivelyenumerate the elements of A in some order.

(We)e∈N is an effective listing of all the c.e. sets.


Basics of computability theory 3

The halting problem is a universal c.e. set:

H = 〈x ,e〉 : x ∈We.

For sets X ,Y ⊆ N, we write

X ≤T Y

(X is Turing below Y ) if an oracle Turing machine computes X with Yon the oracle tape.


Machines, and descriptive string complexity K

A machine M is a partial computable function from binary strings tobinary strings. It is called prefix-free if its domain is an antichain underthe prefix relation of strings.

I There is a universal prefix-free machine U.I The prefix-free Kolmogorov complexity is the length of a shortest

U-description of y :

K (y) = min|σ| : U(σ) = y.

I One can show that

2−K (y) is proportional to Prob[U(σ) = y ].

Thus, K (y) is the self-information, or surprisal, of y (in the sense ofShannon/Tribus).


Degree of randomness for sequences of bits00000000 00000000 00000000 00000000 0000. . .

10100100 01000010 00001000 00010000 0001. . .

00100100 00111111 01101010 10001000 1000 . . .

10010100 00010001 11110100 00101101 1111 . . .

11101101 01111010 10101111 11001110 1110 . . .

Explanations:

Only zeros∏i 0i1

π − 3 in binary

Coin tossing

Coin tossing


Schnorr’s 1973 Theorem

For an infinite sequence of bits Z let

Z n = Z (0) . . .Z (n − 1).

We think of a string τ as random if K (τ) ≥ |τ | − b for some smallconstant b. An infinite sequence of bits Z is Martin-Löf random iff eachof its initial segments is random (incompressible) as a string:

Theorem (Schnorr 1973)Z is ML-random⇐⇒ there is b ∈ N such that ∀n [K (Z n) > n − b].

An example of a ML-random real is Chaitin’s halting probability

Ω =∑

U(σ)↓ 2−|σ|.


Definition of K -trivialityIn the following we identify a natural number n with the string that is itsbinary representation. For a string τ of length n, up to constants wehave K (n) ≤ K (τ), since we can compute n from τ .

Definition (Chaitin, 1975)A sequence of bits A is K -trivial if, for some b ∈ N,

∀n [K (An) ≤ K (n) + b],

namely, all its initial segments have minimal K -complexity.

It is not hard to see that K (n) ≤ 2 log2 n + O(1).

Z is ML-random ⇔ ∀n [K (Z n) > n −O(1)]

A is K -trivial ⇔ ∀n [K (An) ≤ K (n) +O(1)]

Thus, K -triviality means being far from random.


Existence of K -trivials

I Solovay (1976) built a non-computable K -trivial set.

I This was improved to a computably enumerable example byDowney, Hirschfeldt, Nies, and Terwijn (2002).


Lowness for Martin-Löf randomnessThe following specifies a sense in which a set A is computationallyweak when used as an oracle.

DefinitionA is low for Martin-Löf randomness if every ML-random set Z isalready ML-random with “oracle” A.

Kucera and Terwijn (1999) built a computably enumerable, butnon-computable set of this kind.The following shows that

far from random⇐⇒ close to computable.

Theorem (N, Adv. in Math., 2005)Let A ⊆ N. Then

A is K -trivial⇐⇒ A is low for ML-randomness.


Lowness for K

A is called low for K (Muchnik, 1998) if enhancing the computationalpower of the universal machine by an oracle A does not decreaseK (y):

∀y [K (y) ≤ K A(y) + O(1)].

I The straightforward implications arelow for K ⇒ low for ML andlow for K ⇒ K -trivial.

I In N (2005) we show the converseimplications. Low for K

Low for ML

easyeasy

K-trivial

Low for K

Low for ML

easyeasy

K-trivial

harderhardest,non-uniform


References

!"#$%&'#()*+,$-#'.##&$/01)2'+-%*%',$+&3$(+&301&#44$"+4$-##&$

+&$+/'%5#$+(#+$06$(#4#+(/"$%&$(#/#&'$,#+(47$(#6*#/'#3$-,$+1)*#$62&3%&8$%&$'"#$9:;7$&21#(024$.0(<4"0)47$+&3$)2-*%/+'%0&4$0&$'"#$42-=#/'>$!"#$/01)*#?%',$+&3$'"#$(+&301&#44$+4)#/'$06$+$4#'$06$&+'2(+*$&21-#(4$+(#$/*04#*,$(#*+'#3>$!(+3%'%0&+**,7$/01)2'+-%*%',$'"#0(,$%4$/0&/#(&#3$.%'"$'"#$/01)*#?%',$+4)#/'>$@0.#5#(7$/01)2'+-%*%',$'"#0(#'%/$'00*4$/+&$+*40$-#$24#3$'0$%&'(032/#$1+'"#1+'%/+*$/02&'#()+('4$60($'"#$%&'2%'%5#$&0'%0&$06$(+&301&#44$06$+$4#'>$A#/#&'$(#4#+(/"$4"0.4$'"+'7$/0&5#(4#*,7$/0&/#)'4$+&3$1#'"034$0(%8%&+'%&8$6(01$(+&301&#44$#&(%/"$/01)2'+-%*%',$'"#0(,>$$

B05#(%&8$'"#$-+4%/4$+4$.#**$+4$(#/#&'$(#4#+(/"$(#42*'47$'"%4$-00<$)(05%3#4$+$5#(,$(#+3+-*#$%&'(032/'%0&$'0$'"#$#?/%'%&8$%&'#(6+/#$06$/01)2'+-%*%',$+&3$(+&301&#44$60($8(+32+'#4$+&3$(#4#+(/"#(4$%&$/01)2'+-%*%',$'"#0(,7$'"#0(#'%/+*$/01)2'#($4/%#&/#7$+&3$1#+42(#$'"#0(,>$$!"#$%#$&'((C(#6+/#$$D>$!"#$/01)*#?%',$06$4#'4$E>$!"#$3#4/(%)'%5#$/01)*#?%',$06$4'(%&84$F>$G+('%&H$IJ6$(+&301&#44$+&3$%'4$5+(%+&'4$K>$L%+80&+**,$&0&/01)2'+-*#$62&/'%0&4$M>$I0.&#44$C(0)#('%#4$+&3$!H'(%5%+*%',$N>$:01#$+35+&/#3$/01)2'+-%*%',$'"#0(,$O>$A+&301&#44$+&3$-#''%&8$4'(+'#8%#4$P>$B*+44#4$06$/01)2'+'%0&+*$/01)*#?%',$Q>$@%8"#($/01)2'+-%*%',$+&3$(+&301&#44$

!"##$%&'!"#$%&'%!#()*+,-!

.&/%000000000000000000000000000000000000000000000000000000000000000!

1*'+)+2+)3*000000000000000000000000000000000000000000000000000000000000!

455(%''00000000000000000000000000000000000000000000000000000000000000!

6)+7000000000000000000000000! !!!!!!!!!!!!!!!!!!!!8+&+%00000000!!!!!!!!!!!!9)#000000000000!

:&7+)/%!;<3*%00000000000000000000000000000!$

()"*$%&'!")=!5)==%(%*+!=(3/!&>3?%-!

.&/%000000000000000000000000000000000000000000000000000000000000000!

1*'+)+2+)3*000000000000000000000000000000000000000000000000000000000000!

455(%''00000000000000000000000000000000000000000000000000000000000000!

6)+7000000000000000000!000000000000000000!!8+&+%0000000!!9)#0000000000000!

!

!"#!"#!"#!"#$%&'($')%'&*$%&'($')%'&*$%&'($')%'&*$%&'($')%'&*$$$$

;$%&'%!(%+2(*!+<)'!=3(/@!&$3*A!B)+<!732(!C(%5)+!C&(5!)*=3(/&+)3*@!+3,!!DE=3(5!F*)?%(')+7!;(%''@!D(5%(!:%#+G@!HIIJ!K?&*'!L3&5@!6&(7@!.6!HMNJO!

;$%&'%!'%*5!/%,! ! ! ! ! ! !

0000!C3#)%'!3=!63/#2+&>)$)+7!&*5!L&*53/*%''!!!!!!!!!!!!PMQRIRJPRPHOIMSRJ!!!!TJJIGII!U+,,-..$

;$%&'%!)*C$25%!TNGNI!'<)##)*A!&*5!<&*5$)*A!=3(!+<%!=)('+!>33V@!TJGNI!=3(!%&C<!&55)+)3*&$!>33VG!"W4@!W1@!64@!X!.6!(%')5%*+'!#$%&'%!&55!'&$%'!+&EG-!$

*/0123%$"34&51/%"&3'$

$K*C$3'%5!)'!&!C<%CV!3(!/3*%7!3(5%(!/&5%!32+!+3!DE=3(5!F*)?%(')+7!;(%''!=3(!T0000$

!;$%&'%!C<&(A%!/7!C(%5)+!C&(5,$

!!!!!!!!!!! !Y184!!! !Z&'+%(!6&(5!!!! !4/%()C&*!KE#(%''!

6(%5)+!6&(5!*3G0000000000000000000000000000000000000000!!KE#G!:&+%000000000!

8)A*&+2(%00000000000000000000000000000000000000000000000000000000000000!!"#$%&'%!()*%+!,-%./*!0&-.!)-.%-'!&-%!1)*!2&$/.!3/*4)5*!&!'/61&*5-%!&1.!7/$$/16!&..-%''8!9%!'5-%!*)!0):;$%*%!*4%!7/$$/16!/1<)-:&*/)1!'%0*/)1=!4$$!'&$%'!'2>[%C+!+3!&CC%#+&*C%!>7!DE=3(5!F*)?%(')+7!;(%''!&+!)+'!3==)C%'!)*!6&(7@!.6G!D==%(!Y&$)5!)*!+<%!FG8G!3*$7G!!

BRGC9!;STIT!U$;VL$A;VLRGVW::$

;&3(X$V%#4$

EYYQ$$KEY$))>$DF$*%&#$%**24>$$

QOPHYHDQHQEFYONHD$ZDDY>YY$)**+,,$

-./%(((((0,1(

&5625$4&517$(/82$9.:$

B+**(2+*,,+342+5446([GH\7$PHM]FY$W:!^$0($6+?$2+727+655+28,8$$R(3#($0&*%&#$+'(999+":;+<"=>:&(+&3$#&'#($)(010'%0&$/03#(0637,((

'0(&./%(0,1$

Y)')+!2'!&+!BBBG32#GC3/U2'!+3!'%%!32(!3+<%(!%EC)+)*A!+)+$%'\!

My book “Computability and Randomness”,Oxford University Press, 447 pages, Feb. 2009.

Forthcoming book “Algorithmic randomness and complexity” , >800pages, by Downey and Hirschfeldt.

These and other slides, on my web page.


Random objects, and objects of low complexity

Documents

Transcript of Random objects, and objects of low complexity