Sharpening Occam ’s razor with Quantum Mechanics
description
Transcript of Sharpening Occam ’s razor with Quantum Mechanics
Sharpening Occam’s razor with Quantum
MechanicsSISSA Journal Club
Matteo Marcuzzi 8th April, 2011
Niclas Koppernigck(Copernicus)
Clausius Ptolemaeus(Ptolemy)
Tyge Brahe(Tychonis)
Describing Systems
Describing Systems
Johannes Kepler
Describing Systems
.3
2
cRT
Describing Systems
Algorithmic Abstraction
212
21
rmmGF
Describing Systems
Algorithmic AbstractionSame output
Describing Systems
Same outputDifferent intrinsic information!
Solar system
celestial objects
Sun FlaresPlanet Orography
MeteorologyPeople behaviour
Compton Scattering
Describing Systems
Same outputDifferent intrinsic information!
Much more memory
required!OCCAM’S RAZOR
Describing SystemsN Spin Chain
Up parity
1 spin-flip per second10 if even
if odd
0
Describing SystemsN Spin Chain
Up parity10 if even
if odd
0
1 spin-flip per second
1
Describing SystemsN Spin Chain
Up parity10 if even
if odd
0
1 spin-flip per second
1 0
Describing SystemsN Spin Chain
Up parity10 if even
if odd
0
1 spin-flip per second
1 0 1
Describing SystemsN Spin Chain
Up parity10 if even
if odd
0
1 spin-flip per second
1 0 1 0
Describing SystemsN Spin Chain
Up parity10 if even
if odd
0
1 spin-flip per second
1 0 1 0 1
Describing SystemsN Spin Chain
Up parity10 if even
if odd
0
1 spin-flip per second
1 0 1 0 1 0
Describing SystemsN Spin Chain
Up parity10 if even
if odd
0
1 spin-flip per second
1 0 1 0 1 0 1
Describing SystemsN Spin Chain
Up parity10 if even
if odd
0
1 spin-flip per second
1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
N bits needed
Describing SystemsHidden System
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
read x
return (x+1) mod 21-bit only!
Statistically equivalent output
N bits
Computational Mechanics
• Statistical equivalence
•Measure of complexity
• Pattern identification
0m
Computational Mechanics
0m 0stm
• Statistical equivalence
•Measure of complexity
• Pattern identification
Computational Mechanics
0m 0stm 04 m
• Statistical equivalence
•Measure of complexity
• Pattern identification
Computational Mechanics
0m 0stm 04 m ?
• Statistical equivalence
•Measure of complexity
• Pattern identification
Computational Mechanics
�
21012 SSSSSS Stochastic Process
DiscreteStationary
iSRandom Variables A: Alphabet
edcbaA ,,,,
1,0'A
GCTAA ,,,''
Computational Mechanics
�
21012 SSSSSS Stochastic Process
DiscreteStationary
iSRandom Variables A: Alphabet
123 SSSS �
Pasts
210 SSSS Futures
Computational Mechanics
�
21012 SSSSSS Stochastic Process
DiscreteStationary
iSRandom Variables A: Alphabet
123 ssssA ��
Set of histories
210 ssssA Set of future strings
Computational Mechanics
�
21012 SSSSSS Stochastic Process
DiscreteStationary
Machine
000101000101110101101…
Statistical Equivalence
)()( sSsS MS
� PP
1,0A
Computational Mechanics
�
21012 SSSSSS Stochastic Process
DiscreteStationary
Machine
A�
1,0A…010100010
…1100111
…01010101
States
Partition R
Computational Mechanics
�
21012 SSSSSS Stochastic Process
DiscreteStationary
Machine
A�
States
Partition R
1R
2R
ijS RsRas ���P
a
Computational Mechanics
�
21012 SSSSSS Stochastic Process
DiscreteStationary
Machine
A�
1R
2R ijM
aij RRaT ,)( P Transition Rates
)(11
aT
)(12
aT
Rj
Ri
Computational Mechanics
�
21012 SSSSSS Stochastic Process
DiscreteStationary
OCCAM POOL
Computational MechanicsA little information theory
spspSHAs
log Shannon entropy
XSH Conditional entropy YXSH ,
XSHSHXSI : Mutual information
SSIE�
: Excess entropy
SH
Computational Mechanics
Machine Cannot distinguish between them
SSHSH�
R
Partition RA�
We want to preserve information
SSHSH�
R
Computational Mechanics
Machine
SSHSH�
R
Partition RA�
We want to preserve information
SSHSH�
R
with the least possible memory
0C Log(# states)minimize
Computational Mechanics
Machine
SSHSH�
R
Partition RA�
We want to preserve information
SSHSH�
R
with the least possible memory
minimize RHC Statisticalcomplexity
Computational Mechanics
SSHSH�
R
We want to preserve information
SSHSH�
R
with the least possible memory
minimize RHC Statisticalcomplexity
OCCAM POOL Optimal partition
Computational Mechanics
Optimal partition
We want to preserve informationwith the least possible memory
minimize RHC Statisticalcomplexity
SSHSH�
R
)'()( ssss SS
�� �� PP
ε-machine
ε
'~ ss ��if
Causal States
(unique)
Computational Mechanics: Examples
2-periodic sequence
2-periodic, ends with
2-periodic, ends with
1p1p
A
B
I
initial state
Computational Mechanics: Examples
2-periodic sequence
1p1p
A
B
I
initial state21p
21precurrent transient
Computational Mechanics: Examples1D Ising model
p p
p
p
ijV transfer matrix
jij
i Vuujip
Computational Mechanics: Examples1D Next-nearest-neighbours Ising
p
p
p
p
p
p p
p
31 J
12 J
2.0T
2
Computational Mechanics: Examples1D Next-nearest-neighbours Ising
p
p
p
p
p
p p
p
31 J
12 J
2.0T
23
Computational Mechanics: Examples1D Next-nearest-neighbours Ising
p
p
p
p
p
p p
p
31 J
12 J
2.0T
23
1
Computational Mechanics: Examples1D Next-nearest-neighbours Ising
p
p
p
p
p
p p
p
31 J
12 J
2.0T
23
1
Computational Mechanics: Examples1D Next-nearest-neighbours Ising
p
p
p
p
p
p p
p
31 J
12 J
2.0T
negligible
1
1
8B
Computational Mechanics: Examples1D Next-nearest-neighbours Ising
p
p
31 J
12 J
2.0T
1
1
period 3
period 18B
Sharpening the razor with QM
EC Statistical complexit
y
Excess entropy SSI
�: RH
EC Ideal system
Sharpening the razor with QM
,,AA�
ε
ε-machines are deterministic
ε
,,AA�
ε
Sharpening the razor with QM
1R
3R2R
4R
0,, )(44
)(34
)(24 blueblueblue TTT EC
,,AA�
ε
Sharpening the razor with QM
EC 0)( cijT
fixed i,c unique j
fixed j,c unique i
ideal
Sharpening the razor with QM
ε qεcausal state Ri system state i
symbol “s” symbol state s
siTSis
skik
,
)()(sijT
q-machine states
qεsystem state i
symbol state s
siTSis
skik
,
)(
q-machine states
Sharpening the razor with QM
CLASSICAL
QUANTUM
Prepare kS
Measure C.S.j t
2kSjt )(t
kjTProbability
tjS
)( iiq RS PP ip
qεsystem state i
symbol state s
siTSis
skik
,
)(
q-machine states
Sharpening the razor with QM
CLASSICAL
QUANTUM )( iiq RS PP ip
ii
i ppC log
logtrCq
iii
i SSp
qCC
E
E
qεsystem state i
symbol state s
siTSis
skik
,
)(
q-machine states
Sharpening the razor with QM
CLASSICAL
QUANTUM
ii
i ppC log
logtrCq
iii
i SSp
qCC
ijji SS
E
E
ks
sjk
sik TT
,
)()(Ideal system
E
E
qεsystem state i
symbol state s
siTSis
skik
,
)(
q-machine states
Sharpening the razor with QM
CLASSICAL
QUANTUM
qCC
ijji SS
Non-ideal systems
Quantum mechanics improves efficiency
Sharpening the razor with QM
single spinp
21
p
p
p
p1 p1 21
p
?21
21
C
E
qC
References
M. Gu, K. Wiesner, E. Rieper & V. Vedral - "Sharpening Occam's razor with Quantum Mechanics" - arXiv: quant-ph/1102.1994v2 (2011)
C. R. Shalizi & J. P. Crutchfield - "Computational Mechanics: Pattern and Prediction, Structure and Simplicity" - arXiv: cond-mat/990717v2 (2008)
D. P. Feldman & J. P. Crutchfield - "Discovering Noncritical Organization: Statistical Mechanical, Information Theoretic, and Computational Views of Patterns in One-Dimensional Spin Systems" - Santa Fe Institute Working Paper 98-04-026 (1998)