A Note on Foundations of Bayesianism

Stefan Arnborg, KTH

Gunnar Sjödin, SICS

BAYES RULES!-in finite models (ECCAI2000)

-but not in infinite! (MaxEnt2000)

Stefan Arnborg, KTH

Gunnar Sjödin, SICS

Normative claim of Bayesianism

EVERY type of uncertainty should be treated as probability

Aristotle, Sun Zi(300BC), Bayes(1763), Laplace, de Finetti, Jeffreys, Keynes, Ramsey, Adams, Lindley, Cheeseman, Jaynes,…

This claim is controversial and not universally accepted: Fisher(1922), Cramér, Zadeh, Dempster, Shafer, Walley(1999) …

Fundamental Justifications

Consistent Betting Paradigm:de Finetti, Savage(1950), Lindley(1982), … Snow(1999)

Information Based: Cox(1946), Aczél(1966), Jaynes(1994)Criticized by Paris(1994), Halpern(1999)

Main Result:

Cox information based justification can be derived with weak common sense assumptions. Difference between finite and infinite models.

Assumptions are:RefinabilityInformation independenceStrict monotonicityInfinite case: Model is closed

Caveats

Classical Bayesianism only for finite model

And for closed infinite model embeddable in the real line

Ordered closed infinite model: Extended probability (with infinitesimals) also permissible (non-monotonic and default logics).

Jaynes’s Desiderata on Uncertainty Management Uncertainty is measured by real number,

dependent on information subject possesses:A|C : plausibility of A given C.

Consistency.

Common sense.

Real Numbered uncertainties Given set of statements (possible world sets) A, B, C, … Plausibility A|C: plausibility of A given that

C is known to be true - a real number Conjunction: AB Disjunction A+B, Difference A-B AB|C=F(A|BC, B|C) A+B|C=G(A|C,B-A|C) not A|C=S(A|C)

RESCALABILITY THEOREMS

Under suitable assumptions there is a strictly monotone function w(x) such that

w(F(x,y))=w(x)w(y) w(G(x,y))=w(x)+w(y) I.E., by rescaling the plausibility measure by w,

model becomes a probability model I.E., if you accept the assumptions, then

Bayes Rules!

Invariance under rescaling

* and + are strictly monotone, symmetric, associative and jointly distributive

These properties are invariant under strictly monotone rescaling

If F and G violate the properties, rescaling is impossible.

Consistency

AB|C==BA|C, thus F(A|BC,B|C)=F(B|AC,A|C) A+B|C==B+A|C (AB)C|D==A(BC)|D (A+B)C|D==AC+BC|D Does this mean that F,G must be associative,

symmetric and jointly distributive??? No, not without additional assumptions!

OUR common sense assumptions REFINABILITY: Assume B’|B=c was defined; It

should be possible to refine another event A by A’ so that A’|A=c(cf Tribus, Jimison, Heckerman)

INFORMATION INDEPENDENCE:New events obtained by refinement of sameevent can be postulated independent:A|BC=A|C and B|AC=B|C’Knowledge of one has no effect on plausibility of the other’

Halpern’s Example: 4 Worlds

A

BC

DE

G

H I

J

KL

M

D|E=H|J

B|C = L|M

A|C = I|J E|G = A|B

H|J≈K|M

D|G = K|LM

Example: F(F(x,y),z)≈F(x,F(y,z))

C

DE

G

H I

J

KL

M

D|E=H|J=x

B|C = L|M=z

A|C = I|J E|G = A|B=y

H|J≈K|MD|G = K|LM

Refine:A’|A=D|E: INCONSISTE

C

DE

G

H I

J

KL

M

D|E=H|J=x

B|C = L|M=z

A|C = I|J E|G = A|B=y

H|J≈K|MD|G = K|LM

A’

H|J=A’AB|C=K|M !!!!!!!!!!!!!

OBSERVATION

The functions F and G must be symmetric and associative if refinability and information independence accepted

F , G must likewise be jointly distributiveF(G(x,y),z)=G(F(x,z),F(y,z))

But only on the finite range of definition Not enough for rescalability

Rescalability is solvability of LP

L4+L4-La=0L3+L5-La=0L2+L4-Lb=0L1+L5-Lb=0L4+L6-Lc=0L3+L7-Lc=0L2+L6-Ld=0L1+L8-Ld=0L1<L2<… L8

F(x4,x4)=F(x3,x5)=aF(x2,x4)=F(x1,x5)=bF(x4,x6)=F(x3,x7)=cF(x2,x6)=F(x1,x8)=d

x1<x2<…<x8

Find L such that ML=0 and DL>0

Unfortunately, theequalities implyL7=L8 !!!!

Theorem 4, finite case: Rescalability follows for finite models from

weak common sense assumptions: refinability and information independenceusing finite-dimensional duality theory

Conjecture: Savages and Lindleys consistent betting behavior analyses can be similarly strengthened

Probability model Counterexample, x^(i+1)=F(x^i,x)

Log probability

i

INFINITE CASE: NON-SEPARABILITY

x y

Extended Probability

Probability values are taken from ordered field. An ordered field is generated by rationals, reals

and infinitesimals (Conway) Previous example explained by x=0.5, y=0.5+. Extended probability has been shown equivalent

to non-monotonic reasoning schemes (Benferhat, Dubois, Prade, 1997).

Ordered rings and fields

Integers

Modular

Rational

Gaussian

Algebraic reals

Complex

Reals

Ordered infinitesimals

Conway’s No

No: a universal ordered field, extension of any ordered field.

Infinite Models

Theorem 6:Model is rescalable iff all plausibilities are separable !!- but separability somewhat contrived.

Assume plausibility model can be closed: F(x,y) defined on D^2 G(x,y) defined on D^2 if y<S(x) Range of F in D Range of G in D.

Theorems, Infinite Closed Models:

Theorem 9: Every plausibility measure is equivalent to extended probability.

Corollary 10:Every plausibility measure that can be embedded in the reals is equivalent to standard probability.

Proof Sketch Assume plausibility measure closed in D

0 1

1

F

G S(x)

Introduce subtraction (a,b)

(a,b)≈(c,d) if G(a,d)=G(c,b) orG(S(a),S(d))=G(S(c),S(b))

D DxD/≈

Extend from -infty to +infty

…. ….

0 1 2 3-1

(i+d)+(j+e)=i+j+d+e

(i+d)(j+e)=ij+ie+jd+de

We now have an ordered ringwhich is also an integral domain* and + are extensions of F and G

D ZxD

Create an ordered field

Standard quotient construction for ordered integral domain gives an ordered field (MacLane-Birkhoff)

This field is a subfield of the universal ordered field No of Conway

Proof Sketch, continued Ordered fields are generated by reals and

infinitesimals, non-zero values smaller than any real number, i.e., are models of extended probability

Proof Sketch, continued...

If the closed model is required to consist of real numbers, the model is equivalent to standard probability:All subfields of No with lowest upper bounds are embeddable in the real number field,and if field is real, then D was also real(the embedding process does not introduce infinitesimals in separable model).

SUMMARY

With assumptions of refinability, independence and strict monotonicity, finite ordered plausibility models are equivalent to probability models

With further assumption of closability, (infinite) plausibility models are equivalent to extended probability models

And closed plausibility models embeddable in the reals are equivalent to probability models.

OPEN PROBLEMS

NONE? (for well-defined statements)

However, Bayesian analysis of imprecise statements interesting alternative to fuzzy/rough/possibilistic logics(cf Wittgenstein (1956) word games).

Meaning of A fuzzy -> meaning depends on context -> modelled as conditional meaning.