Probability Theory Probability – Models for random phenomena.
Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1:...
Transcript of Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1:...
![Page 1: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/1.jpg)
LECTURE 1: Probability models and axioms
• Sample space
• Probability laws
Axioms
Properties that follow from the axioms
• Examples
Discrete
Continuous
• Discussion
Countable additivity
Mathematical subtleties
• Interpretations of probabilities 1
![Page 2: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/2.jpg)
Sample space
• Two steps:
Describe possible outcomes
- Describe beliefs about l ikelihood of outcomes
2
![Page 3: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/3.jpg)
• • • •
• • • •
•
Sample space
-I•• List (set) of possible outcomes, \l
• List must be:
Mutua Ily excl usive
Collectively exhaustive
At the "right" granu larity
. T • • o.. ...J .... D
3
![Page 4: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/4.jpg)
Sample space: discrete/ finite example
• Two rolls of a tetrahedral die
4
3 ,~
2 3,1
I ') I
I 2 3 4
Y = Secondroll
x = Firsl roll
sequential description
---::. 1. 1 :- 1.2
':::: 1.3 1.4
4.4
\ Y'ee...
4
![Page 5: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/5.jpg)
Sample space: continuous example
• (x,y ) such that 0 <x,y< 1 •
y
1
•
1 x
5
![Page 6: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/6.jpg)
Probability axioms I ---:---"J
/ / .// I
o I
f(A)
ater)
) -..n.. /fl./
( e:
•
• Event: a subset of the sample space
- Probability is assigned to events
• Axioms:
Nonnegativity: peA) > 0
Normalization: perl) = 1
(Finite) additivity: (to be strengthened l
If A n B = 0. then peA u B) = peA) + PCB, e...pJr sJ
6
![Page 7: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/7.jpg)
Some simple consequences of the axioms
Axioms
P(A) > 0
P(r2) = 1
For disjoint events:
P(A U B) = P(A) + P(B)
Consequences
P(A) < 1
P(0) = 0
P(A) + P(N) = 1
P(A U B U C) = P(A) + P(B) + P(C)
and similarly for k disjoint events
P( {S1, S2,···, Sk}) = P( {S1}) + ... + p.( {Sk })
= P(S1) + ... + P(Sk)
7
![Page 8: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/8.jpg)
c. .!
Some simple consequences of the axioms
Axioms
(0.1 peA) 2: 0
(\,1 P("') = 1
For d isjoi nt events:
«1 peA u B) = peA) + PCB)
i :: f (Jl) }i (S2.C
)
i:o 1 +j(cpj ~l(r:j;)=O. 8
![Page 9: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/9.jpg)
Some simple consequences of the axioms
• A , B , C disjoint: P(A U B U C) = P (A) + P(B) + P(C )
-------, I( AU!3uc.):::! ( (!luB) UC) ::- i [,4(//3) +1(c.)
=- f (/» t f(J3) +.f (c.) l<.
~) 1 (l-i,u ••• UA):.) :: ~ E (!f<')
1 (15.~ u~SJl.S() •• ·u ~s):~)
:: f' (5.) ~ '" -+ £ (S ... )- 9
![Page 10: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/10.jpg)
More consequences of the axioms
• If A C B , t hen P (A) $ P (B) B ':: A U (13 nAc)
.e (B): l? (4) t- -e. ('P/)t/~) ~ f(~)
0.0
• P (A U B) = P (A) + P (B) - 'P (A n B)1
10
![Page 11: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/11.jpg)
More consequences of the axioms
• p eA u B u C ) = p eA) + P (A C n B) + p eN n B C n C ) • K
l(Aue,uc ) =
11
![Page 12: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/12.jpg)
Probability calculation: discrete/finite example
• Two rolls of a t etrahedral die • Let every possible outcome have probability 1/ 16
I - I • P (X = 1) = '-I • 16
!,:::;- 17........ Let Z = mi n(X, Y)4 ~ :0?:. ., .... ~ 2-:=2"/ )(~,..,I";o)Y = Second 3 ~
roll
%- '0 %;:: • P(Z=4)= '/'62 ~ ~ ~ I Ii. I ,~ • P(Z = 2) = ~ 1 G •
I 2 3 4
x = First roll
12
![Page 13: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/13.jpg)
• •
• •
./-"--......... • • • A
• ~.----/.
1 prob =
n
Discrete uniform law
/f,." h - Assume 0 consist s of n equally likely elements - Assume A consist s of k elements
, peA) = k •,--
'11.
13
![Page 14: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/14.jpg)
Probability calculation: continuous example
• (x,y) such that 0 <x,y< 1 • Uniform probability law: Probability = Area
y
1 r--~::-A
• 1 x
I I I I -p ({(x,y) I x +y < 1( 2}) = 2· . z" . -2. 8
14
![Page 15: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/15.jpg)
Probability calculation steps
• Specify the sam pie space
• Specify a probability law
• Iden tify an event of interest
•• Ca lculate ...
15
![Page 16: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/16.jpg)
Probability calculation: discrete but infinite sample space
• Sample space: {1 ,2, ...} p
112 o
•••• ••• 114 •
• • •• o
11.•
1116 • •• •• Q •••• •
• o
1 2 3 4
- We are given Pen) =~, n = 1, 2 , ... 2 n
<b QO
c.. ~_I"'\_ .. _~'"')~_, I _I L ..,,_-. -",~.!! 2. .. ,0 2. 2. 1- ('/Ill
• P(outcomeiseven)= l ((9./1](,)"'5)
=.f Cf2~U~'1~ lJ{C.~u ' •• ) 0 1' (i) +1 ("1)+1 (<;)1- •••
, , .. I I I ( I ,
"
I ... - I • -~.. ~ t' + ... :: '1 I + '1- "+ • • • - ,
'-j!L '-t I - - 3"
16
![Page 17: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/17.jpg)
Countable additivity axiom
• Strengthens the finite additivity axiom
Countable Additivity AxiOm:~__~
If AI, A2, A3,'" is an infinite(S~quenc"3> of disjoint events,
then P(AI U A2 U A3 U ... ) = P(AI) + P(A2) + P(A3) + ... •
17
![Page 18: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/18.jpg)
Mathematical subtleties
Countable Additivity Axiom:
If AI, A2, A3,'" is an infinite sequence of d isj oint events,
then P(AI U A2 U A 3 U ... ) = P (A I) + P(A2) + P(A3 ) + ... •
-,
1= i (52) =1 ?
, U ?C'1:,YJ? . , GJ~ f (T(?:
• Additivity holds on ly for "countable" sequences of events
• The unit square (simlarly, the real line, etc.) is not countable ( its elements cannot be arranged in a sequence)
• "Area" is a leg itimate probability law on the unit square, as long as we do not try to assign probabilities/areas to "very strange" sets
18
![Page 19: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/19.jpg)
Interpretations of probability theory
• A narrow view: a branch of math
- Axioms =? theorems "Thm:" "Frequency" of event A "is" peA)
• Are probabilities frequencies?
- P(coin toss yields heads) = 1/ 2
- P(the president of ... will be reelected) = 0.7
• Probabilities are often intepreted as:
Description of beliefs
Betting preferences
19
![Page 20: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/20.jpg)
The role of probability theory
• A framework for analyzing phenomena with uncertain outcomes
- Rules for consistent reasoning
- Used for predictions and decisions
Predictions Proba bi lity theoryReal world (Analysis)Decisions
Data Models
In feren ceI 5 tati sti cs
20
![Page 21: Introduction to Probability: Lecture 1: Probability Models and Axioms · 2020-01-04 · LECTURE 1: Probability models and axioms • Sample space • Probability laws Axioms Properties](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f0ad6087e708231d42d93e5/html5/thumbnails/21.jpg)
MIT OpenCourseWarehttps://ocw.mit.edu
Resource: Introduction to ProbabilityJohn Tsitsiklis and Patrick Jaillet
The following may not correspond to a particular course on MIT OpenCourseWare, but has been provided by the author as an individual learning resource.
For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.