Chapter 3 Probability - Queen's Economics Department...

Chapter 3

Probability

• Knownpopulation and unknown sample which will develop language and concepts

• For more important problem of statistical inference (know the sample and make

inferential statements about the population)

3.1 Background: Set Theory

3.2 Set

• A set is a collection of things or objects which are called elements of the set.

3.3 Notation

• A set is denoted by a capital letter like A

• An element is denoted by a lower case letter, like 1, for the first element in the set

A.

• We may describe a set as

= {1 2 3 4 5}

• Two sets are equal if they contain the same elements (i.e. regardless of the order)

{1 2 3} = {3 1 2} = {2 1 3}

3.4 Null Set

• The null set is a set containing no elements.

1

2 CHAPTER 3. PROBABILITY

Example

= { 2 8} =

• By convention we assume 0 is in the null set {0} = .

3.5 Subset

• A set is a subset of , ⊂ , if every element of is also an element in

.

Example.

= {1 2 3}; = {1 2 3 4 5 6}; = { all real numbers from 0 to 4 };

• Then:

⊂ ⊂ 6⊂ 6⊂

• Note: Every set is a subset of itself, i.e. ⊂

3.6 Union of Sets

• The union of the sets and , ∪ , is another set containing all the elements

belonging to either or or both and .

Example:

= {1 2 3} = {2 4 6},

• ∪ = {1 2 3 4 6}

3.6.1 Notes

• If ⊂ , then ∪ = .

• ∪ =

• ∪ ∪ ∪ is a set containing all the elements belonging to at least one of

the sets , , , .

3.7. INTERSECTION OF SETS 3

3.7 Intersection of Sets

• The intersection of and , ∩ , (LMM write this as: ) is a set

containing all the elements common to both and :

Example: = {1 2 3}, = {2 4 6},• Then ∩ = = {2}

3.8 Disjoint Sets

• Two sets are disjoint if they have no elements in common, that is ∩ =

Example: = {1 2 3}, = {4 5 6},• Then ∩ =

3.9 Sample Set or Universal Set

• The sample set or universal set , is the set of all elements of interest in a

particular problem.

3.10 Complement of a Set

• If ⊂ , then the compliment of , , (in other text it is often ) is the

set of elements in that are not elements in .

Example: = {1 2 3 4 5 6}, = {1 2 3},then

= {4 5 6}Notes

• ∩ = { }• ∪ = { }


3.11 Experiments and Events

3.11.1 Experiment

• An experiment is any repeatable process that leads to the occurrence of one and

only one of several possible observations.

• When the outcomes cannot be predicted with certainty, then the experiment

is a random experiment. A random experiment is a process leading to two or more

possible outcomes with uncertainty as to which outcome will occur.

3.11.2 Simple Event or Outcome

• Each outcome of an experiment is called a simple event or outcome.

• Denote a simple event where represents the i th possible outcome.

3.11.3 Sample Space

• The possible outcomes of a random experiment are called the basic outcomes and

the set of all basic outcomes is called the sample space. The symbol S will be used

to denote the sample space.

3.11.4 Event

• An event, E, is any subset of basic outcomes from the sample space. An event

occurs if the random experiment results in one of its constituent basic outcomes.

Notes

• An event is a set whose elements are simple events.

• Events will be denoted by capital letters.

• An event is said to occur if any sample points (outcomes) in the event occur.

• is a subset of the sample space and is called the null event.

• The null event cannot occur since one of the outcomes in the sample space must

occur.

3.12. PROBABILITY OF AN EVENT 5

Example of Events

Consider tossing a coin twice.

Simple Events:

1 = () 2 = ( ) 3 = () 4 = ( )

Sample Space:

= {1 2 3 4} = {() ( ) () ( )}

Event

= { at least one head } = {1 2 3}

= { no heads } = {4}

3.11.5 Mutually Exclusive Events

• If and have no elements in common, then and are said to be mutually

exclusive events.

• The occurrence of any one event means that none of the others can occur at the

same time.

Example of Mutually Exclusive Event: tossing both an even and odd number on a

die in one toss

3.12 Probability of an Event

3.12.1 Classical View of Probability: Relative Frequency

• In classical probability theory, the probability of any event is the limiting relative

frequency with which the event occurs when the experiment is repeated a large

number of times under identical circumstances.

• That is the relative frequency in long run. That is () = lim→∞.


• A subtext for classical probability is the concept of equally likely:

• Suppose some experiment must occur in one of different ways and suppose

each of these outcomes can be considered equally likely then the probability of any

particular event outcome is 1.

( an event with equally likely outcomes) =Number of ‘favourable’ outcomes

Number of possible outcomes

3.12.2 Example of Equally Likely

Six- sided fair dice. What is the probability of an even number on a roll:

() =number of ways to have even

number of possible outcomes=3

6= 5

3.12.3 Subjective View of Probability or Modern View

• A subjective view of probability is a number in the interval [0,1] that reflects an

individual’s “degree of belief” in the occurrence of an event.

• Example of Subjective Probability: Jays beat the Yanks in the next 4 games.

3.13. PROBABILITY POSTULATES 7

3.13 Probability Postulates

Let = {1 2 }.

• For each simple event , 0 ≤ () ≤ 1 = 1 2

• If = {1 2 } ≤ , then

() = (1) + (2) + + ()

• For every event A,

0 ≤ () ≤ 1


• () =P

=1 () = 1.

• Probability of Complement of a Set (Complement Rule)

() = 1 − ()

• Odds: The odds in favour of

= ()

()=

()

1 − ()

3.13.1 Example of Probability

= { } = { }

3.14 The Addition Rule of Probabilities

( ∪ ) = () + () − ( ∩ )

• where ( ∪ ) is the probability of all the simple events in or or both

in and

• ( ∩ ) is the probability of and occurring (a joint probability)

Example: Return to the two coin tosses

= { 1 } = {1 2} () = {12}

= { 2 } = {1 3} () = {12}

( ∩ ) = (1) =1

4

Therefore

( ∪ ) = () + () − ( ∩ ) =1

2+1

2− 1

4

3.14. THE ADDITION RULE OF PROBABILITIES 9

Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 4-16

Probability Rules

The Complement rule:

The Addition rule: The probability of the union of two events is

1)AP(P(A)i.e., P(A)1)AP(

B)P(AP(B)P(A)B)P(A

Figure 3.1:


3.14.1 Question about Addition Rule of Probabilities and Sim-

ple Events

• Why can we sum the probabilities of the simple events without taking into ac-

count the intersection probabilities as indicated in the Additive Law of Probability?

• Answer: They do not intersect by definition.

3.14.2 Notes on Mutually Exclusive

• If and are mutually exclusive events, then ∩ = .

• Two events are mutually exclusive if they cannot both occur in a single trial of an

experiment.

• Simple events by definition are mutually exclusive.

• For mutually exclusive events

( ∪ ) = () + ()

since ( ∩ ) = 0.

3.15 Conditional and Joint Probabilities

• In this section we examine the probability concepts when two or more events are

jointly considered

3.15.1 Joint Probability

• ( ∩) represents the probability of events and occurring simultaneously

(jointly).

• Recall the Addition rule of Probabilities

( ∪ ) = () + () − ( ∩ )

3.15.2 Marginal Probability

• If we are considering the events and , we speak of themarginal probability

of or as () or ( ).

3.15. CONDITIONAL AND JOINT PROBABILITIES 11


A Probability Table

A

B

A

B

)BP(A

)BAP( B)AP(

P(A)B)P(A

)AP(

)BP(P(B) 1.0P(S)

Probabilities and joint probabilities for two events A and B are summarized in this table:

Figure 3.2:


3.15.3 Conditional Probability

• (|) represents the probability of event occurring given that has already

occurred.

• The formula for calculating this conditional probability is

(|) = ( ∩ )

()

• where we assume () 0 (impossible to condition on an event with a proba-

bility of 0 of occurring).

• Notice that if and are mutually exclusive then ( ∩ ) = 0 and

therefore

(|) = 0 (if Mutually Exclusive)

3.16 The Multiplication Rule of Probabilities

A simple rearrangement of the conditional probability gives the multiplicative law of

probability:

( ∩ ) = ()× (|)

or

( ∩ ) = ()× (|)

3.17. JOINT PROBABILITY TABLE 13

3.17 Joint Probability Table

Occupations after Undergraduate Education

Economics Math Poli Science Marginal Prob.

Law School 0.12 0.06 0.26 0.44

Job 0.08 0.10 0.14 0.32

Business School 0.14 0.09 0.01 0.24

Marginal Probability 0.34 0.25 0.41 1.00

This table allows us to compute a number of joint and conditional probabilities.

• () = ( ∩) + ( ∩) + ( ∩ ) = 12 + 06 + 26 = 44

() = ( ∩ ) + ( ∩ ) + ( ∩) = 06 + 10 + 09 = 25

•

( ∩ ) = 006

( |) = ( ∩ ) ()

=006

044= 1363

(|) = ( ∩ )

()=026

041= 634



What is the probability that a car has a CD player, given that it has AC ?

i.e., we want to find P(CD | AC)

Conditional Probability Example

Of the cars on a used car lot, 70% have air conditioning (AC) and 40% have a CD player (CD). 20% of the cars have both.

Figure 3.3:

3.17. JOINT PROBABILITY TABLE 15


Conditional Probability Example

No CDCD Total

AC .2 .5 .7

No AC .2 .1 .3

Total .4 .6 1.0

Of the cars on a used car lot, 70% have air conditioning (AC) and 40% have a CD player (CD). 20% of the cars have both.

.2857.7.2

P(AC)AC)P(CD

AC)|P(CD

(continued)

Figure 3.4:


3.18 Statistical Independence

• Two events and are independent if the probability of occurring is

unaffected (unchanged) by whether or not has occurred.

(|) = () ( () 0)

• The reciprocal arrangement is always true with independent events

(|) = () ( () 0)

• In the previous example on education and subsequent activity we note that the

events of a math undergraduate ( ) and then going to law school are not inde-

pendent since

( |) = 1363 6= () = 25

• A simple example of 2 independent events would be tossing a coin and rolling a die

at the same time.

• In this case we may ask what is the probability of a head given that you observe a

2 on the die (Answer is 1/2 regardless of the outcome of rolling the die).

• We may modify our multiplicative law of probability for independent events

3.18.1 Formula for Statistically Independent Events

Events and are independent if and only if

( ∩) = ()× ()

• This suggests an easy way to check if 2 events are independent is to simplymultiply

their marginal probabilities together and see if it equals the joint probability.

3.18. STATISTICAL INDEPENDENCE 17

Default Status: Men Versus Women (number)

Gender Default Not Default Total

Man 18 42 60

woman 12 28 40

Total 30 70 100

• We can easily convert this into a joint probability table by dividing each of the

entries by the total number in the table (100)

Question

• Is the probability of default independent of whether the individual is a man or

woman?

(|) = ()?

• First note that () = ( ∩) + ( ∩ ) = 18100+ 12

100= 30

• To calculate the conditional probability use the multiplicative formula

(|) = ( ∩) ()

=1810060100

= 30

• We see that the probability of default is independent of gender.


3.18.2 Independence and Sampling with and without Replace-

ment

• With replacement: Successive outcomes are independent

• Without replacement: Successive outcomes are not independent.

• Example

• What are the chances of drawing a heart on the second draw of two, given that

the first card drawn is the Queen of hearts.

Answer: If we have replacement:

(2|1) = 1352 (since we replaced the Queen of Hearts)

Answer: If we have no replacement:

(2|1) = 1251 (since we did not replace the Queen of Hearts)

3.18. STATISTICAL INDEPENDENCE 19


Odds

The odds in favor of a particular event are given by the ratio of the probability of the event divided by the probability of its complement

The odds in favor of A are

)AP(

P(A)

P(A)1-

P(A) odds

Figure 3.5:



Odds: Example

Calculate the probability of winning if the odds of winning are 3 to 1:

Now multiply both sides by 1 – P(A) and solve for P(A):

3 x (1- P(A)) = P(A)3 – 3P(A) = P(A)3 = 4P(A)P(A) = 0.75

P(A)1-

P(A)

1

3 odds

Figure 3.6:

3.19. BAYES’ THEOREM 21

3.19 Bayes’ Theorem

Let be an event having a prior probability () . Let be some new information

or event. The posterior probability of is given by

(|) = () × (|)

()

• Note that this formula can in fact be derived from the multiplicative law of proba-

bility:

•

( ∩) = () × (|)and

( ∩) = () × (|)

combine these two to give Bayes’ theorem.

• In the text book Bayes’ theorem appears in a slightly different way but is exactly

equivalent.

3.20 Mutually Exclusive and Exhaustive Events

• Let 1 2 be mutually exclusive events (i.e. (∩) = ), and suppose

1 ∪ 2 ∪ ∪ = ( the sample space )

• Then 1 2 are said to bemutually exclusive and collectively exhaus-

tive events In this case

() =

X=1

( ∩)

or

() =

X=1

() (|)

Either two of these formulae for () may be used in Bayes theorem:

(|) = () × (|)P

=1 () (|)

or

( |) = ( ∩ )P

=1 ( ∩ )


S tatis tics for Busi ness and Economics, 6e © 2007 Pearson Educat ion, Inc. Chap 4-36

Bayes’ Theorem

where:

Ei = ith event of k mutually exclusive and collectively

exhaustive events

A = new event that might impact P(Ei)

))P(EE|P(A))P(EE|P(A))P(EE|P(A

))P(EE|P(A

P(A)

))P(EE|P(AA)|P(E

kk2211

ii

iii

Figure 3.7:

3.21. BAYES THEOREM IN TERMS OF COMPLEMENTS 23

3.21 Bayes Theorem in Terms of Complements

• Consider two events and and we want to know (|). We note that

and form a mutually exclusive and exhaustive event.

• Therefore we have, using the above formula for Bayes’ Rule

(|) = () × (|) ()× (|) + ()× (|)

• Many forms in which we can express Bayes’ Theorem (see the summary at the

end of this lecture for all the different ways to express it)

• The basic notion is the same: with a prior probability () and if we observe

, how does the probability of change after taking into account the information

from .

• That is, what is the posterior probability: (|)?

3.21.1 Bayes’ Examples 1

• From the previous example on undergraduate education and subsequent occupa-

tion, we may ask if an individual is currently in law what is the probability of an

undergraduate in Political Studies?

(|) = ()× (|)

()=

41× 634

44= 59

3.21.2 Bayes’ Example 2

• Mr. Hat owns 2 plants that make jet engines. Let 1 and 2 be an engine made

from Plant 1 and 2 respectively. Mr. Hat knows that 2% of the engines from Plant

1 are defective and that 3% of the engines from Plant 2 are defective. Assume that

Plant 1 makes 40% of all engines with Plant 2 making the rest. An engine is selected

at random and is found to be defective. Find the probability it was made at Plant

1.


• This is a typical Bayes’ Theorem question where we need to calculate

(1|) = (1)× (|1) ()

where is a defective engine. We need first to calculate () ?

•

() = (1)× (|1) + (2)× (|2) = 4× 02 + 6× 03 = 026

• Therefore

(1|) = 4× 02

026= 308

3.22. PRINCIPLES OF COUNTING 25

3.22 Principles of Counting

• These techniques are used in discrete probability (Chapter 5) for the binomial

distribution)

3.22.1 Number of Elements in a Set

.The number of elements in a set is denoted by ()

Example:

= {2 4 6 8} () = 4

• A set can have either a finite or an infinite number of elements.

( ∪) = () + () − ( ∩)• If ∩ = then

( ∪) = () + ()

3.22.2 Multiplication Formula

• If event can occur in different ways and can occur in different ways,

then and can occur together in × different ways.

3.22.3 Factorial Notation

Let N be some positive integer then

! ≡ × ( − 1)× ( − 2)× 3× 2× 1

• We define 0! = 1.

An example

5! = 5× 4× 3× 2× 1 = 120


3.22.4 Permutation

A permutation of n distinct objects taken at a time is an arrangement in a specific

order of any of the things.

Example:

The permutations of 4 letters , , , and taken 2 at a time:

Notice that for the first choice we have 4 possibilities, for the second choice we have

3 possibilities, so the number of permutations is 4× 3 = 12.

3.22.5 Rule for Permutations

The number of permutations of things taken at a time denoted by

=

!

(− )!

3.22.6 Combinations

A combination of distinct objects taken at a time is an arrangement of any of

these objects without regard to order. We denote combinations using the formula

µ

¶=

=

!=

!

! (− )!

Note

1. ≥

2. = × !

Example: The number of combinations of the 4 letters , , , taken 3 at

a time are:

3.22. PRINCIPLES OF COUNTING 27

or using the formula µ4

3

¶=

4!

1!× 3! =4× 3× 2× 11× 3× 2× 1

QUESTIONS: NCT 453 454 465 470 476 478

Chapter 3 Probability - Queen's Economics Department...

Documents

Transcript of Chapter 3 Probability - Queen's Economics Department...