Chapters 14 & 15 Probability

math2200

Randomness v.s. chaos

• Neither of their outcomes can be anticipated with certainty

• Randomness– In the long run, the outcomes are settled

down in a way that is actually consistent and predictable

Random phenomenon• A random phenomenon is a situation in which we know

what outcomes could happen, but we don’t know which particular outcome did or will happen.

• Example: the color of the traffic light at a particular intersection– Suppose leaving at home at the same time every day– The time arriving at the light is random– Red or green is also random– In the long run, we can expect a fixed fraction of the time the

light will be red– How do we figure out this fraction?

Accumulated percentage Day Light fraction [1,] G 1.0000000 [2,] G 1.0000000 [3,] G 1.0000000 [4,] R 0.7500000 [5,] G 0.8000000 [6,] G 0.8333333 [7,] G 0.8571429 [8,] R 0.7500000 [9,] R 0.6666667[10,] G 0.7000000

0 20 40 60 80 100

0.5

0.6

0.7

0.8

0.9

1.0

# of outcomes

fract

ion

Probability• In the long run, the relative frequency of red lights settles

down to about 50%• Probability

– An event’s long-run relative frequency– The probability to see a red light is 50%

• In general, each occasion upon which we observe a random phenomenon is called a trial.

• At each trial, we note the value of the random phenomenon, and call it an outcome.

• When we combine outcomes, the resulting combination is an event.

• Sample space: the collection of all possible outcomes• Independence

– Outcome of one trial does not influence or change the outcome of another trial

Example

• Trial: flip two coins at the same time • Outcome: combination of heads (H) or

tails (T)• Sample space: all possible outcomes

– S = {HH,TT,HT,TH}• Events:

– A =two coins give the same results= {HH,TT}– B = two coins give different results={HT,TH}

The Law of Large Numbers (LLN)

• The Law of Large Numbers (LLN) says that the long-run relative frequency of repeated independent events gets closer and closer to a single value.

• We call the single value the probability of the event.

• Because this definition is based on repeatedly observing the event’s outcome, this definition of probability is often called empirical probability.

Jacob Bernoulli

The Nonexistent Law of Averages

• The LLN says nothing about short-run behavior.

• Relative frequencies even out only in the long run, and this long run is really long (infinitely long, in fact).

• The so called Law of Averages (that an outcome of a random event that hasn’t occurred in many trials is “due” to occur) doesn’t exist at all.

Probability• When probability was first studied, a group of French

mathematicians looked at games of chance in which all the possible outcomes were equally likely.– It’s equally likely to get any one of six outcomes from the

roll of a fair die.– It’s equally likely to get heads or tails from the toss of a fair

coin.• However, keep in mind that events are not always

equally likely.– A skilled basketball player has a better than 50-50 chance

of making a free throw.

• The probability of an event is the number of outcomes in the event divided by the total number of possible outcomes.

P(A) =

Probability (cont.)

# of outcomes in A

# of possible outcomes

Personal Probability

• In everyday speech, when we express a degree of uncertainty without basing it on long-run relative frequencies, we are stating subjective or personal probabilities.

• Personal probabilities don’t display the kind of consistency that we will need probabilities to have, so we’ll stick with formally defined probabilities.

Formal Probability

1. Two requirements for a probability:– A probability is a number between 0 and 1. – For any event A, 0 ≤ P(A) ≤ 1.

Formal Probability (cont.)

2. Probability Assignment Rule:– The probability of the set of all possible

outcomes of a trial must be 1.– P(S) = 1 (S represents the set of all possible

outcomes.)

Formal Probability (cont.)

3. Complement Rule: The set of outcomes that are not in the event A is

called the complement of A, denoted AC. The probability of an event occurring is 1 minus the

probability that it doesn’t occur: P(A) = 1 – P(AC)

Formal Probability - Notation

Notation alert:• In this class we use the notation P(A or

B) and P(A and B). • In other situations, you might see the

following:– P(A B) instead of P(A or B) – P(A B) instead of P(A and B)

Formal Probability (cont.)

4. Addition Rule:– Events that have no outcomes in common

(and, thus, cannot occur together) are called disjoint (or mutually exclusive).

Formal Probability (cont.)

4. Addition Rule (cont.):– For two disjoint events A and B, the

probability that one or the other occurs is the sum of the probabilities of the two events.

– P(A or B) = P(A) + P(B), provided that A and B are disjoint.

For more events: P(A or B or C) = P(A) + P(B) + P(C), where A, B

and C are mutually exclusive

Example: M&M• The Masterfoods company, the manufacturers of M&M’s milk

chocolate candies, says that before 2001, yellow candies made 20% of their plain M&M’s, red another 20%, and orange, blue, and green each made up 10%. The rest were brown.– If you pick one M&M at random, what is the probability it is yellow or

orange?

Example

• The probability of owning an MP3 player is 0.50 and the probability of owning a computer is 0.90. What is the probability of owning either an MP3 player or a computer?

Be careful about natural language!

– Often, “or” in our natural language has an exclusive meaning as in “Would you like the steak or the vegetarian entrée?”.

– In this class, when we ask for the probability that A or B occurs, we mean A or B or both.

– P(A or B but not both) = P(A or B) – P(A and B) = P(A) + P(B) – 2 * P(A and B)

The General Addition Rule (cont.)

• General Addition Rule:– For any two events A and B,

P(A or B) = P(A) + P(B) – P(A and B) – P(A or B but not both) = P(A or B) – P(A and B)

= P(A) + P(B) – 2 * P(A and B)

General addition rule

For three events:

P(A or B or C) = P(A) + P(B) + P(C)- P(A and B) – P(A and C) – P(B and C) + P(A and B and C)

Formal Probability

5. Multiplication Rule (cont.):– For two independent events A and B, the

probability that both A and B occur is the product of the probabilities of the two events.

– P(A and B) = P(A) x P(B), provided that A and B are independent.

– P(A and B and C) = P(A) x P(B) x P(C) , if A and B and C are mutually independent

Formal Probability (cont.)

5. Multiplication Rule (cont.):– Two independent events A and B are not

disjoint, provided the two events have probabilities greater than zero:

Multiplication rule

• For two independent events A and B, the probability that both A and B occur is the product of the probabilities of the two events.

P(A and B) = P(A) * P(B), if A and B are independent

• For more events, P(A and B and C) = P(A) * P(B) * P(C) , if A and B

and C are mutually independent

Example: M&M

• In 2001, Masterfoods decided to add another color to the standard color lineup of brown, yellow, red, orange, blue and green. To decide which color, they surveyed kids in nearly every country and asked them to vote for purple, pink and teal.

The global winner is purple!

Example: M&M

• In Japan, the result is somehow different. – 38% for pink– 36% for teal– 16% purple

• What is the probability that a Japanese M&M’s survey respondent selected at random preferred either pink or teal?

Another question

• If we pick two respondents at random, what’s the probability that they both said pink or teal?

Another question

• If we pick three respondents at random, what’s the probability that at least one preferred purple?

What can go wrong?

• Be aware of probabilities that don’t add up to 1.

• Don’t add probabilities of events if they are NOT disjoint.

• Don’t multiply probabilities of events if they are NOT independent.

• Independent ≠ Disjoint

Independent ≠ Disjoint• Disjoint events cannot be independent! Well, why not?

– Since we know that disjoint events have no outcomes in common, knowing that one occurred means the other didn’t.

– Thus, the probability of the second occurring changed based on our knowledge that the first occurred.

– It follows, then, that the two events are not independent.• A common error is to treat disjoint events as if they

were independent, and apply the Multiplication Rule for independent events—don’t make that mistake.

Example

• Police report that 78% of drivers stopped on suspicion of drunk driving are given a breath test, 36% a blood test, and 22% both tests. What is the probability that a randomly selected suspect is given– a blood test or a breath test?– neither test?

• A = {the suspect is given a breath test}• B = {the suspect is given a blood test}• We know that

– P(A) = 0.78– P(B) = 0.36– P(A and B) = 0.22

• P(A or B) = P(A) + P(B) – P(A and B) =0.78 + 0.36 – 0.22 = 0.92

• P(A or B but NOT both) – P(A or B) – P(A and B) = 0.92 – 0.22 = 0.70– P(A or Bc) + P(B or Ac)

• P(Ac and Bc) = 1 – P(A or B)

Checking disjoint

• Are giving a suspect a blood test and a breath test mutually exclusive?– This is to see whether P(A and B) = 0– In this case, P(A and B) = 0.22– So, not mutually exclusive

Checking independence

• Are giving a suspect a blood test and a breath test independent? – P(A and B) = P(A) * P(B)?

Example

• Two psychologists surveyed 478 children. • They asked the students whether their

primary goal was – to get good grades– to be popular– or to be good at sports

• Purpose of the study– Did boys and girls at this age have similar

goals?

grades popular sports Total

Boy 117 50 60 227

Girl 130 91 30 251

Total 247 141 90 478

Conditional distributions

grades popular sports Total

Girl 130 (51.79%)

91 (36.25%)

30 (11.95%)

251

grades popular sports TotalBoy 117

(51.54%)50 (22.03%)

60 (26.43%)

227

Conditional probability

• When we want the probability of an event from a conditional distribution, we write P(B|A) and pronounce it “the probability of B given A.”

• A probability that takes into account a given condition is called a conditional probability.

• To find the probability of the event B given the event A, we restrict our attention to the outcomes in A. We then find in what fraction of those outcomes B also occurred.

• Note: P(A) cannot equal 0, since we know that A has occurred.

(A and B)( | )( )

PPP

B AA

The General Multiplication Rule

• We encountered the general multiplication rule in the form of conditional probability.

• Rearranging the equation in the definition for conditional probability, we get the General Multiplication Rule:– For any two events A and B,

P(A and B) = P(A) x P(B|A) or

P(A and B) = P(B) x P(A|B)

Independence

• Independence of two events means that the outcome of one event does not influence the probability of the other.– Events A and B are independent whenever P(B|A)

= P(B). (Equivalently, events A and B are independent whenever P(A|B) = P(A).)

– P(A and B) = P(A) * P(B)

P(A and B) = P(A) * P(B|A) = P(B) * P(A|B) = P(A) * P(B)

• Sample space: primary goals of the set of 478 students

• Trial: select a student at random (equally likely)

• What is the probability the selected student is a girl? (251/478 = 0.525)

• P(girl and popular) = ?• P(sports) = 90/478 = 0.188

Conditional probability

• Given that the selected student is a girl,– what is the probability the selected student’s

goal is sports?– P(sports | girl) = P(sports and girl) / P(girl) = 30 / 251 = 0.120– P(sports | boy) = 60/227 = 0.264– P(sports) ≠ P(sports | girl) + P(sports | boy) – P(girl|sports) = ?

Example

• Police report that 78% of drivers stopped on suspicion of drunk driving are given a breath test, 36% a blood test, and 22% both tests.

• Are giving a DWI suspect a blood test and a breath test independent? – That is, to check whether P(B|A) = P(B)– P(B|A) = P(A and B) / P(A) = 0.22 / 0.78 = 0.28– P(B) = 0.36– P(B|A) ≠ P(B), not independent

Drawing Without Replacement

• Sampling without replacement means that once one object is drawn it doesn’t go back into the pool. – We often sample without replacement, which doesn’t

matter too much when we are dealing with a large population.

– However, when drawing from a small population, we need to take note and adjust probabilities accordingly.

• Drawing without replacement is just another instance of working with conditional probabilities.

Drawing without replacement

• Assign dormitory rooms to students• Suppose that 12 rooms left when it is time

for you and your friend to draw– Three are in Gold Hall– Four in Silver Hall– Five in Wood Hall

• What is the probability that both of you get rooms in Gold Hall?

• Key point: after one room is drawn, that room can not be considered for the next draw– Drawing without replacement

• P(you get Gold & your friend gets Gold) = P(you get Gold)* P(friend gets Gold|you get Gold) = (3/12)* (2/11) = 1/22

Example: Binge drinking

• A study by Harvard School of Public Health, for college students– 44% engage in binge drinking– 37% drink moderately– 19% abstain entirely

• Meanwhile, another study– Among binge drinkers, 17% involved in alcohol-related automobile

accident– Among students who drink moderately , only 9% have been

involved• If we know a student has had an alcohol-related accident,

what is the probability that the student is a binge drinker?

Tree Diagrams

• A tree diagram helps us think through conditional probabilities by showing sequences of events as paths that look like branches of a tree.

Reversing the conditioning

• If we know a student has had an alcohol-related accident, what is the probability that the student is a binge drinker?– P(binge | accident) = ?– P(binge and accident) / P(accident) – P(accident) = P(binge and accident) +

P(moderate and accident) + P(abstain and accident) = 0.075+ 0.033 + 0 = 0.108

Bayes’ rule• If we know P(A|B), but we want to have P(B|A), how do we do

the calculation?

• P(B|A) = P(A and B)/P(A) = P(A|B)*P(B)/P(A)

• P(A) = P(A and B) + P(A and Bc) = P(A|B)*P(B) + P(A|Bc)*P(Bc)

• P(B|A) = P(A|B)*P(B)/[P(A|B)*P(B) + P(A| Bc)*P(Bc)]

• We need P(B), P(A|B) and P(A| Bc)

What can go wrong?

• Be aware of probabilities that don’t add up to 1.

• Don’t add probabilities of events if they are NOT disjoint.

• Don’t multiply probabilities of events if they are NOT independent.

• Independent ≠ Disjoint