Introduction to Probability

ASW, Chapter 4Skip section 4.5

September 17 and 22, 2008

Why study probability

Inferential statistics – inferences about population from:– Sample – Experiment (or random experiment – ASW, 149)

Economic models are often probabilisticQuantifying uncertainty for purposes of:

– Prediction – Assessing preferred course of action

Definition of probabilityProbability is a numerical measure of the

likelihood that an event will occur (ASW, 142).

Notation: The probability of an event E is written P(E) and pronounced “P of E”

Scale: 0 ≤ P(E) ≤ 1If P(E) = 0, event E cannot occurIf P(E) = 1, event E is certain to occur

Terms for probability

• Probability• Likelihood• Odds• Chances• Risk or hazard• Random selection• Random or stochastic processes

Experiments and outcomes

An experiment is a process that generates well-defined outcomes (ASW, 142) – Tossing a coin– Rolling a single die or a pair of dice– Applying for a job– Salary expected after completing a degree– Drilling an oil well– Federal election

ASW note that these are sometimes termed random experiments (150).

Assigning probabilities• Classical or theoretical method (ASW, 146)• Relative frequency method (ASW, 148)

– Data and experiments• Subjective or judgment method (ASW, 148). A

degree of belief that an outcome will occur– Best guess / experience / wise judgment produces

good estimates of the chance of a particular outcome occurring

– Poor guess / poor judgment produces unreliable estimates of the chance of a particular outcome occurring

Classical methodExperiment (eg. Flip of a coin – outcome of H or T)• Exact outcome is unknown before conducting

experiment• All possible outcomes of experiment are known• Each outcome is equally likely• Experiment can be repeated under uniform

conditionsTogether these conditions produce regularities or

patterns in outcomes

Examples of classical method• P (2 heads in two tosses of a coin) = 1/4 = 0.250• P (obtain 4 in roll of one die) = 1/6 = 0.167• P (total of 4 when rolling two dice) = 3/36 = 1/12 =

0.083If one randomly selects individuals from a large

population that is ½ male and ½ female: • P (one male and one female if 2 individuals selected)

= 2/4 = 1/2 = 0.500• P (two females and one male if 3 individuals

selected) = 3/8 = 0.375

Relative frequency method

This method used for an experiment where it is not possible to apply the classical approach (usually because outcomes not equally likely or the experiment is not repeatable under uniform conditions).

The probability of an event E is the relative frequency of occurrence of E or the proportion of times E occurs in a large number of trials of the experiment.

Example of relative frequency method

Party supported

Per cent support

Conservative 38

Liberal 27

NDP 17

Green 9

Bloc Québécois

8

If I meet an individual Canadian and I know nothing about this person, the best estimate of the probabilities that the person supports the various parties are:

P (individual is a Conservative supporter) = 0.38

P (individual supports the Green Party) = 0.09

Source: http://www.theglobeandmail.com/national/politics/, 12:15 p.m., Sept. 17, 2008

Subjective or judgment method• Make your best guess of the likelihood of the event,

based on reliable information and good judgment.• Inform yourself about the issue or discuss it with

someone knowledgeable.• Probability is a number between 0 and 1 that

represents your degree of belief the event will occur.• P(E) close to 0 implies a judgment that the event is

unlikely to occur.• P(E) close to 1 implies a judgment that the event is

quite likely to occur.

Examples of subjective method – my subjective judgment

• P (UR Rams will defeat the SFU Clan this Saturday) = 0.7

• P (S & P TSX index will increase in trading on Monday) = 0.3

• P (I will have an auto accident during the next 6 months) = 0.02

• P (some snow will fall in Regina before the end of October) = 0.97

Which method to use?• Whenever it is possible to apply it, use the

classical method.• When dealing with economic activity, the

classical method often cannot be used (not equally likely events and not repeatable). Obtain data and use the relative frequency method.

• If neither of these is possible, use the subjective method. It is a weak method, but if judgments are sound, these subjective probabilities can be used in coordination with economic theory and more soundly based probabilities.

Sample space (ASW, 141)• An experiment is a process that generates well-

defined outcomes.• The sample space, S, for an experiment is the set

of all experimental outcomes.• Each of these outcomes is referred to as a sample

point. For an experiment with n sample points, label these E1, E2, E3, ... , En.

• The sample space of an experiment is the set of all sample points, ie. S = {E1, E2, E3, ... , En}.

Two requirements for assigning probabilities

• If Ei is the ith outcome of an experiment, then its probability is no less than zero and no greater than 1. That is, for all sample points or outcomes, Ei, 0 ≤ P(Ei) ≤ 1, for i = 1, 2, ... n.

• The sum of the probabilities of all the outcomes equals 1, that is, P(E1) + P(E2) + P(E3) + ... + P(En) = 1 or

1)(1

ni

i iEP

Examples of sample spaces• Flip a coin (n = 2). S = {H, T}, where H = head, T= tail.

P(H) = 1/2 = 0.50 and P(T) = 1/2 = 0.50. P(H) + P(T) = 0.50 + 0.50 = 1.

• One roll of a ten-sided die (n = 10). S = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}.

Probability of each side occurring is 1/10.P(0) + P(1) + P(2) + ... + P(9) =1.

• Party that will form the government following the Canadian federal election. S = {C, L, N, G, B} where C = Conservative, L = Liberal, N = NDP, G = Green, B = Bloc.

What are P(C), P(L), P (N), P(G), P(B)? P(C) + P(L) + P(N) + P(G) + P(B) = 1.

Multiple-step experiments• If an experiment with k steps has n1 possible

outcomes on the first step, n2 possible outcomes on the second step, etc., then the sample space has n1 × n2 × n3 × ... × nk sample points (ASW, 143).

• If 3 persons are randomly selected from a large population with half females and half males, there are 2 × 2 × 2 = 8 outcomes for the sex of the persons selected. Each outcome is equally likely.

•

•

••

•

•

(F, F, F)

F

F

F

F

F

F

F

M

(F, F, M)

(F, M, F)

(F, M, M)

(M, F, F)

(M, F, M)

(M, M, F)

(M, M, M)

First selection

Second selection

Third selection

M

M

M

M

M

M

Sample points Probability

1/8 = 0.125

1/8 = 0.125

1/8 = 0.125

1/8 = 0.125

1/8 = 0.125

1/8 = 0.125

1/8 = 0.125

1/8 = 0.125

Sum = 1.000

Selection of 3 persons from a large population of half females and half males.

•

Event• An event is a collection of sample points (ASW,

152).• The probability of any event is equal to the sum

of the probabilities of the sample points in the event (ASW, 153).

• Event of selecting exactly two females (2F).P (2F) = 1/8 + 1/8 + 1/8 = 3/8 = 0.375.This is the sum of the probabilities of the events FFM, FMF, and MFF.

Results from Econ 224 survey• Sample of N = 46 students.• Step one – results organized into four groups by

major: Math (including Actuarial and Statistics), Business, Economics, and Other.

• Step two – results organized into four groups of experience level with Excel – None (N), Low (L), Medium (M), or High (H).

• Number of students reporting each characteristic reported in a tree diagram.

Step one: Major

Step Two:Excel Skill

Outcome:

N = 46

Math(6)

N = 0

L = 2

M = 4

H = 0

MAJORBusiness

(13)

N = 1

L = 3

M = 6

H = 3

Economics(24)

N = 2

L = 12

M = 8

H = 2

Other(3)

N = 0

L = 1

M = 1

H = 1

Probabilities for Excel skill levelsIf each outcome equally likely, P(E) = NE/N where NE

is the number of outcomes in event E and N is the total number of outcomes.

Probability of selecting a student with low Excel skills = (2 + 3 + 12 +1)/46 = 18/46 = 0.391

P(High Excel skills) = 6/46 = 0.130P(No Excel experience) = 3/46 = 0.065P(Medium Excel experience) = 19/46 = 0.413Note that 0.065 + 0.391 + 0.413 + 0.130 = 0.999

Relationships of Probability

• Complement of an event• Addition law – intersection and union• Mutually exclusive• Conditional probability• Independence• Multiplication law

Complement of an event• The complement Ac of an event A is the set of

all sample points that are not in event A.• P(A) + P(Ac) = 1 or P(Ac) = 1 – P(A).

• Examples: The probability that a student in this class is not an Economics major (E) is 22/46.

P(E) = 24/46 = 0.522P(Ec) = 1 – 0.522 = 0.478

P(Excel skills not High) = 1 – 0.130 = 0.870

Combining events (ASW, 157-160)

What is the probability that more than one event has occurred? If there are two events:

Both events could occur – this is referred to as the intersection of the two events.

At least one of the events could occur – this is referred to as the union of the two events.

Neither event occurs.

Union of two events• The union of events A and B is the event containing

all the sample points of either A or B, or both (ASW, 157).

• The notation for the union is P(AB).• Read this as “probability of A union B” or the

“probability of A or B.”• The probability of A or B is the sum of the

probabilities of all the sample points that are in either A or B, making sure that none are counted twice.

Intersection of two events• The intersection of events A and B is the event

containing only the sample points belonging to both A and B (ASW, 158).

• The notation for the intersection is P(AB).• Read this as “probability of A intersection B” or

the “probability of A and B.”• The probability of A and B is the sum of the

probabilities of all the sample points common to both A and B.

Addition law (ASW, 158)

P(AB) = P(A) + P(B) – P(AB)

The probability that at least one event occurs is the probability of one event plus the probability of the other. But to avoid double counting, the probability of the intersection of the two events is subtracted.

Examples• Probability of a randomly selected student being an

Economics major (E) or having high Excel skills (H)? P(EH) = P(E) + P(H) – P(EH)

= 24/46 + 6/46 – 2/46 = 28/46 = 0.609

• Probability of Business (B) or low skills (L)? P(BL) = P(B) + P(L) – P(BL)

= 28/46 = 0.609• P(Other or medium) = 0.456

Mutually exclusive events (ASW, 160)• Two events are mutually exclusive if the events

have no sample points in common. • If two events A and B are mutually exclusive,

the probability of A and B is zero. • In this case, the probability of A or B is the sum

of the probability of A and the probability of B. That is, P(AB) = 0 if A and B are mutually exclusive. Then P(AB) = P(A) + P(B)

Example of mutually exclusive events If 3 persons are randomly selected from a large

population of half females and half males, what is the probability of selecting 3 females (A) and at least 2 males (B)? Event A has sample point (F, F, F) .Event B has sample points (F, M, F), (M, F, M), (M, M, F) and (M, M, M) .Events A and B have no sample points in common, so P(AB) = 0.

Conditional probabilities (ASW, 162-167)

• A conditional probability refers to the probability of an event A occurring, given that another event B has occurred.

• Notation: P(A B)• Read this as the “conditional probability of A

given B” or the “probability of A given B.“• These are especially useful in economic analysis

because probabilities of an event differ, depending on other events occurring.

Formulae for conditional probabilities

• The probability of A given B is

• The probability of B given A is

)(

)()(

BP

BAPBAP

)(

)()(

AP

BAPABP

Number of students by major and Excel skill level

Major of student

Excel skill level Total

None (N) Low (L) Medium (M) High (H)

Math (MA) 0 2 4 0 6

Business (B) 1 3 6 3 13

Economics (E) 2 12 8 2 24

Other (O) 0 1 1 1 3

Total 3 18 19 6 46

This table contains the same data as examined earlier, but reorganized as a table rather than in a tree diagram.

Examples of conditional probabilities from student survey

• Probability that each major has low skill level?P(L MA) = P(L MA) / P(MA) = (2/46) / (6/46) = 2/6 = 0.333P(L B) = 3 / 13 = 0.231P(L E) = 0.500P(L O) = 0.333

If a student has a high skill level is Excel, what is the probability his or her major is Business? Other?P(B H) = P(B H) / P(H) = (3/46) / (6/46) = 3/6 = 0.500P(O H) = 0.167

Next day

• Independence (ASW, 166)• Multiplication law (ASW, 166)• Random variables (ASW, 186)• Discrete probability distributions (ASW, 189)• Expected value – mean and variance (ASW,

195)• Binomial probability distribution (ASW, 199)