Probability Theory - POLI 270 - Mathematical and ...pages.ucsd.edu/~ssaiegh/Slides7.pdf ·...

ProbabilityProbability in Simple Sample Spaces

Conditional Probability

Probability TheoryPOLI 270 - Mathematical and Statistical Foundations

Sebastian M. Saiegh

Department of Political ScienceUniversity California, San Diego

November 11, 2010

Sebastian M. Saiegh Probability Theory

Introduction to Probability Theory

1 ProbabilitySome Background

2 Probability in Simple Sample SpacesSample Space and EventsProbability of an EventCounting Techniques

3 Conditional ProbabilityConditional and Compound ProbabilitiesBayes’ TheoremIndependent Events

Conditional ProbabilitySome Background

Probability Theory

Lady Luck

The radical concept of replacing randomness with systematicprobability and its implicit suggestion that the future might bepredictable and even controllable to some degree, is whatseparates the modern times and the past.

Yet, probability theory was not developed until the XVIIcentury.

And, it was a result of the efforts to analyze games of chance!

Lady Luck

Lady Luck (cont.)

The game of craps provides a useful illustration of therelationship between gambling and the laws of probability:

Throwing a pair of six-sided dice will produce, not eleven (fromtwo to twelve), but thirty-six possible combinations, all theway from snake eyes (two ones) to box cars (double six).

There is only one way for each of double-one and double-six toappear. While seven, the key number in craps, is the easiest tothrow. There are six different ways to arrive at seven. Theprobability of a seven-throw is 6

36 or 16.66%.

Lady Luck (cont.)

36 or 16.66%.

Lady Luck (cont.)

36 or 16.66%.

Lady Luck (cont.)

The probability of an outcome is the ratio of favorableoutcomes to the total opportunity set.

The odds on an outcome are the ratio of favorable outcomesto unfavorable outcomes.

Lady Luck (cont.)

The probability of an outcome is the ratio of favorableoutcomes to the total opportunity set.

The odds on an outcome are the ratio of favorable outcomesto unfavorable outcomes.

Lady Luck (cont.)

The odds obviously depend on the probability, but the oddsare what matter when you are placing a bet:

If the probability of throwing a 7 in craps is one out of sixthrows, the odds on throwing a number other than 7 are 5 to 1.This means that you should bet no more than $1 that 7 willcome up the next throw when the other guy bets $5 that itwill not.

Games of chance must be distinguished from games in whichskill makes a difference. The odds are all you know for bettingin a game of chance.

Lady Luck (cont.)

Gambling in the Dark

Gambling has been a popular pastime and often an addictionsince the beginning of recorded history. However, until theXVII century, people had wagered and played games of chancewithout using any system of odds to determine winnings andlosings.

The Greeks understood that more things might happen in thefuture than actually will happen.

In fact, the word, εικoς, meaning plausible or probable, hadthe same sense as the modern concept of probability: “to beexpected with some degree of certainty.”

However, they never made any advances in measuring thisdegree of certainty.

Gambling in the Dark (cont.)

When the Greeks wanted a prediction about what tomorrowmight bring, they turned to the oracles instead of consultingtheir wisest philosophers.

Up to the time of the Renaissance, people perceived the futureas little more than a matter of luck or the result of randomvariations, and most of their decisions were driven by instinct.

The inappropriateness of the numbering system was importantreason of why Europeans were not induced to explore themastery of risk.

Without numbers, there are no odds and no probabilities;without odds and probabilities, the only way to deal with riskis to appeal to the gods and the fates.

It is hard for us to imagine a time without numbers. However,if we were able to bring a well-educated man from the year1000 to the present, he probably would not recognize thenumber zero and would surely flunk third-grade arithmetic;few people from the year 1500 would fare much better.

The story of numbers in the West begins in 1202, when abook titled Liber Abaci, or Book of Abacus, appeared in Italy.The author, Leonardo Pisano, was known for most of his lifeas Fibonacci (yes, the “Fibonacci series”).

It took almost three hundred years, though, for theHindu-Arabic numbering system to be widely adopted in theWestern world.

In 1494, a Franciscan monk named Luca Paccioli published hisSumma de arithmetic, geometria et proportionalita. He posedthe following problem in the book:

A and B are playing a fair game of balla. They agree tocontinue until one has won six rounds. The game actuallystops when A has won five and B three. How should thestakes be divided?

The puzzle, which came to be known as the problem of thepoints, was more significant than it appears. The resolution ofhow to divide the stakes in an uncompleted game marked thebeginning of a systematic analysis of probability.

Non-Degenerate Gamblers

In 1654, the Chevalier de Mere, a French nobleman with a taste forboth gambling and mathematics, challenged the famed Frenchmathematician Blaise Pascal to solve the puzzle.

Pascal turned for help to Pierre de Fermat, a lawyer who wasalso a brilliant mathematician.Their collaboration led to the discovery of the theory ofprobability.

Non-Degenerate Gamblers

In 1654, the Chevalier de Mere, a French nobleman with a taste forboth gambling and mathematics, challenged the famed Frenchmathematician Blaise Pascal to solve the puzzle.

Pascal turned for help to Pierre de Fermat, a lawyer who wasalso a brilliant mathematician.Their collaboration led to the discovery of the theory ofprobability.

Non-Degenerate Gamblers (cont.)

Given that more things can happen than will happen, Pascaland Fermat established a procedure for determining thelikelihood of each of the possible results – assuming alwaysthat the outcomes can be measured mathematically.

As the years passed, mathematicians transformed probabilitytheory from a gamblers’ toy into a powerful instrument fororganizing, interpreting, and applying information.

In particular, they showed how to infer previously unknownprobabilities from the empirical facts of reality.

In 1703, Gottfried von Leibniz commented to the Swiss scientistand mathematician Jacob Bernoulli that “[N]ature has establishedpatterns originating in the return of events, but only for the mostpart,” thereby prompting Bernoulli to invent the Law of LargeNumbers.

In 1730, Abraham de Moivre suggested the structure of thenormal distribution – also known as the bell curve – anddiscovered the concept of standard deviation.

In 1703, Gottfried von Leibniz commented to the Swiss scientistand mathematician Jacob Bernoulli that “[N]ature has establishedpatterns originating in the return of events, but only for the mostpart,” thereby prompting Bernoulli to invent the Law of LargeNumbers.

In 1730, Abraham de Moivre suggested the structure of thenormal distribution – also known as the bell curve – anddiscovered the concept of standard deviation.

Around 1760, a dissident English minister named ThomasBayes made a striking advance in statistics by demonstratinghow to make better-informed decisions by mathematicallyblending new information into old information.

With some exceptions, all the tools we use today in theanalysis of decisions and choice, from the strict rationality ofgame theory to the challenges of chaos theory, stem most fromthe developments that took place between 1654 and 1760.

Around 1760, a dissident English minister named ThomasBayes made a striking advance in statistics by demonstratinghow to make better-informed decisions by mathematicallyblending new information into old information.

With some exceptions, all the tools we use today in theanalysis of decisions and choice, from the strict rationality ofgame theory to the challenges of chaos theory, stem most fromthe developments that took place between 1654 and 1760.

Today’s Class

In today’s class we will focus the ideas developed by theseremarkable thinkers.

Then, we will apply the calculus of probabilities to a widevariety of situations requiring the use of sophisticatedcounting techniques.

We shall often refer to games of chance. Yet these gameshave applications that extend far beyond the spin of theroulette wheel.

Today’s Class

Sample Space and EventsProbability of an EventCounting Techniques

Probability Theory

Introduction

In everyday life, we often talk loosely about chance. What are thechances of getting a job? of meeting someone? of rain tomorrow?But for the purposes of this class, we need to give the word chancea definite, clear interpretation.This turns out to be hard, and, mathematicians have struggledwith the job for centuries. We will now focus on the frequencytheory, which works best for processes which can be repeated overand over again, independently and under the same conditions.Games of chance fail into category. And, as we already know, muchof the frequency theory was developed to solve gambling problems.

Introduction

Tossing Coins

One of the simplest games of chance involves betting on the tossof a coin.

If a coin is tossed in the air, then it is certain that the coinwill come down, but it is not certain that, say, it will come up“head”.

When trying to figure chances, it is usually very helpful to listall the possible ways that a chance process can turn out.

In the case of a coin toss, we ordinarily agree to regard“head” and “tail” as the only possible outcomes.

If we denote these outcomes by H and T respectively, theneach outcome would correspond to exactly one of theelements of the set H,T.

Tossing Coins

Sample Space

Suppose now that we repeat this experiment of tossing a coin.

The process of tossing the coin can certainly be repeated overand over again, independently and under the same conditions:the outcomes of this experiment would still correspond toexactly one of the elements of the set H,T.This set, Ω = H,T is called a sample space for theexperiment.

Suppose now that we toss a die and observe the number thatappears on top. Then the sample space consist of the six possiblenumbers: Ω = 1, 2, 3, 4, 5, 6.

Sample Space

Sample Space (cont.)

Definition

The set Ω of all possible outcomes associated with a real orconceptual experiment is called the sample space. A particularoutcome ω, i.e. an element in Ω, is called a sample point orsample.

Tossing Coins, Again

Suppose now that we perform the following experiment: we toss acoin three successive times.Let Ω = HHH,HHT ,HTH,THH,HTT ,THT ,TTH,TTT bethe associated sample space.

We may be interested in the event, “the number of headsexceeds the number of tails”.

For any outcome of the experiment we can determine whetherthis event does or does not occur.

We find that HHH, HHT ,HTH and THH are the onlyelements of Ω corresponding to outcomes for which this eventdoes occur.

Events

To say that the event “the number of heads exceeds the number oftails” occurs is the same as saying the experiment results in anoutcome corresponding to an element of the setA = HHH,HHT ,HTH,THH.

Notice that A is a subset of the sample space Ω.

Definition

An event A is a set of outcomes or, in other words, a subset ofsome underlying sample space Ω.

Events

Definition

Events

Definition

Events (cont.)

The event ω consisting of a single sample ω ∈ Ω is calledan elementary event.

The certain (or sure) event, which always occurs regardless ofthe outcome of the experiment, is formally identical with thewhole space Ω.

The impossible event is the empty set ∅, containing none ofthe elementary events ω.

Events (cont.)

We can combine events to form new events using the various setoperations:

(i) A1 ∪A2 is the event that occurs iff A1 occurs or A2 occurs (orboth);

(ii) A1 ∩ A2 is the event that occurs iff A1 occurs and A2 occurs;

(iii) A ′ = A−Ω, the complement of A, is the event that occurs iffA does not occur.

Events (cont.)

Given two events A1 and A2, suppose A1 occurs if and only if A2

occurs.

Then, A1 and A2 are said to be identical (or equivalent), andwe write A1 = A2.

Two events A1 and A2 are called mutually exclusive if they aredisjoint, i.e. if A1 ∩ A2 = ∅.

In other words, A1 and A2 are mutually exclusive if theycannot occur simultaneously.

Events (cont.)

occurs.

Events (cont.)

occurs.

Events (cont.)

occurs.

Events: Example

Example

Suppose that we toss a die and observe the number that appearson top. The sample space consist of the six possible numbers:Ω = 1, 2, 3, 4, 5, 6. Let A1 be the event that an even numberoccurs, A2 that an odd number occurs, and A3 that a numberhigher than 3 occurs.