ENGG2450B Probability and Statistics for...

ENGG2450B Probability and Statistics for EngineersENGG2450B Probability and Statistics for Engineers

1 Introduction3 Probability4 Probability distributions5 Probability densities2 Organization and description of datag6 Sampling distributions7 Inferences concerning a mean7 Inferences concerning a mean8 Comparing two treatments9 Inferences concerning variances9 Inferences concerning variancesA Random Processes

1 Introduction

1.1 Why study statistics?

1 Introduction3 Probability4 Probability distributions5 Probability densities2 Organization & descriptiony y

1.2 Modern Statistics2 Organization & description6 Sampling distributions7 Inferences .. mean8 Comparing 2 treatments9 Inferences .. variancesA Random processes

1.3 Statistics and engineering

1 4 The role of the scientist and engineer in

A Random processes

1.4 The role of the scientist and engineer in quality improvement

1.6 Two basic concepts – population and samplep

Statistics is the study of the collection analysis interpretation

1.1 Why study statistics? (3)

Statistics is the study of the collection, analysis, interpretation, presentation, and organization of data in an investigation.

Four steps in an investigation.

1. Set clearly defined goals for the investigation.

2. Make a plan of what data to collect and how to collect.

3. Apply appropriate statistical methods to extract information pp y pp pfrom the data.

4 Interpret the information and draw conclusions4. Interpret the information and draw conclusions.

Statistics is a vital tool to engineers

1.3 Statistics and engineering (4)

Statistics is a vital tool to engineers.

It enables engineers to understand phenomena subject to variation and to effectively predict or control them.

1.4 The role of the scientist and engineer in quality improvement

The world market is increasingly competitive.That has foster a revolution in improving product qualityThat has foster a revolution in improving product quality.Quality improvement is based on the philosophy of ‘make it right the first time’.right the first time .Techniques to improve quality relates closely to statistical techniques.q

1.6 Two basic concepts – population and sample (5)

In statistics, the term population is applied to any set (collection) of objects, actual or conceptual, finite or infinite.

E.G. All students currently enrolled in CUHK.

This population is actual and finite

E.G. The set of positive integers is a population.

This population is actual and finite.

This population is conceptual and infinite.

A sample is a part of the population.

1.2 Modern Statistics (6)

Two main statistical methodologies are used in data analysis:

descriptive statisticsdescriptive statistics

It summarizes data from a sample using indexes such as the mean, median, standard deviation, and graphs , , , g plike bar charts.

inferential statistics

It draws conclusions and make predictions from sample dataIt draws conclusions and make predictions from sample data that are subject to random variation (e.g., observational errors, sampling variation). g )

3 Probability(7)

3.1 Sample spaces and events

1 Introduction3 Probability4 Probability distributions5 Probability densities2 Organization & descriptionp p

3.2 Counting2 Organization & description6 Sampling distributions7 Inferences .. mean8 Comparing 2 treatments9 Inferences .. variancesA Random processes

3.3 Probability

3 4 The Axioms of Probability

A Random processes

3.4 The Axioms of Probability

3.5 Some Elementary Theorems y

3.6 Conditional Probability

3.7 Bayes’ Theorem

3.1 Sample spaces and events (8)

Casino game sic bo 骰寶, also known as tai sai 大細


We shall use the case when only one dice is thrown once as an example.

Sample space is the set of all possible outcomes.E.G. Sample space is the set {1, 2, 3, 4, 5, 6}.

Sample space can be discrete or continuousSample space can be discrete or continuous. E.G. Sample space {1, 2, 3, 4, 5, 6} is discrete.E G Sample space { 1 number 6 } is continuous

A i t i f t i l d th b ti f

E.G. Sample space { 1 number 6 } is continuous.

An experiment is a process of trial and then observation of the outcome.E G Throwing the dice once is an experiment WeE.G. Throwing the dice once is an experiment. We

observe that the outcome of the experiment is 1.


We shall use the case when only one dice is thrown once as an example.

Any subset of a sample space is an eventAny subset of a sample space is an event.

E.G. The event of odd numbers is the set {1, 3, 5}.

E G The event of even numbers is the set {2 4 6}E.G. The event of even numbers is the set {2, 4, 6}.

E.G. The event of large numbers is the set {4, 5, 6}.

E.G. The event of small numbers is the set {1, 2, 3}.

How to determine the number of elements in a finite3.2 Counting (11)

How to determine the number of elements in a finite sample space?

Th 1 If t A A A t i ti lTheorem 1: If sets A1, A2, ..., Ak contains, respectively n1, n2, ..., nk elements, then there are n1, n2, ..., nkways of choosing first an element of A then anways of choosing first an element of A1, then an element A2, ..., and finally an element of Ak.

E.G. The sample space of

throwing 3 dices is the set

{ (1,1,1), (1,1,2), (1,1,3),...(6,6,6) }.

The number of elements in this sample space is

{ ( ) ( ) ( ) ( ) }

6 x 6 x 6 = 216.

3.2 Counting (12)

Theorem 2: The number of permutations of r objectsTheorem 2: The number of permutations of r objects selected from a set of n distinct objects is

P ( 1) ( 2) ( 3) ( +1) !nPn P r = n (n-1) (n-2) (n-3) ... (n-r+1) .)!(

!rn

nrPn

♦

E.G. Consider a deck of 52 cards. We are to draw 4 cards consecutively from this deck. What is the number of permutations of ♣A

♦♥♣A♠A

♦What is the number of permutations of selecting these 4 objects from this set of 52 distinct objects?

♠A♥♣A

♣A♥♠A

Sln The number of permutations

♠A♥♣A♦

Sln. The number of permutations

= n (n-1) (n-2) (n-3)

♦♣A♥♠K

= 52 x 51 x 50 x 49 = 6,497,400 ♠2 ♥♣2♦

3.2 Counting (13)

Theorem 3: The number of ways in which r objects canTheorem 3: The number of ways in which r objects can be selected from a set of n distinct objects is

!nnC

.

)!(! rn

nrnrCn

r!

E.G. Consider a deck of 52 cards. We are to draw 4 cards consecutively from this ♦♣A♥

♦♥♣A♠A

y

deck. What is the number of ways of selecting these 4 objects from this set of 52 distinct objects?

♦♣A♥♠A

♠A♥♣A♦

52 distinct objects?

Sln. The number of ways

♥♣A♦

= n (n-1) (n-2) (n-3)/ 4!

= 52 x 51 x 50 x 49/ 4x3x2

♦♣A♥♠K 52 x 51 x 50 x 49/ 4x3x2

= ♠2 ♥♣2♦

3.2 Counting (14)

E.G. In how many different ways can the director of a research laboratory choose 2 chemists from among 7

li t d 3 h i i t f 9 li t ?applicants and 3 physicists from among 9 applicants?

Sln The 2 chemists can be chosen in C = 21 waysSln. The 2 chemists can be chosen in 7C2 = 21 ways.

The 3 physicists can be chosen in 9C3 = 84 ways.

The whole selection can be made in

p y 9 3 y

21 x 84 = 1,764 ways.

3.3 Probability (15)

The classical probability concept:If there are m equally likely possibilities, of which one must occur and s are regarded as favorable, or as a “success,” then the probability of a “success” is given by .

msm

E.G. Consider a dice is thrown once. What is the probability of having an odd number?

sln. There are 6 equally likely possibilities {1, 2, 3, 4, 5, 6} of which one must occur 3 of them which are odd numberswhich one must occur. 3 of them, which are odd numbers1, 3, 5, are regarded as “success”.

The probability of having an odd number is 3/6 = ½The probability of having an odd number is 3/6 = ½.

3.3 Probability (16)

Frequency Interpretation: The probability of an event is the proportion of times the event will occur in a long

f t d i trun of repeated experiments.

E.G. Consider a dice is thrown once. What is the probability of having an dd b ? odd number?

sln. We repeat the experiments of throwing the dice M times.

sln. We repeat the experiments of throwing the dice M times. We count the number of experiments in which the outcome is an odd number. Suppose it is S.

The probability of having an odd number SlimThe probability of having an odd number .MM


In the following discussion you mayIn the following discussion, you may use the example where a dice is thrown once.

once.

Sample space is the set S ={1, 2, 3, 4, 5, 6} S

Any subset of a sample space is an event. A B

E.G. The event of odd numbers is the set A= {1, 3, 5}.

E G The event of the largest number is the set B = {6}E.G. The event of the largest number is the set B {6}.

3.4 The Axioms of Probability (18)

Consider an experiment where a dice is thrown once. S

We assign to any event A a number P(A) which we call

Consider an experiment where a dice is thrown once. S = {1, 2, 3, 4, 5, 6} , A = {1, 3, 5} and B = {6} A B

We assign to any event A a number P(A) which we call the probability of A. This number satisfies the 3 axioms:

Axiom 1 0 P(A) 1

P(A) = ½

P(B) = 1/6Axiom 1 0 P(A) 1

Axiom 2 P(S) = 1P(AB) = P(A) + P(B)

= 4/6

Axiom 3 If A and B are mutually exclusive events in S,i.e. AB = , then P(AB) = P(A) + P(B) e , t e ( ) ( ) ( )

Prove: P() = 0

Proof: S = so S and are mutually exclusive

P(S) P(S)=P(S) +P() b i 3P(S)=P(S)=P(S) +P() by axiom 3

P()=0 (Q.E.D.)

Fill in the missing words Fill in the missing words (19)

Def. someall

1) Event S (universal set) occurs at ________ experiment.

2) Event (empty set) occurs

allnoall

SA3) Event AB occurs when event A

2) Event (empty set) ________ occurs.

BA

S4) E t A B h t A

3) Event AB occurs when event A_______ event B occurs.

A BS4) Event AB occurs when event A

_______ event B occur.

S

Properties:-

1) AB = event A & event BA B

) ________ occur in the same experiment.

A2) Event A occurs Event ______ occurs.AGo to slide 7

3.5 Some Elementary Theorems (20)

Theorem 4: If A1, A2, ..., An are mutually exclusive events in a

sample space S, then p p ,

P( A1A2 ... An ) = P(A1) + P(A2) + ...+ P(An) .

This theorem is a generalization of Axiom 3 which states that

Axiom 3 If A and B are mutually exclusive events in S, then P(AB) = P(A) + P(B)

SA1,3,5

B6

SP(A) = ½ P(B) = 1/6

2 4 P(AB) = P(A) + P(B) = 4/62, 4


Theorem 5: If A is an events in the finite sample space S,

then P(A) equals the sum of the probabilities of the ( ) q p

individual outcomes comprising A.

E.G. Consider an experiment where a dice is thrown once. S = {1, 2, 3, 4, 5, 6}

Let the events of odd numbers be A = {1, 3, 5}.

The events of individual outcomes comprising A are {1}, {3}, {5}.

These 3 events with individual outcomes as elements are mutually exclusive. Therefore, y ,

P(A) = P(1}+P(3}+P(5}.

Theorem 6: If A and B are any events in S then

General addition rule for probability (22)

Theorem 6: If A and B are any events in S, then

P( AB ) = P(A) + P(B) - P(AB) .S

E.G. Consider an experiment where a dice is thrown once. S = {1, 2, 3, 4, 5, 6} 41 32

B AS

56

Let the event of odd numbers be A = {1, 3, 5} and the t f ll b b B {1 2 3}

6

event of small numbers be B = {1, 2, 3}.

Hence, AB = {1, 3} and AB = {1, 2, 3, 5}

P(A) = ½ , P(B)= ½ , P(AB) = 1/3

, { , } { , , , }

By Theorem 6, P( AB ) = P(A) + P(B) - P(AB)

= ½ + ½ - 1/3 = 4/6

You are required to prove this theorem in assignment 1.


General rule of the complement

Theorem 7: If A is any events in S, then

General rule of the complement

P( A ) = 1 - P(A) .

E G C id i t h di i thE.G. Consider an experiment where a dice is thrown once.S = {1, 2, 3, 4, 5, 6}

Let the event of the largest number be the set A = {6}.

P(A) 1/

A = {1 2 3 4 5}

P(A) = 1/6

A = {1, 2, 3, 4, 5}

By Theorem 7 P( A ) = 1 - P(A) = 5/6By Theorem 7, P( A ) 1 P(A) /6.

You are required to prove this theorem in assignment 1.

3.6 Conditional Probability (24)

If A d B t i S d P( B ) 0If A and B are any events in S, and P( B ) 0,

the conditional probability of A given B is

P( A|B ) = )(

)(BP

BAP )(BP

E.G.Let the event of odd numbers be A = {1 3 5}Let the event of odd numbers be A = {1, 3, 5}

and the event of small numbers be B = {1, 2, 3}. S

The conditional probabilityof A given B is 41 32

B AS

5of A given B is

)()|( BAPBAP 62 2

2 6

)()|(

BPBAP

2/16 .

3


General multiplication rule of probability

Theorem 8: If A and B are any events in S, then

General multiplication rule of probability

P( AB ) = P(A) P( B|A ) if P(A) 0

= P(B) P( A|B ) if P(B) 0( ) ( | ) ( )

The rule is a straightforward application ofThe rule is a straightforward application of

condition probability definition

,0)( if )(

)()|(

APAP

BAPABP

0)()()|(

BPBAPBAP if

)(

.0)()(

)|( BPBP

BAP if


Definition of Independent events

Theorem 9: A and B are independent events if and only if

A B A B

p

P( AB ) = P(A) P( B )

Events A A & A are independent iffEvents A1, A2 & A3 are independent iff

P(A1A2A3) = P(A1) P(A2) P(A3) &

P(A1A2)=P(A1)P(A2) & P(A1A3)=P(A1)P(A3) & P(A2A3)= P(A2)P(A3).

E.G. Given P(A1) = ½, P(A2) = ¼, P(A3) = ¼,

P(A1A2) = 1/8, P(A1A3) = 1/8, P(A2A3) = 1/8P(A1A2) 1/8, P(A1A3) 1/8, P(A2A3) 1/8

P(A1A2A3) = 1/32.

A A A & A i d d t?Are A1, A2 & A3 independent?


In general, n events A1, A2, …, An are independent iff

P(A1A2 …An) = P(A1) P(A2)… P(An)

: :: :

P(AiAjAk) = P(Ai) P(Aj) P(Ak)

P(AiAj) = P(Ai) P(Aj)

for all combinations of i, j, k,… where 1 i j k … n.

3.7 Bayes’ Theorem (29)

Th G l lti li ti l i f l i l i General multiplication The General multiplication rule is useful in solving problems in which the ultimate outcome of an experiment depends on the outcomes of various

prule of probability Theorem 8 P( AB ) = P(A) P(B|A) if P(A)0p p

intermediate stages. P(A) P(B|A) if P(A)0

E.G. A factory receives its voltage regulators from 3 suppliers.E.G. A factory receives its voltage regulators from 3 suppliers.Supplier Amount Probability of meeting specifications

B1 60% 95%B2 30% 80%B3 10% 65%

What is the probability that any one voltage regulator received by the factory meet specifications ?

Sln. Let A be the event that a voltage regulator received meets specifications.Let B1, B2, B3 be the events that a voltage regulator comes from respective suppliers.

E.G. A factory receives its voltage regulators from 3 suppliers.Supplier Amount Probability of meeting specificationsB1 60% 95%= P(B ) = P(A|B1)

(30)

B1 60% 95%B2 30% 80%B3 10% 65%

What is the probability of event A (a voltage regulator meets specifications) ?

P(B1)= P(B2)= P(B3)

P(A|B1)= P(A|B2)= P(A|B3)

A = A (B1B2B3) where B1,B2,B3 are mutually exclusive.


= ( AB1) ( AB2) ( AB3) AB1,AB2,AB3 mutually exclusive.

Theorem 4: If A1,A2,...,An areTheorem 4: If A1,A2,...,An are mutually exclusive, then

P(A1...An ) = P(A1) + ...+ P(An). P(A) = P(AB1) + P(AB2) + P(AB3)

General multiplication rule of probability Theorem 8

( A B ) (A) (B|A)= P(B1)P(A|B1) + P(B2)P(A|B2) + P(B3)P(A|B3) P( AB ) = P(A) P(B|A)

if P(A)0

( 1) ( | 1) ( 2) ( | 2) ( 3) ( | 3)

= 0.60 x 0.95 + 0.30 x 0.80 + 0.10 x 0.65

= 0.875

E.G. A factory receives its voltage regulators from 3 suppliers.Supplier Amount Probability of meeting specificationsB1 60% 95% = P(A|B1)= P(B )

(31)

B1 60% 95%B2 30% 80%B3 10% 65%


P(A|B1)= P(A|B2)= P(A|B3)

P(B1)= P(B2)= P(B3)


P(A)= P(B1)P(A|B1) + P(B2)P(A|B2) + P(B3)P(A|B3)

B1

0.60

0.95A|B1A

B20.30 0.80 A|B2A

B30.10 0 65

| 2

0.65 A|B3A

Theorem 10: If B1, B2, …, Bn are mutually exclusive events of which

Rule of total probability (32)

n

one must occur, then)|()()(

1i

n

ii BAPBPAP


B1

0.60

0.95A|B1

B20.30 0.80 A|B2

B30.10 0 65

| 2

0.65 A|B3


(33)

B1 60% 95%B2 30% 80%B3 10% 65%

P(A|B1)= P(A|B2)= P(A|B3)

P(B1)= P(B2)= P(B3)



B1

0.60

0.95A|B1

B20.30 0.80 A|B2

B30.10 0 65

| 2

0.65 A|B3


(34)

B1 60% 95%B2 30% 80%B3 10% 65%

P(A|B1)= P(A|B2)= P(A|B3)

P(B1)= P(B2)= P(B3)

What is the probability of event B3 given A has occurred, i.e. P(B3|A)?


P(B |A) =P(AB3)

B1

0.60

0.95A|B1

P(B3|A) = P(A)

P(AB3) = P(A|B3) P(B3)

B20.30 0.80 A|B2

Therefore,

P(B3|A) = P(A|B3) P(B3)

( ) ( | ) ( ) ( | ) ( ) ( | )

B30.10 0 65

| 2 P(B1)P(A|B1)+P(B2)P(A|B2)+P(B3)P(A|B3)

650108030950601.065.0

0.65 A|B365.01.08.03.095.06.0

= 0.074


(35)

B1 60% 95%B2 30% 80%B3 10% 65%

P(A|B1)= P(A|B2)= P(A|B3)

P(B1)= P(B2)= P(B3)

What is the probability of event B3 given A has occurred, i.e. P(B3|A)?

Bayes’ theorem

Theorem 11: If B1, B2, …, Bn are mutually exclusive events of which one must occur then

Bayes theorem

one must occur, then

.,...,2,1)|()(

)|()()|( nrBAPBP

BAPBPABP nrr

r

for)|()(

1BAPBP i

ii

Wh t i th b bilit th t t A i d b t B ?What is the probability that event A is caused by event B3 ?

The probability of event B3 is called the prior, or a priori probability of the cause.

ENGG2450B Probability and Statistics for...

Documents

Transcript of ENGG2450B Probability and Statistics for...