AN INTRODUCTION TOMATHEMATICAL STATISTICSAND ITS APPLICATIONSFifth EditionRichard J. LarsenVanderbilt UniversityMorris L. MarxUniversity of West FloridaPrentice HallBoston Columbus Indianapolis New York San FranciscoUpper Saddle River Amsterdam Cape Town DubaiLondon Madrid Milan Munich Paris MontralToronto Delhi Mexico City So Paulo SydneyHong Kong Seoul Singapore Taipei TokyoEditor in Chief: Deirdre LynchAcquisitions Editor: Christopher CummingsAssociate Editor: Christina LepreAssistant Editor: Dana JonesSenior Managing Editor: Karen WernholmAssociate Managing Editor: Tamela AmbushSenior Production Project Manager: Peggy McMahonSenior Design Supervisor: Andrea NixCover Design: Beth PaquinInterior Design: Tamara NewnamMarketing Manager: Alex GayMarketing Assistant: Kathleen DeChavezSenior Author Support/Technology Specialist: Joe VetereManufacturing Manager: Evelyn BeatonSenior Manufacturing Buyer: Carol MelvilleProduction Coordination, Technical Illustrations, and Composition: Integra Software Services, Inc.Cover Photo: Jason Reed/Getty ImagesMany of the designations used by manufacturers and sellers to distinguish their products are claimed astrademarks. Where those designations appear in this book, and Pearson was aware of a trademarkclaim, the designations have been printed in initial caps or all caps.Library of Congress Cataloging-in-Publication DataLarsen, Richard J.An introduction to mathematical statistics and its applications /Richard J. Larsen, Morris L. Marx.5th ed.p. cm.Includes bibliographical references and index.ISBN 978-0-321-69394-51. Mathematical statisticsTextbooks. I. Marx, Morris L. II. Title.QA276.L314 2012519.5dc222010001387Copyright 2012, 2006, 2001, 1986, and 1981 by Pearson Education, Inc. All rights reserved. No part ofthis publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by anymeans, electronic, mechanical, photocopying, recording, or otherwise, without the prior writtenpermission of the publisher. Printed in the United States of America. For information on obtainingpermission for use of material in this work, please submit a written request to Pearson Education, Inc.,Rights and Contracts Department, 501 Boylston Street, Suite 900, Boston, MA 02116, fax your requestto 617-671-3447, or e-mail at http://www.pearsoned.com/legal/permissions.htm.1 2 3 4 5 6 7 8 9 10EB14 13 12 11 10ISBN-13: 978-0-321-69394-5ISBN-10: 0-321-69394-9Table of ContentsPreface viii1Introduction 11.1 An Overview 11.2 Some Examples 21.3 A Brief History 71.4 A Chapter Summary 142Probability 162.1 Introduction 162.2 Sample Spaces and the Algebra of Sets 182.3 The Probability Function 272.4 Conditional Probability 322.5 Independence 532.6 Combinatorics 672.7 Combinatorial Probability 902.8 Taking a Second Look at Statistics (Monte Carlo Techniques) 993Random Variables 1023.1 Introduction 1023.2 Binomial and Hypergeometric Probabilities 1033.3 Discrete Random Variables 1183.4 Continuous Random Variables 1293.5 Expected Values 1393.6 The Variance 1553.7 Joint Densities 1623.8 Transforming and Combining Random Variables 1763.9 Further Properties of the Mean and Variance 1833.10 Order Statistics 1933.11 Conditional Densities 2003.12 Moment-Generating Functions 2073.13 Taking a Second Look at Statistics (Interpreting Means) 216Appendix 3.A.1 Minitab Applications 218iiiiv Table of Contents4Special Distributions 2214.1 Introduction 2214.2 The Poisson Distribution 2224.3 The Normal Distribution 2394.4 The Geometric Distribution 2604.5 The Negative Binomial Distribution 2624.6 The Gamma Distribution 2704.7 Taking a Second Look at Statistics (Monte CarloSimulations) 274Appendix 4.A.1 Minitab Applications 278Appendix 4.A.2 A Proof of the Central Limit Theorem 2805Estimation 2815.1 Introduction 2815.2 Estimating Parameters: The Method of Maximum Likelihood andthe Method of Moments 2845.3 Interval Estimation 2975.4 Properties of Estimators 3125.5 Minimum-Variance Estimators: The Cramr-Rao LowerBound 3205.6 Sufficient Estimators 3235.7 Consistency 3305.8 Bayesian Estimation 3335.9 Taking a Second Look at Statistics (Beyond ClassicalEstimation) 345Appendix 5.A.1 Minitab Applications 3466Hypothesis Testing 3506.1 Introduction 3506.2 The Decision Rule 3516.3 Testing Binomial DataH0: p = po3616.4 Type I and Type II Errors 3666.5 A Notion of Optimality: The Generalized Likelihood Ratio 3796.6 Taking a Second Look at Statistics (Statistical Signicance versusPractical Signicance) 382Table of Contents v7Inferences Based on the NormalDistribution 3857.1 Introduction 3857.2 ComparingY/n andYS/n3867.3 Deriving the Distribution ofYS/n3887.4 Drawing Inferences About 3947.5 Drawing Inferences About 24107.6 Taking a Second Look at Statistics (Type II Error) 418Appendix 7.A.1 Minitab Applications 421Appendix 7.A.2 Some Distribution Results for Y and S2423Appendix 7.A.3 A Proof that the One-Sample t Test is a GLRT 425Appendix 7.A.4 A Proof of Theorem 7.5.2 4278Types of Data: A Brief Overview 4308.1 Introduction 4308.2 Classifying Data 4358.3 Taking a Second Look at Statistics (Samples Are NotValid!) 4559Two-Sample Inferences 4579.1 Introduction 4579.2 Testing H0: X=Y4589.3 Testing H0: 2X=2YThe F Test 4719.4 Binomial Data: Testing H0: pX = pY4769.5 Confidence Intervals for the Two-Sample Problem 4819.6 Taking a Second Look at Statistics (Choosing Samples) 487Appendix 9.A.1 A Derivation of the Two-Sample t Test (A Proof ofTheorem 9.2.2) 488Appendix 9.A.2 Minitab Applications 49110Goodness-of-Fit Tests 49310.1 Introduction 49310.2 The Multinomial Distribution 49410.3 Goodness-of-Fit Tests: All Parameters Known 49910.4 Goodness-of-Fit Tests: Parameters Unknown 50910.5 Contingency Tables 519vi Table of Contents10.6 Taking a Second Look at Statistics (Outliers) 529Appendix 10.A.1 Minitab Applications 53111Regression 53211.1 Introduction 53211.2 The Method of Least Squares 53311.3 The Linear Model 55511.4 Covariance and Correlation 57511.5 The Bivariate Normal Distribution 58211.6 Taking a Second Look at Statistics (How Not to Interpretthe Sample Correlation Coefcient) 589Appendix 11.A.1 Minitab Applications 590Appendix 11.A.2 A Proof of Theorem 11.3.3 59212The Analysis of Variance 59512.1 Introduction 59512.2 The F Test 59712.3 Multiple Comparisons: Tukeys Method 60812.4 Testing Subhypotheses with Contrasts 61112.5 Data Transformations 61712.6 Taking a Second Look at Statistics (Putting the Subject ofStatistics TogetherThe Contributions of Ronald A. Fisher) 619Appendix 12.A.1 Minitab Applications 621Appendix 12.A.2 A Proof of Theorem 12.2.2 624Appendix 12.A.3 The Distribution ofSSTR/(k1)SSE/(nk)When H1 is True 62413Randomized Block Designs 62913.1 Introduction 62913.2 The F Test for a Randomized Block Design 63013.3 The Paired t Test 64213.4 Taking a Second Look at Statistics (Choosing between aTwo-Sample t Test and a Paired t Test) 649Appendix 13.A.1 Minitab Applications 65314Nonparametric Statistics 65514.1 Introduction 65614.2 The Sign Test 657Table of Contents vii14.3 Wilcoxon Tests 66214.4 The Kruskal-Wallis Test 67714.5 The Friedman Test 68214.6 Testing for Randomness 68414.7 Taking a Second Look at Statistics (Comparing Parametricand Nonparametric Procedures) 689Appendix 14.A.1 Minitab Applications 693Appendix: Statistical Tables 696Answers to Selected Odd-Numbered Questions 723Bibliography 745Index 753PrefaceThe rst edition of this text was published in 1981. Each subsequent revision sincethenhas undergonemorethanafewchanges. Topics havebeenadded, com-putersoftwareandsimulationsintroduced, andexamplesredone. Whathasnotchangedovertheyearsisourpedagogicalfocus.Asthetitleindicates,thisbookisanintroductiontomathematicalstatisticsanditsapplications.Thoselastthreewords are not an afterthought. We continue to believe that mathematical statisticsis best learnedandmost effectivelymotivatedwhenpresentedagainst aback-dropof real-worldexamples andall theissues that thoseexamples necessarilyraise.Werecognizethatcollegestudentstodayhavemoremathematicscoursestochoose from than ever before because of the new specialties and interdisciplinaryareasthatcontinuetoemerge. Forstudentswantingabroadeducationalexperi-ence, an introduction to a given topic may be all that their schedules can reasonablyaccommodate. Our response to that reality has been to ensure that each edition ofthistextprovidesamorecomprehensiveandmoreusabletreatmentofstatisticsthan did its predecessors.Traditionally, the focus of mathematical statistics has been fairly narrowthesubjects objective has been to provide the theoretical foundation for all of the var-iousproceduresthatareusedfordescribingandanalyzingdata.Whatithasnotspoken to at much length are the important questions of which procedure to usein a given situation, and why. But those are precisely the concerns that every userof statistics must inevitably confront. To that end, adding features that can createa path from the theory of statistics to its practice has become an increasingly highpriority.New to This EditionBeginningwiththethirdedition, Chapter8, titledDataModels,wasadded.It discussed some of the basic principles of experimental design, as well as someguidelines for knowing how to begin a statistical analysis. In this fth edition, theData Models (Types of Data: A Brief Overview) chapter has been substantiallyrewritten to make its main points more accessible.Beginning with the fourth edition, the end of each chapter except the rst fea-tured a section titled Taking a Second Look at Statistics. Many of these sectionsdescribe the ways that statistical terminology is often misinterpreted in what wesee, hear, andreadinourmodernmedia. Continuinginthisveinofinterpre-tation, wehaveaddedinthisftheditioncommentscalledAbouttheData.These sections are scattered throughout the text and are intended to encouragethe reader to think critically about a data sets assumptions, interpretations, andimplications.Manyexamples andcasestudies havebeenupdated, whilesomehavebeendeleted and others added.Section 3.8, Transforming and Combining RandomVariables, has beenrewritten.viiiPreface ixSection 3.9, Further Properties of the Mean and Variance, now includes a dis-cussion of covariances so that sums of random variables can be dealt with in moregenerality.Chapter 5, Estimation, now has an introduction to bootstrapping.Chapter 7, Inferences Based on the Normal Distribution, has new material onthe noncentral t distribution and its role in calculating Type II error probabilities.Chapter 9, Two-Sample Inferences, has a derivation of Welchs approx-imation for testing the differences of two means in the case of unequalvariances.We hope that the changes in this edition will not undo the best features of therst four. What made the task of creating the fth edition an enjoyable experiencewas the nature of the subject itself and the way that it can be beautifully elegant anddown-to-earth practical, all at the same time. Ultimately, our goal is to share withthe reader at least some small measure of the affection we feel for mathematicalstatistics and its applications.SupplementsInstructorsSolutionsManual. Thisresourcecontainsworked-out solutionstoalltextexercisesandisavailablefordownloadfromthePearsonEducationInstructor Resource Center.Student Solutions Manual ISBN-10: 0-321-69402-3; ISBN-13: 978-0-321-69402-7. Featuring complete solutions to selected exercises, this is a great toolfor students as they study and work through the problem material.AcknowledgmentsWewouldliketothankthefollowingreviewersfortheirdetailedandvaluablecomments, criticisms, and suggestions:Dr. Abera Abay, Rowan UniversityKyle Siegrist, University of Alabama in HuntsvilleDitlev Monrad, University of Illinois at Urbana-ChampaignVidhu S. Prasad, University of Massachusetts, LowellWen-Qing Xu, California State University, Long BeachKatherine St. Clair, Colby CollegeYimin Xiao, Michigan State UniversityNicolas Christou, University of California, Los AngelesDaming Xu, University of OregonMaria Rizzo, Ohio UniversityDimitris Politis, University of California at San DiegoFinally, weconveyour gratitudeandappreciationtoPearsonArts &SciencesAssociateEditorforStatisticsChristinaLepre; AcquisitionsEditorChristopherCummings; andSeniorProductionProjectManagerPeggyMcMahon, aswellasx Prefaceto Project Manager Amanda Zagnoli of Elm Street Publishing Services, for theirexcellent teamwork in the production of this book.Richard J. LarsenNashville, TennesseeMorris L. MarxPensacola, FloridaChapt erIntroduction11.1 An Overview1.2 Some Examples1.3 A Brief History1.4 A Chapter SummaryUntil the phenomena of any branch of knowledge have been submitted tomeasurement and number it cannot assume the status and dignity of a science.Francis Galton1.1An OverviewSir Francis Galton was a preeminent biologist of the nineteenth century. A passion-ate advocate for the theory of evolution (his nickname was Darwins bulldog),Galton was also an early crusader for the study of statistics and believed the subjectwould play a key role in the advancement of science:Some people hate the very name of statistics, but I nd them full of beauty and inter-est. Whenever they are not brutalized, but delicately handled by the higher methods,and are warily interpreted, their power of dealing with complicated phenomena isextraordinary. They are the only tools by which an opening can be cut through theformidable thicket of difculties that bars the path of those who pursue the Scienceof man.Did Galtons prediction come to pass? Absolutelytry reading a biology journalor the analysis of a psychology experiment before taking your rst statistics course.Science and statistics have become inseparable, two peas in the same pod. What thegood gentleman from London failed to anticipate, though, is the extent to which allof usnot just scientistshave become enamored (some would say obsessed) withnumerical information. The stock market is awash in averages, indicators, trends,and exchange rates; federal education initiatives have taken standardized testing tonew levels of specicity; Hollywood uses sophisticated demographics to see whoswatching what, and why; and pollsters regularly tally and track our every opinion,regardless of how irrelevant or uninformed. In short, we have come to expect every-thing to be measured, evaluated, compared, scaled, ranked, and ratedand if theresults are deemed unacceptable for whatever reason, we demand that someone orsomething be held accountable (in some appropriately quantiable way).To be sure, many of these efforts are carefully carried out and make perfectlygoodsense; unfortunately, others areseriouslyawed, andsomearejust plainnonsense. What they all speak to, though, is the clear and compelling need to knowsomething about the subject of statistics, its uses and its misuses.12 Chapter 1IntroductionThisbookaddressestwobroadtopicsthemathematicsofstatisticsandthepractice of statistics. The two are quite different. The former refers to the probabil-ity theory that supports and justies the various methods used to analyze data. Forthe most part, this background material is covered in Chapters 2 through 7. The keyresult is the central limit theorem, which is one of the most elegant and far-reachingresults in all of mathematics. (Galton believed the ancient Greeks would have per-sonied and deied the central limit theorem had they known of its existence.) Alsoincluded in these chapters is a thorough introduction to combinatorics, the math-ematics of systematic counting. Historically, this was the very topic that launchedthe development of probability in the rst place, back in the seventeenth century.In addition to its connection to a variety of statistical procedures, combinatorics isalso the basis for every state lottery and every game of chance played with a roulettewheel, a pair of dice, or a deck of cards.The practice of statistics refers to all the issues (and there are many!) that arisein the design, analysis, and interpretation of data. Discussions of these topics appearin several different formats. Following most of the case studies throughout the text isa feature entitled About the Data. These are additional comments about either theparticular data in the case study or some related topic suggested by those data. Thennear the end of most chapters is a Taking a Second Look at Statistics section. Severalof these deal with the misuses of statisticsspecically, inferences drawn incorrectlyand terminology used inappropriately. The most comprehensive data-related discus-sion comes in Chapter 8, which is devoted entirely to the critical problem of knowinghow to start a statistical analysisthat is, knowing which procedure should be used,and why.Morethanacenturyago,Galtondescribedwhathethoughtaknowledgeofstatistics should entail. Understanding the higher methods, he said, was the keyto ensuring that data would be delicately handled and warily interpreted. Thegoal of this book is to make that happen.1.2Some ExamplesStatistical methods are often grouped into two broad categoriesdescriptive statis-ticsandinferential statistics. Theformerreferstoall thevarioustechniquesforsummarizing and displaying data. These are the familiar bar graphs, pie charts, scat-terplots, means, medians, and the like, that we see so often in the print media. Themuch more mathematical inferential statistics are procedures that make generaliza-tions and draw conclusions of various kinds based on the information contained ina set of data; moreover, they calculate the probability of the generalizations beingcorrect.Described in this section are three case studies. The rst illustrates a very effec-tiveuseof several descriptivetechniques. Thelatter twoillustratethesorts ofquestions that inferential procedures can help answer.Case Study 1.2.1Pictured at the top of Figure 1.2.1 is the kind of information routinely recordedby a seismographlisted chronologically are the occurrence times and Richtermagnitudes for a series of earthquakes. As raw data, the numbers are largely(Continued on next page)1.2 Some Examples 3meaningless: Nopatterns areevident, nor is thereanyobvious connectionbetween the frequencies of tremors and their severities.3020100 4 5Magnitude on Richter scale, RAverage number of shocks per year, N6 7N = 80,338.16eEpisode numberDateTimeSeverity (Richter scale)2176/194:53 P.M.2.72187/26:07 A.M.3.12197/48:19 A.M.2.02208/71:10 A.M.4.12218/710:46 P.M.3.6 1.981RFigure 1.2.1Shown at the bottom of the gure is the result of applying several descrip-tive techniques to an actual set of seismograph data recorded over a period ofseveral years in southern California (67). Plotted above the Richter (R) value of4.0, for example, is the average number (N) of earthquakes occurring per yearin that region having magnitudes in the range 3.75 to 4.25. Similar points areincluded forR-values centered at 4.5, 5.0, 5.5, 6.0, 6.5, and 7.0. Now we can seethat earthquake frequencies and severities are clearly related: Describing the(N, R)s exceptionally well is the equationN =80,338.16e1.981R(1.2.1)which is found using a procedure described in Chapter 9. (Note: Geologists haveshown that the modelN =0e1Rdescribes the (N, R) relationship all over theworld. All that changes from region to region are the numerical values for0and 1.)(Continued on next page)4 Chapter 1Introduction(Case Study 1.2.1 continued)NoticethatEquation1.2.1ismorethanjustanelegantsummaryoftheobserved(N, R)relationship. Rather, itallowsustoestimatethelikelihoodof future earthquake catastrophes for large values of Rthat have never beenrecorded. For example, many Californians worry about the Big One, a mon-ster tremorsay, R =10.0that breaks off chunks of tourist-covered beachesandsendsthemoatingtowardHawaii. Howoftenmightweexpectthattohappen? Setting R =10.0 in Equation 1.2.1 givesN =80,338.16e1.98(10.0)=0.0002 earthquake per yearwhichtranslatestoapredictionofonesuchmegaquakeeveryvethousandyears (=1/0.0002). (Of course, whether that estimate is alarming or reassuringprobably depends on whether you live in San Diego or Topeka. . . .)About the Data The megaquake prediction prompted by Equation 1.2.1 raises anobvious question: Why is the calculation that led to the model N =80,338.16e1.981Rnot considered an example of inferential statistics even though it did yield a pre-diction forR =10? The answer is that Equation 1.2.1by itselfdoes not tell usanything about the error associated with its predictions. In Chapter 11, a moreelaborate probability method based on Equation 1.2.1 is described that does yielderror estimates and qualies as a bona de inference procedure.Case Study 1.2.2Claims of disputed authorship can be very difcult to resolve. Speculation haspersisted for several hundred years that some of William Shakespeares workswerewrittenbySir Francis Bacon(or maybeChristopher Marlowe). Andwhether it was Alexander Hamilton or James Madison who wrote certain ofthe Federalist Papers is still an open question. Less well known is a controversysurrounding Mark Twain and the Civil War.One of the most revered of all American writers, Twain was born in 1835,which means he was twenty-six years old when hostilities between the Northand South broke out. At issue is whether he was ever a participant in the warand, if he was, on which side. Twain always dodged the question and took theanswer to his grave. Even had he made a full disclosure of his military record,though, his role in the Civil War would probably still be a mystery because ofhis self-proclaimed predisposition to be less than truthful. Reecting on his life,Twain made a confession that would give any would-be biographer pause: I aman old man, he said, and have known a great many troubles, but most of themnever happened.What some historians think might be the clue that solves the mystery is a setof ten essays that appeared in 1861 in the New Orleans Daily Crescent. Signed(Continued on next page)1.2 Some Examples 5QuintusCurtiusSnodgrass,theessayspurportedtochronicletheauthorsadventures as a member of the Louisiana militia. Many experts believe that theexploitsdescribedactuallydidhappen, butLouisianaeldcommandershadno record of anyone named Quintus Curtius Snodgrass. More signicantly, thepieces display the irony and humor for which Twain was so famous.Table 1.2.1 summarizes data collected in an attempt (16) to use statisticalinferencetoresolvethedebateovertheauthorshipoftheSnodgrassletters.Listed are the proportions of three-letter words (1) in eight essays known tohave been written by Mark Twain and (2) in the ten Snodgrass letters.Researchershavefoundthat authorstendtohavecharacteristicword-length proles, regardless of what the topic might be. It follows, then, that ifTwain and Snodgrass were the same person, the proportion of, say, three-letterwords that they used should be roughly the same. The bottom of Table 1.2.1showsthat,ontheaverage,23.2%ofthewordsinaTwainessaywerethreeletters long; the corresponding average for the Snodgrass letters was 21.0%.If Twain and Snodgrass were the same person, the difference between theseaveragethree-letterproportionsshouldbecloseto0: forthesetwosetsofessays, the difference in the averages was 0.022 (=0.232 0.210). How shouldwe interpret the difference 0.022 in this context? Two explanations need to beconsidered:1. The difference, 0.022, is sufciently small (i.e., close to 0) that it does notrule out the possibility that Twain and Snodgrass were the same person.or2. The difference, 0.022, is so large that the only reasonable conclusion is thatTwain and Snodgrass were not the same person.Choosingbetweenexplanations1and2isanexampleofhypothesistesting,which is a very frequently encountered form of statistical inference.The principles of hypothesis testing are introduced in Chapter 6, and theparticular procedurethat applies toTable1.2.1rst appears inChapter 9.Soasnot tospoil theendingof agoodmystery, wewill deferunmaskingMr. Snodgrass until then.Table 1.2.1Twain Proportion QCS ProportionSergeant Fathom letter 0.225 Letter I 0.209Madame Caprell letter 0.262 Letter II 0.205Mark Twain letters in Letter III 0.196Territorial Enterprise Letter IV 0.210First letter 0.217 Letter V 0.202Second letter 0.240 Letter VI 0.207Third letter 0.230 Letter VII 0.224Fourth letter 0.229 Letter VIII 0.223First Innocents Abroad letter Letter IX 0.220First half 0.235 Letter X 0.201Second half 0.217Average: 0.232 0.2106 Chapter 1IntroductionCase Study 1.2.3It may not be made into a movie anytime soon, but the way that statistical infer-ence was used to spy on the Nazis in World War II is a pretty good tale. And itcertainly did have a surprise ending!The story began in the early 1940s. Fighting in the European theatre wasintensifying, andAlliedcommanderswereamassingasizeablecollectionofabandonedandsurrenderedGermanweapons. Whentheyinspectedthoseweapons, the Allies noticed that each one bore a different number. Aware ofthe Nazis reputation for detailed record keeping, the Allies surmised that eachnumber represented the chronological order in which the piece had been man-ufactured. But if that was true, might it be possible to use the captured serialnumbers to estimate the total number of weapons the Germans had produced?That was precisely the question posed to a group of government statisticiansworking out of Washington, D.C. Wanting to estimate an adversarys manufac-turing capability was, of course, nothing new. Up to that point, though, the onlysources of that information had been spies and traitors; using serial numberswas something entirely new.The answer turned out to be a fairly straightforward application of the prin-ciples that will be introduced in Chapter 5. If n is the total number of capturedserial numbers and xmax is the largest captured serial number, then the estimatefor the total number of items produced is given by the formulaestimated output =[(n +1)/n]xmax1 (1.2.2)Suppose, for example, that n =5 tanks were captured and they bore the serialnumbers 92, 14, 28, 300, and 146, respectively. Then xmax=300 and the estimatedtotal number of tanks manufactured is 359:estimated output =[(5 +1)/5]300 1=359Did Equation 1.2.2 work? Better than anyone could have expected (proba-bly even the statisticians). When the war ended and the Third Reichs trueproductiongureswererevealed, itwasfoundthatserial numberestimateswerefar moreaccurateineveryinstancethanall theinformationgleanedfrom traditional espionage operations, spies, and informants. The serial num-berestimateforGermantankproductionin1942, forexample, was3400, agureveryclosetotheactual output. Theofcialestimate, ontheotherhand, based on intelligence gathered in the usual ways, was a grossly inated18,000 (64).About the Data Large discrepancies, like 3400 versus 18,000 for the tank estimates,were not uncommon. The espionage-based estimates were consistently erring on thehigh side because of the sophisticated Nazi propaganda machine that deliberatelyexaggeratedthe countrys industrial prowess. On spies and would-be adversaries,the Third Reichs carefully orchestrated dissembling worked exactly as planned; onEquation 1.2.2, though, it had no effect whatsoever!1.3 A Brief History 71.3A Brief HistoryFor those interested in how we managed to get to where we are (or who just wantto procrastinate a bit longer), Section 1.3 offers a brief history of probability andstatistics. The two subjects were not mathematical littermatesthey began at dif-ferent times in different places for different reasons. How and why they eventuallycame together makes for an interesting story and reacquaints us with some toweringgures from the past.Probability: The Early YearsNooneknowswhereorwhenthenotionofchancerstarose; itfadesintoourprehistory. Nevertheless, evidence linking early humans with devices for generatingrandom events is plentiful: Archaeological digs, for example, throughout the ancientworld consistently turn up a curious overabundance of astragali, the heel bones ofsheep and other vertebrates. Why should the frequencies of these bones be so dis-proportionately high? One could hypothesize that our forebears were fanatical footfetishists, but two other explanations seem more plausible: The bones were used forreligious ceremonies and for gambling.Astragali have six sides but are not symmetrical (see Figure 1.3.1). Those foundin excavations typically have their sides numbered or engraved. For many ancientcivilizations, astragali were the primary mechanism through which oracles solicitedthe opinions of their gods. In Asia Minor, for example, it was customary in divinationrites to roll, or cast, ve astragali. Each possible conguration was associated withthe name of a god and carried with it the sought-after advice. An outcome of (1, 3,3, 4, 4), for instance, was said to be the throw of the savior Zeus, and its appearancewas taken as a sign of encouragement (34):One one, two threes, two foursThe deed which thou meditatest, go do it boldly.Put thy hand to it. The gods have given theefavorable omensShrink not from it in thy mind, for no evilshall befall thee.Figure 1.3.1Sheep astragalusA (4, 4, 4, 6, 6), on the other hand, the throw of the child-eating Cronos, would sendeveryone scurrying for cover:Three fours and two sixes. God speaks as follows.Abide in thy house, nor go elsewhere,8 Chapter 1IntroductionLest a ravening and destroying beast come nigh thee.For I see not that this business is safe. But bidethy time.Gradually, overthousandsofyears, astragali werereplacedbydice, andthelatter became the most common means for generating random events. Pottery dicehave been found in Egyptian tombs built before 2000 b.c.; by the time the Greekcivilization was in full ower, dice were everywhere. (Loaded dice have also beenfound. Mastering the mathematics of probability would prove to be a formidabletask for our ancestors, but they quickly learned how to cheat!)The lack of historical records blurs the distinction initially drawn between div-ination ceremonies and recreational gaming. Among more recent societies, though,gambling emerged as a distinct entity, and its popularity was irrefutable. The Greeksand Romans were consummate gamblers, as were the early Christians (91).RulesformanyoftheGreekandRomangameshavebeenlost,butwecanrecognize the lineage of certain modern diversions in what was played during theMiddleAges. Themostpopulardicegameofthatperiodwascalledhazard, thenamederivingfromtheArabicalzhar, whichmeansadie.Hazardisthoughtto have been brought to Europe by soldiers returning from the Crusades; its rulesaremuchlikethoseofourmodern-daycraps.Cardswererstintroducedinthefourteenth century and immediately gave rise to a game known as Primero, an earlyformofpoker.Boardgamessuchasbackgammonwerealsopopularduringthisperiod.Given this rich tapestry of games and the obsession with gambling that char-acterized so much of the Western world, it may seem more than a little puzzlingthat a formal study of probability was not undertaken sooner than it was. As wewill seeshortly, therstinstanceofanyoneconceptualizingprobabilityintermsof a mathematical model occurred in the sixteenth century. That means that morethan2000yearsof dicegames, cardgames, andboardgamespassedbybeforesomeonenallyhadtheinsighttowritedowneventhesimplestofprobabilisticabstractions.Historians generally agree that, as a subject, probability got off to a rocky startbecause of its incompatibility with two of the most dominant forces in the evolutionof our Western culture, Greek philosophy and early Christian theology. The Greekswere comfortable with the notion of chance (something the Christians were not),but it went against their nature to suppose that random events could be quantied inany useful fashion. They believed that any attempt to reconcile mathematically whatdid happen with what should have happened was, in their phraseology, an improperjuxtaposition of the earthly plane with the heavenly plane.Making matters worse was the antiempiricism that permeated Greek thinking.Knowledge, to them, was not something that should be derived by experimentation.It was better to reason out a question logically than to search for its explanation in aset of numerical observations. Together, these two attitudes had a deadening effect:The Greeks had no motivation to think about probability in any abstract sense, norwere they faced with the problems of interpreting data that might have pointed themin the direction of a probability calculus.If the prospects for the study of probability were dim under the Greeks, theybecame even worse when Christianity broadened its sphere of inuence. The Greeksand Romans at least accepted the existence of chance. However, they believed theirgods to be either unable or unwilling to get involved in matters so mundane as theoutcome of the roll of a die. Cicero writes:1.3 A Brief History 9Nothing is so uncertain as a cast of dice, and yet there is no one who plays often whodoes not make a Venus-throw1and occasionally twice and thrice in succession. Thenare we, like fools, to prefer to say that it happened by the direction of Venus ratherthan by chance?For the early Christians, though, there was no such thing as chance:Every eventthat happened, no matter how trivial, was perceived to be a direct manifestation ofGods deliberate intervention. In the words of St. Augustine:Nos eas causas quae dicuntur fortuitae . . . non dicimusnullas, sed latentes; easque tribuimus vel veri Dei . . .(We say that those causes that are said to be by chanceare not non-existent but are hidden, and we attributethem to the will of the true God. . .)TakingAugustinespositionmakesthestudyof probabilitymoot, andit makesa probabilist a heretic. Not surprisingly, nothing of signicance was accomplishedin the subject for the next fteen hundred years.Itwasinthesixteenthcenturythatprobability,likeamathematicalLazarus,arose from the dead. Orchestrating its resurrection was one of the most eccentricgures in the entire history of mathematics, Gerolamo Cardano. By his own admis-sion, CardanopersoniedthebestandtheworsttheJekyll andtheHydeofthe Renaissance man. He was born in 1501 in Pavia. Facts about his personal lifeare difcult to verify. He wrote an autobiography, but his penchant for lying raisesdoubts about much of what he says. Whether true or not, though, his one-sentenceself-assessment paints an interesting portrait (127):Naturehasmademecapableinallmanualwork, ithasgivenmethespiritofaphilosopherandabilityinthesciences, tasteandgoodmanners, voluptuousness,gaiety, it has made me pious, faithful, fond of wisdom, meditative, inventive, coura-geous, fondof learningandteaching, eagertoequal thebest, todiscovernewthings and make independent progress, of modest character, a student of medicine,interested in curiosities and discoveries, cunning, crafty, sarcastic, an initiate in themysterious lore, industrious, diligent, ingenious, living only from day to day, imper-tinent, contemptuous of religion, grudging, envious, sad, treacherous, magician andsorcerer, miserable, hateful, lascivious, obscene, lying, obsequious, fond of the prat-tle of old men, changeable, irresolute, indecent, fond of women, quarrelsome, andbecause of the conicts between my nature and soul I am not understood even bythose with whom I associate most frequently.Formally trained in medicine, Cardanos interest in probability derived from hisaddiction to gambling. His love of dice and cards was so all-consuming that he issaid to have once sold all his wifes possessions just to get table stakes! Fortunately,something positive came out of Cardanos obsession. He began looking for a math-ematical model that would describe, in some abstract way, the outcome of a randomevent. What he eventually formalized is now called the classical denition of prob-ability: If the total number of possible outcomes, all equally likely, associated withsome action is n, and if m of those n result in the occurrence of some given event,then the probability of that event is m/n. If a fair die is rolled, there are n =6 pos-sible outcomes. If the event Outcome is greater than or equal to 5 is the one in1When rolling four astragali, each of which is numbered on four sides, a Venus-throw was having each of thefour numbers appear.10 Chapter 1IntroductionFigure 1.3.2135246Possible outcomesOutcomes greaterthan or equal to5; probability = 2/6which we are interested, then m =2 (the outcomes 5 and 6) and the probability ofthe event is26, or13(see Figure 1.3.2).Cardanohadtappedintothemostbasicprincipleinprobability. Themodelhe discovered may seem trivial in retrospect, but it represented a giant step forward:His was the rst recorded instance of anyone computing a theoretical, as opposed toan empirical, probability. Still, the actual impact of Cardanos work was minimal.He wrote a book in 1525, but its publication was delayed until 1663. By then, thefocus of the Renaissance, as well as interest in probability, had shifted from Italy toFrance.The date cited by many historians (those who are not Cardano supporters) asthe beginning of probability is 1654. In Paris a well-to-do gambler, the Chevalierde Mr, asked several prominent mathematicians, including Blaise Pascal, a seriesof questions, the best known of which is the problem of points:Two people, A and B, agree to play a series of fair games until one person has wonsix games. They each have wagered the same amount of money, the intention beingthat the winner will be awarded the entire pot. But suppose, for whatever reason,the series is prematurely terminated, at which point A has won ve games and Bthree. How should the stakes be divided?[Thecorrect answeristhat Ashouldreceiveseven-eighthsof thetotal amountwagered. (Hint: Suppose the contest were resumed. What scenarios would lead toAs being the rst person to win six games?)]Pascal was intrigued by de Mrs questions and shared his thoughts with PierreFermat, a Toulouse civil servant and probably the most brilliant mathematician inEurope. Fermat graciously replied, and from the now-famous Pascal-Fermat corre-spondence came not only the solution to the problem of points but the foundationfor more general results. More signicantly, news of what Pascal and Fermat wereworking on spread quickly. Others got involved, of whom the best known was theDutch scientist and mathematician Christiaan Huygens. The delays and the indif-ferencethat hadplaguedCardanoacenturyearlierwerenot goingtohappenagain.Best remembered for his work in optics and astronomy, Huygens, early in hiscareer, was intrigued by the problem of points. In 1657 he published De Ratiociniisin Aleae Ludo (Calculations in Games of Chance), a very signicant work, far morecomprehensive than anything Pascal and Fermat had done. For almost fty years itwas the standard textbook in the theory of probability. Not surprisingly, Huygenshas supporters who feel that he should be credited as the founder of probability.Almostallthemathematicsofprobabilitywasstillwaitingtobediscovered.What Huygens wrote was only the humblest of beginnings, a set of fourteen propo-sitions bearing little resemblance to the topics we teach today. But the foundationwas there. The mathematics of probability was nally on rm footing.1.3 A Brief History 11Statistics: From Aristotle to QueteletHistoriansgenerallyagreethatthebasicprinciplesofstatisticalreasoningbeganto coalesce in the middle of the nineteenth century. What triggered this emergencewas the union of three different sciences, each of which had been developing alongmore or less independent lines (195).Therstofthesesciences, whattheGermanscalledStaatenkunde, involvedthecollectionofcomparativeinformationonthehistory, resources, andmilitaryprowessofnations. Althougheffortsinthisdirectionpeakedintheseventeenthandeighteenthcenturies,theconceptwashardlynew: Aristotlehaddonesome-thing similar in the fourth century b.c. Of the three movements, this one had theleast inuence on the development of modern statistics, but it did contribute someterminology: Thewordstatistics, itself, rst aroseinconnectionwithstudies ofthis type.Thesecondmovement, knownaspolitical arithmetic, wasdenedbyoneofitsearlyproponentsastheartof reasoningbygures, uponthingsrelatingtogovernment. Of more recent vintage than Staatenkunde, political arithmetics rootswere in seventeenth-century England. Making population estimates and construct-ingmortalitytablesweretwooftheproblemsitfrequentlydealtwith. Inspirit,political arithmetic was similar to what is now called demography.The third component was the development of a calculus of probability. As wesawearlier, thiswasamovementthatessentiallystartedinseventeenth-centuryFrance in response to certain gambling questions, but it quickly became the enginefor analyzing all kinds of data.Staatenkunde: The Comparative Description of StatesTheneedforgatheringinformationonthecustomsandresourcesofnationshasbeen obvious since antiquity. Aristotle is credited with the rst major effort towardthat objective:HisPoliteiai, written in the fourth century b.c., containeddetaileddescriptions of some 158 different city-states. Unfortunately, the thirst for knowl-edge that led to the Politeiai fell victim to the intellectual drought of the Dark Ages,and almost two thousand years elapsed before any similar projects of like magnitudewere undertaken.The subject resurfaced during the Renaissance, and the Germans showed themost interest. They not only gave it a name, Staatenkunde, meaning the compara-tive description of states, but they were also the rst (in 1660) to incorporate thesubject into a university curriculum. A leading gure in the German movement wasGottfried Achenwall, who taught at the University of Gttingen during the middleof the eighteenth century. Among Achenwalls claims to fame is that he was the rstto use the word statistics in print. It appeared in the preface of his 1749 book Abrissder Statswissenschaft der heutigen vornehmsten europaishen Reiche und Republiken.(ThewordstatisticscomesfromtheItalianrootstato, meaningstate,implyingthat a statistician is someone concerned with government affairs.) As terminology,it seems to have been well-received: For almost one hundred years the word statisticscontinued to be associated with the comparative description of states. In the middleof the nineteenth century, though, the term was redened, and statistics became thenew name for what had previously been called political arithmetic.How important was the work of Achenwall and his predecessors to the devel-opment of statistics? That would be difcult to say. To be sure, their contributionswere more indirect than direct. They left no methodology and no general theory. But12 Chapter 1Introductionthey did point out the need for collecting accurate data and, perhaps more impor-tantly, reinforced the notion that something complexeven as complex as an entirenationcan be effectively studied by gathering information on its component parts.Thus, they were lending important support to the then-growing belief that induction,rather than deduction, was a more sure-footed path to scientic truth.Political ArithmeticIn the sixteenth century the English government began to compile records, calledbills of mortality, on a parish-to-parish basis, showing numbers of deaths and theirunderlying causes. Their motivation largely stemmed fromthe plague epidemics thathad periodically ravaged Europe in the not-too-distant past and were threatening tobecome a problemin England. Certain government ofcials, including the very inu-ential Thomas Cromwell, felt that these bills would prove invaluable in helping tocontrol the spread of an epidemic. At rst, the bills were published only occasionally,but by the early seventeenth century they had become a weekly institution.2Figure 1.3.3 (on the next page) shows a portion of a bill that appeared in Londonin 1665. The gravity of the plague epidemic is strikingly apparent when we look atthe numbers at the top: Out of 97,306 deaths, 68,596 (over 70%) were caused bythe plague. The breakdown of certain other afictions, though they caused fewerdeaths, raises some interesting questions. What happened, for example, to the 23people who were frighted or to the 397 who suffered from rising of the lights?AmongthefaithfulsubscriberstothebillswasJohnGraunt,aLondonmer-chant. Graunt not onlyreadthebills, hestudiedthemintently. Helookedforpatterns, computeddeathrates, devisedwaysofestimatingpopulationsizes, andevensetupaprimitivelifetable.Hisresultswerepublishedinthe1662treatiseNatural and Political Observations upon the Bills of Mortality. This work was a land-mark: Graunt had launched the twin sciences of vital statistics and demography, and,although the name came later, it also signaled the beginning of political arithmetic.(Graunt did not have to wait long for accolades; in the year his book was published,he was elected to the prestigious Royal Society of London.)High on the list of innovations that made Graunts work unique were his objec-tives. Not content simply to describe a situation, although he was adept at doing so,Graunt often sought to go beyond his data and make generalizations (or, in currentstatistical terminology,drawinferences).Havingbeenblessedwiththisparticularturn of mind, he almost certainly qualies as the worlds rst statistician. All Grauntreally lacked was the probability theory that would have enabled him to frame hisinferences more mathematically. That theory, though, was just beginning to unfoldseveral hundred miles away in France (151).Other seventeenth-century writers were quick to follow through on Grauntsideas. William Pettys Political Arithmetick was published in 1690, although it hadprobably been written some fteen years earlier. (It was Petty who gave the move-ment its name.) Perhaps even more signicant were the contributions of EdmundHalley (of Halleys comet fame). Principally an astronomer, he also dabbled inpolitical arithmetic,andin 1693wroteAnEstimateoftheDegreesoftheMortal-ity of Mankind, drawn from Curious Tables of the Births and Funerals at the city ofBreslaw; with an attempt to ascertain the Price of Annuities upon Lives. (Book titles2An interesting account of the bills of mortality is given in Daniel Defoes A Journal of the Plague Year, whichpurportedly chronicles the London plague outbreak of 1665.1.3 A Brief History 13Thebill fortheyearAGeneral Bill forthispresentyear, endingthe19ofDecember, 1665, accordingtotheReportmadetotheKingsmostexcellentMajesty, by the Co. of Parish Clerks of Lond., & c.gives the following sum-mary of the results; the details of the several parishes we omit, they being madeas in 1625, except that the out-parishes were now 12:Buried in the 27 Parishes within the walls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15,207Whereof of the plague . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9,887Buried in the 16 Parishes without the walls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41,351Whereof of the plague. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28,838At the Pesthouse, total buried. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159Of the plague. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156Buried in the 12 out-Parishes in Middlesex and surrey . . . . . . . . . . . . . . . . . . 18,554Whereof of the plague . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21,420Buried in the 5 Parishes in the City and Liberties of Westminster. . . . . . . . 12,194Whereof the plague. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8,403The total of all the christenings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9,967The total of all the burials this year . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97,306Whereof of the plague . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68,596Abortive and Stillborne . . . . . . . . . . 617 Griping in the Guts . . . . . . . . . . . . . . . . 1,288 Palsie . . . . . . . . . . . . . . . . . . . . . . 30Aged . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1,545 Hangd & made away themselved . . 7 Plague . . . . . . . . . . . . . . . . . . . . . 68,596Ague & Feaver . . . . . . . . . . . . . . . . . . 5,257 Headmould shot and mould fallen. . 14 Plannet . . . . . . . . . . . . . . . . . . . . 6Appolex and Suddenly . . . . . . . . . . . 116 Jaundice . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Plurisie . . . . . . . . . . . . . . . . . . . . 15Bedrid. . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Impostume . . . . . . . . . . . . . . . . . . . . . . . . 227 Poysoned . . . . . . . . . . . . . . . . . . 1Blasted . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Kill by several accidents . . . . . . . . . . . 46 Quinsie . . . . . . . . . . . . . . . . . . . . 35Bleeding . . . . . . . . . . . . . . . . . . . . . . . . . 16 Kings Evill . . . . . . . . . . . . . . . . . . . . . . . . 86 Rickets . . . . . . . . . . . . . . . . . . . . 535Cold & Cough . . . . . . . . . . . . . . . . . . . 68 Leprosie. . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Rising of the Lights . . . . . . . . 397Collick & Winde . . . . . . . . . . . . . . . . . 134 Lethargy . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Rupture . . . . . . . . . . . . . . . . . . . 34Comsumption & Tissick . . . . . . . . . . 4,808 Livergrown. . . . . . . . . . . . . . . . . . . . . . . . 20 Scurry . . . . . . . . . . . . . . . . . . . . . 105Convulsion & Mother . . . . . . . . . . . . 2,036 Bloody Flux, Scowring & Flux . . . . . 18 Shingles & Swine Pox . . . . . . 2Distracted . . . . . . . . . . . . . . . . . . . . . . . 5 Burnt and Scalded . . . . . . . . . . . . . . . . . 8 Sores, Ulcers, Broken andDropsie & Timpany . . . . . . . . . . . . . . 1,478 Calenture . . . . . . . . . . . . . . . . . . . . . . . . . 3 Bruised Limbs . . . . . . . . . . . . . 82Drowned . . . . . . . . . . . . . . . . . . . . . . . . 50 Cancer, Cangrene & Fistula . . . . . . . . 56 Spleen . . . . . . . . . . . . . . . . . . . . . 14Executed . . . . . . . . . . . . . . . . . . . . . . . . 21 Canker and Thrush . . . . . . . . . . . . . . . . 111 Spotted Feaver & Purples . . 1,929Flox & Smallpox . . . . . . . . . . . . . . . . . 655 Childbed . . . . . . . . . . . . . . . . . . . . . . . . . . 625 Stopping of the Stomach . . . 332Found Dead in streets, elds, &c. . 20 Chrisomes and Infants . . . . . . . . . . . . . 1,258 Stone and Stranguary . . . . . . 98French Pox . . . . . . . . . . . . . . . . . . . . . . 86 Meagrom and Headach . . . . . . . . . . . . 12 Surfe . . . . . . . . . . . . . . . . . . . . . . 1,251Frighted . . . . . . . . . . . . . . . . . . . . . . . . . 23 Measles . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Teeth & Worms . . . . . . . . . . . . 2,614Gout & Sciatica . . . . . . . . . . . . . . . . . . 27 Murthered & Shot . . . . . . . . . . . . . . . . . 9 Vomiting. . . . . . . . . . . . . . . . . . . 51Grief . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Overlaid & Starved . . . . . . . . . . . . . . . . 45 Wenn . . . . . . . . . . . . . . . . . . . . . . 8Christened-Males . . . . . . . . . . . . . . . . 5,114 Females . . . . . . . . . . . . . . . . . . . . . . . . . . . 4,853 In all . . . . . . . . . . . . . . . . . . . . . . . 9,967Buried-Males . . . . . . . . . . . . . . . . . . . . 58,569 Females . . . . . . . . . . . . . . . . . . . . . . . . . . . 48,737 In all . . . . . . . . . . . . . . . . . . . . . . . 97,306Of the Plague . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68,596Increase in the Burials in the 130 Parishes and the Pesthouse this year. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79,009Increase of the Plague in the 130 Parishes and the Pesthouse this year . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68,590Figure 1.3.3were longer then!) Halley shored up, mathematically, the efforts of Graunt and oth-ers to construct an accurate mortality table. In doing so, he laid the foundation forthe important theory of annuities. Today, all life insurance companies base their pre-miumschedules on methods similar to Halleys. (The rst company to followhis leadwas The Equitable, founded in 1765.)For all its initial urry of activity, political arithmetic did not fare particularlywell in the eighteenth century, at least in terms of having its methodology ne-tuned.Still, the second half of the century did see some notable achievements in improvingthe quality of the databases: Several countries, including the United States in 1790,14 Chapter 1Introductionestablished a periodic census. To some extent, answers to the questions that inter-ested Graunt and his followers had to be deferred until the theory of probabilitycould develop just a little bit more.Quetelet: The CatalystWith political arithmetic furnishing the data and many of the questions, and the the-ory of probability holding out the promise of rigorous answers, the birth of statisticswas at hand. All that was needed was a catalystsomeone to bring the two together.Several individuals served with distinction in that capacity. Carl Friedrich Gauss, thesuperb German mathematician and astronomer, was especially helpful in showinghow statistical concepts could be useful in the physical sciences. Similar efforts inFrance were made by Laplace. But the man who perhaps best deserves the title ofmatchmaker was a Belgian, Adolphe Quetelet.Quetelet was a mathematician, astronomer, physicist, sociologist, anthropolo-gist, and poet. One of his passions was collecting data, and he was fascinated by theregularity of social phenomena. In commenting on the nature of criminal tendencies,he once wrote (70):Thus we pass from one year to another with the sad perspective of seeing the samecrimes reproduced in the same order and calling down the same punishments in thesame proportions. Sad condition of humanity! . . . We might enumerate in advancehow many individuals will stain their hands in the blood of their fellows, how manywill be forgers, how many will be poisoners, almost we can enumerate in advance thebirths and deaths that should occur. There is a budget which we pay with a frightfulregularity; it is that of prisons, chains and the scaffold.Given such an orientation, it was not surprising that Quetelet would see in prob-ability theory an elegant means for expressing human behavior. For much of thenineteenth century he vigorously championed the cause of statistics, and as a mem-ber of more than one hundred learned societies, his inuence was enormous. Whenhe died in 1874, statistics had been brought to the brink of its modern era.1.4A Chapter SummaryThe concepts of probability lie at the very heart of all statistical problems. Acknowl-edging that fact, the next two chapters take a close look at some of those concepts.Chapter 2 states the axioms of probability and investigates their consequences. Italso covers the basic skills for algebraically manipulating probabilities and gives anintroduction to combinatorics, the mathematics of counting. Chapter 3 reformulatesmuch of the material in Chapter 2 in terms of random variables, the latter being aconcept of great convenience in applying probability to statistics. Over the years,particularmeasuresofprobabilityhaveemergedasbeingespeciallyuseful: Themost prominent of these are proled in Chapter 4.OurstudyofstatisticsproperbeginswithChapter5, whichisarstlookatthe theory of parameter estimation. Chapter 6 introduces the notion of hypothesistesting, a procedure that, in one form or another, commands a major share of theremainderofthebook. Fromaconceptual standpoint, theseareveryimportantchapters: Mostformal applicationsofstatistical methodologywill involveeitherparameter estimation or hypothesis testing, or both.1.4 A Chapter Summary 15AmongtheprobabilityfunctionsfeaturedinChapter4, thenormaldistribu-tionmore familiarly known as the bell-shaped curveis sufciently important tomerit even further scrutiny. Chapter 7 derives in some detail many of the propertiesand applications of the normal distribution as well as those of several related prob-ability functions. Much of the theory that supports the methodology appearing inChapters 9 through 13 comes from Chapter 7.Chapter8describessomeof thebasicprinciplesof experimental design.Its purpose is to provide a framework for comparing and contrasting the variousstatistical procedures proled in Chapters 9 through 14.Chapters 9, 12, and 13 continue the work of Chapter 7, but with the emphasison the comparison of several populations, similar to what was done in Case Study1.2.2. Chapter 10 looks at the important problem of assessing the level of agreementbetween a set of data and the values predicted by the probability model from whichthose data presumably came. Linear relationships are examined in Chapter 11.Chapter 14 is an introduction to nonparametric statistics. The objective there isto develop procedures for answering some of the same sorts of questions raised inChapters 7, 9, 12, and 13, but with fewer initial assumptions.As a general format, each chapter contains numerous examples and case stud-ies, the latter including actual experimental data taken from a variety of sources,primarily newspapers, magazines, and technical journals. We hope that these appli-cations will make it abundantly clear that, while the general orientation of this textis theoretical, the consequences of that theory are never too far from having directrelevance to the real world.Chapt erProbability22.1 Introduction2.2 Sample Spaces and the Algebra of Sets2.3 The Probability Function2.4 Conditional Probability2.5 Independence2.6 Combinatorics2.7 Combinatorial Probability2.8 Taking a Second Look at Statistics(Monte Carlo Techniques)One of the most inuential of seventeenth-century mathematicians, Fermat earned hisliving as a lawyer and administrator in Toulouse. He shares credit with Descartes forthe invention of analytic geometry, but his most important work may have been innumber theory. Fermat did not write for publication, preferring instead to send lettersand papers to friends. His correspondence with Pascal was the starting point for thedevelopment of a mathematical theory of probability.Pierre de Fermat (16011665)Pascal was the son of a nobleman. A prodigy of sorts, he had already published atreatise on conic sections by the age of sixteen. He also invented one of the earlycalculating machines to help his father with accounting work. Pascals contributionsto probability were stimulated by his correspondence, in 1654, with Fermat. Laterthat year he retired to a life of religious meditation.Blaise Pascal (16231662)2.1IntroductionExperts have estimated that the likelihood of any given UFO sighting being genuineis on the order of one in one hundred thousand. Since the early 1950s, some tenthousand sightings have been reported to civil authorities. What is the probabilitythat at least one of those objects was, in fact, an alien spacecraft? In 1978, Pete Roseof the Cincinnati Reds set a National League record by batting safely in forty-fourconsecutivegames. Howunlikelywasthatevent, giventhatRosewasalifetime.303 hitter? By denition, the mean free path is the average distance a molecule in agas travels before colliding with another molecule. Howlikely is it that the distance amolecule travels between collisions will be at least twice its mean free path? Supposeaboysmotherandfatherbothhavegeneticmarkersforsicklecellanemia, butneither parent exhibits any of the diseases symptoms. What are the chances thattheir son will also be asymptomatic? What are the odds that a poker player is dealt162.1 Introduction 17afullhouseorthatacraps-shootermakeshispoint?Ifawomanhaslivedtoage seventy, how likely is it that she will die before her ninetieth birthday? In 1994,Tom Foley was Speaker of the House and running for re-election. The day after theelection, his race had still not been called by any of the networks: he trailed hisRepublican challenger by 2174 votes, but 14,000 absentee ballots remained to becounted. Foley, however, conceded. Should he have waited for the absentee ballotsto be counted, or was his defeat at that point a virtual certainty?As the nature and variety of these questions would suggest, probability is a sub-ject with an extraordinary range of real-world, everyday applications. What beganasanexerciseinunderstandinggamesofchancehasproventobeusefulevery-where. Maybe even more remarkable is the fact that the solutions to all of thesediverse questions are rooted in just a handful of denitions and theorems. Thoseresults, together with the problem-solving techniques they empower, are the sumand substance of Chapter 2. We begin, though, with a bit of history.The Evolution of the Denition of ProbabilityOver the years, the denition of probability has undergone several revisions. Thereis nothing contradictory in the multiple denitionsthe changes primarily reectedthe need for greater generality and more mathematical rigor. The rst formulation(often referred to as the classical denition of probability) is credited to GerolamoCardano (recall Section 1.3). It applies only to situations where (1) the number ofpossible outcomes is nite and (2) all outcomes are equally likely. Under those con-ditions, the probability of an event comprised of m outcomes is the ratio m/n, wheren is the total number of (equally likely) outcomes. Tossing a fair, six-sided die, forexample, gives m/n =36as the probability of rolling an even number (that is, either2, 4, or 6).While Cardanos model was well-suited to gambling scenarios (for which it wasintended), it was obviously inadequate for more general problems, where outcomesarenotequallylikelyand/orthenumberofoutcomesisnotnite. RichardvonMises, atwentieth-centuryGermanmathematician, isoftencreditedwithavoid-ing the weaknesses in Cardanos model by dening empirical probabilities. In thevon Mises approach, we imagine an experiment being repeated over and over againunder presumably identical conditions. Theoretically, a running tally could be keptof the number of times(m) the outcome belongs to a given event divided by n, thetotal number of times the experiment is performed. According to von Mises, theprobabilityofthegiveneventisthelimit(asngoestoinnity)oftheratiom/n.Figure 2.1.1 illustrates the empirical probability of getting a head by tossing a faircoin: as the number of tosses continues to increase, the ratio m/n converges to12.Figure 2.1.110 1n = numbers of trials2 3 4 512mnlimm/nn18 Chapter 2ProbabilityThe von Mises approach denitely shores up some of the inadequacies seen inthe Cardano model, but it is not without shortcomings of its own. There is someconceptual inconsistency, for example, in extolling the limit of m/n as a way of den-ing a probability empirically, when the very act of repeating an experiment underidentical conditions an innite number of times is physically impossible. And leftunanswered is the question of how large nmust be in order for m/nto be a goodapproximation for lim m/n.Andrei Kolmogorov, the great Russian probabilist, took a different approach.Aware that many twentieth-century mathematicians were having success developingsubjects axiomatically, Kolmogorov wondered whether probability might similarlybe denedoperationally, rather than as a ratio (like the Cardano model) or as alimit (like the von Mises model). His efforts culminated in a masterpiece of mathe-matical elegance when he published Grundbegriffe der Wahrscheinlichkeitsrechnung(Foundations of the Theory of Probability) in 1933. In essence, Kolmogorov was ableto show that a maximum of four simple axioms is necessary and sufcient to denethe way any and all probabilities must behave. (These will be our starting point inSection 2.3.)WebeginChapter2withsomebasic(and, presumably, familiar)denitionsfrom set theory. These are important because probability will eventually be denedas a set functionthat is, a mapping from a set to a number. Then, with the helpof Kolmogorovs axioms in Section 2.3, we will learn how to calculate and manipu-late probabilities. The chapter concludes with an introduction to combinatoricsthemathematics of systematic countingand its application to probability.2.2Sample Spaces and the Algebra of SetsThe starting point for studying probability is the denition of four key terms: exper-iment, sampleoutcome, samplespace, andevent. Thelatterthree, all carryoversfrom classical set theory, give us a familiar mathematical framework within which towork; the former is what provides the conceptual mechanism for casting real-worldphenomena into probabilistic terms.By an experimentwe will mean any procedure that (1) can be repeated, the-oretically, aninnitenumberoftimes; and(2)hasawell-denedsetofpossibleoutcomes. Thus, rolling a pair of dice qualies as an experiment; so does measuringa hypertensives blood pressure or doing a spectrographic analysis to determine thecarbon content of moon rocks. Asking a would-be psychic to draw a picture of animage presumably transmitted by another would-be psychic does not qualify as anexperiment, because the set of possible outcomes cannot be listed, characterized, orotherwise dened.Each of the potential eventualities of an experiment is referred to as a sampleoutcome, s, and their totality is called the sample space, S. To signify the membershipof sinS, we write s S. Any designated collection of sample outcomes, includingindividual outcomes, the entire sample space, and the null set, constitutes an event.The latter is said to occur if the outcome of the experiment is one of the membersof the event.Example2.2.1Consider the experiment of ipping a coin three times. What is the sample space?Which sample outcomes make up the eventA: Majority of coins show heads?Think of each sample outcome here as an ordered triple, its components repre-senting the outcomes of the rst, second, and third tosses, respectively. Altogether,2.2 Sample Spaces and the Algebra of Sets 19there are eight different triples, so those eight comprise the sample space:S ={HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}By inspection, we see that four of the sample outcomes in S constitute the eventA:A ={HHH, HHT, HTH, THH}Example2.2.2Imagine rolling two dice, the rst one red, the second one green. Each sample out-come is an ordered pair (face showing on red die, face showing on green die), andthe entire sample space can be represented as a 6 6 matrix (see Figure 2.2.1).AFace showing on green dieFace showing on red die 1234561(1, 1)(1, 2)(1, 3)(1, 4)(1, 5)(1, 6)2(2, 1)(2, 2)(2, 3)(2, 4)(2, 5)(2, 6)3(3, 1)(3, 2)(3, 3)(3, 4)(3, 5)(3, 6)4(4, 1)(4, 2)(4, 3)(4, 4)(4, 5)(4, 6)5(5, 1)(5, 2)(5, 3)(5, 4)(5, 5)(5, 6)6(6, 1)(6, 2)(6, 3)(6, 4)(6, 5)(6, 6)Figure 2.2.1Gamblers are often interested in the eventA that the sum of the faces showingis a 7. Notice in Figure 2.2.1 that the sample outcomes contained inA are the sixdiagonal entries, (1, 6), (2, 5), (3, 4), (4, 3), (5, 2), and (6, 1).Example2.2.3A local TV station advertises two newscasting positions. If three women(W1, W2,W3) andtwomen (M1, M2) apply, the experiment of hiring twocoanchorsgenerates a sample space of ten outcomes:S ={(W1, W2), (W1, W3), (W2, W3), (W1, M1), (W1, M2), (W2, M1),(W2, M2), (W3, M1), (W3, M2), (M1, M2)}Does it matter here that the two positions being lled are equivalent? Yes. If thestationwereseekingtohire, say, asportsannouncerandaweatherforecaster,the number of possible outcomes would be twenty: (W2, M1), for example, wouldrepresent a different stafng assignment than (M1, W2).Example2.2.4The number of sample outcomes associatedwithanexperiment neednot benite. Supposethatacoinistosseduntilthersttailappears. Ifthersttossisitself a tail, the outcome of the experiment is T; if the rst tail occurs on the secondtoss, the outcome is HT; and so on. Theoretically, of course, the rst tail may neveroccur, and the innite nature of S is readily apparent:S ={T, HT, HHT, HHHT, . . .}Example2.2.5There are three ways to indicate an experiments sample space. If the number of pos-sible outcomes is small, we can simply list them, as we did in Examples 2.2.1 through2.2.3. In some cases it may be possible to characterize a sample space by showing thestructureitsoutcomesnecessarilypossess. ThisiswhatwedidinExample2.2.4.20 Chapter 2ProbabilityA third option is to state a mathematical formula that the sample outcomes mustsatisfy.Acomputer programmer is running a subroutine that solves a generalquadraticequation, ax2+bx +c = 0.Herexperimentconsistsofchoosingval-ues for the three coefcients a, b, and c. Dene (1) S and (2) the eventA: Equationhas two equal roots.First, we must determine the sample space. Since presumably no combinationsof nitea,b, andcare inadmissible, we can characterizeSby writing a series ofinequalities:S ={(a, b, c) : 1 if y 0.90. What is the tests level of signicance?6.4.15. A series of n Bernoulli trials is to be observed asdata for testingH0: p =12versusH1: p >12Thenull hypothesiswill berejectedif k, theobservednumberofsuccesses, equalsn. Forwhatvalueof pwillthe probability of committing a Type II error equal 0.05?6.5 A Notion of Optimality: The Generalized Likelihood Ratio 3796.4.16. LetX1 be a binomial random variable with n =2and pX1 =P(success). Let X2beanindependent bino-mial random variable with n =4 and pX2 = P(success). LetX = X1 + X2. Calculate ifH0: pX1 = pX2 =12versusH1: pX1 = pX2 >12is to be tested by rejecting the null hypothesis when k 5.6.4.17. Asampleof size1fromthepdf fY(y) =(1 +)y, 0 y 1, is to be the basis for testingH0: =1versusH1: 0, is to be used to testH0: =1versus H1: < 1. Thedecisionrulecalls for thenullhypothesis to be rejected if y ln 10. Find as a functionof .6.4.21. A random sample of size 2 is drawn from a uni-formpdf denedover the interval[0, ]. We wishtotestH0: =2versusH1: exponential1.MTB > rmeancl-c6c7MTB > rstdevcl-c6c8MTB > letc9 = sqrt(6)*(((c7)-1.0)/(c8))MTB>histogramc9y 6 s/=6Probability densityf(y) = eYy1.000 40.502ySampledistribution00.40.2f(t)T54 2 2 4t ratio (n = 6)(c)MTB>random100cl-c6;SUBC>poisson5.MTB>rmeancl-c6c7MTB>rstdevcl-c6c8MTB>letc9=sqrt(6)*(((c7)-5.0)/(c8))MTB>histogramc9p(k) =X0.160.0800 2 4 6 8 10Probabilitye 55 kk!k7.4 Drawing Inferences About 409fT5(t ). Specically, very negativet ratios are occurring much more often than theStudent tcurve would predict, while large positive tratios are occurring less often(see Question 7.4.23). But look at Figure 7.4.6(d). When the sample size is increasedto n =15, the skewness so prominent in Figure 7.4.6(b) is mostly gone.Figure 7.4.6 (Continued)Sampledistribution00.40.2f (t)T142 2 4t ratio (n = 15)(d)f(y) = eYy1.000 40.502 6Probability densityMTB > random 100 cl-c15;SUBC> exponential 1.MTB > rmean cl-c15 c16MTB > rstdev cl-c15 c17MTB > let c18= sqrt(15)*(((c16 - 1.0)/(c17))MTB > histogram c18yReectedinthesespecicsimulationsaresomegeneral propertiesof thetratio:1. The distribution ofYS/nis relatively unaffected by the pdf of the yis [providedfY(y) is not too skewed and n is not too small].2. As n increases, the pdf ofYS/nbecomes increasingly similar tofTn1(t ).In mathematical statistics, the term robust is used to describe a procedure that is notheavily dependent on whatever assumptions it makes. Figure 7.4.6 shows that the ttest is robust with respect to departures from normality.From a practical standpoint, it would be difcult to overstate the importanceof the t test being robust. If the pdf ofYS/nvaried dramatically depending on theoriginoftheyis,wewouldneverknowifthetrueassociatedwith,say,a0.05decision rule was anywhere near 0.05. That degree of uncertainty would make the ttest virtually worthless.410 Chapter 7Inferences Based on the Normal DistributionQuestions7.4.23. Explainwhythedistributionof t ratios calcu-latedfromsmall samples drawnfromtheexponentialpdf, fY(y) = ey, y 0, willbeskewedtotheleft[recallFigure 7.4.6(b)]. [Hint: What does the shape offY(y) implyaboutthepossibilityofeachyibeingcloseto0?Iftheentiresampledidconsist of yiscloseto0, what valuewould the t ratio have?]7.4.24.Supposeonehundredsamplesofsizen = 3aretaken from each of the pdfs(1) fY(y) =2y, 0 y 1and(2) fY(y) =4y3, 0 y 1and for each set of three observations, the ratioy s/3iscalculated, whereistheexpectedvalueofthepar-ticular pdf beingsampled. Howwouldyouexpect thedistributions of the two sets of ratios to be different? Howwould they be similar? Be as specic as possible.7.4.25. Suppose that random samples of size n are drawnfrom the uniform pdf, fY(y) =1, 0 y 1. For each sam-ple, theratiot =y0.5s/niscalculated. Parts(b)and(d)ofFigure 7.4.6 suggest that the pdf of twill become increas-inglysimilarto fTn1(t ) as nincreases. Towhichpdf isfTn1(t ), itself, converging as n increases?7.4.26. On which of the following sets of data would yoube reluctant to do a t test? Explain.y (a)y (b)y (c)7.5Drawing Inferences About 2When random samples are drawn from a normal distribution, it is usually the casethat the parameter is the target of the investigation. More often than not, the meanmirrors theeffectof atreatment or condition,inwhichcaseit makessensetoapply what we learned in Section 7.4that is, either construct a condence intervalfor or test the hypothesis that =o.But exceptions are not that uncommon. Situations occur where the precisionassociated with a measurement is, itself, importantperhaps even more importantthanthemeasurementslocation.Ifso, weneedtoshiftourfocustothescaleparameter,2. Two key facts that we learned earlier about the population variancewill now come into play. First, an unbiased estimator for2based on its maximumlikelihood estimator is the sample variance, S2, whereS2=1n 1n

i =1(Yi Y)2And, second, the ratio(n 1)S22=12n

i =1(Yi Y)2has a chi square distribution with n 1 degrees of freedom. Putting these two piecesof information together allows us to draw inferences about 2in particular, we canconstruct condence intervals for 2and test the hypothesis that 2=2o.Chi Square TablesJust as we need a ttable to carry out inferences about (when 2is unknown), weneed a chi square table to provide the cutoffs for making inferences involving 2. The7.5 Drawing Inferences About 2411layout of chi square tables is dictated by the fact that all chi square pdfs (unlikeZand t distributions) are skewed (see, for example, Figure 7.5.1, showing a chi squarecurve having 5 degrees of freedom). Because of that asymmetry, chi square tablesneed to provide cutoffs for both the left-hand tail and the right-hand tail of each chisquare distribution.Figure 7.5.14 8 12 16 00.150.100.05Probability density1.145 15.086f(y) = (32) yeX1 3/2 y/225Area = 0.05Area = 0.01Figure7.5.2shows thetopportionof thechi squaretablethat appears inAppendix A.3. Successive rows refer to different chi square distributions (each hav-ingadifferentnumberofdegreesoffreedom).Thecolumnheadingsdenotetheareas to the left of the numbers listed in the body of the table.Figure 7.5.2pdf .01 .025 .05 .10 .90 .95 .975 .991 0.000157 0.000982 0.00393 0.0158 2.706 3.841 5.024 6.6352 0.0201 0.0506 0.103 0.211 4.605 5.991 7.378 9.2103 0.115 0.216 0.352 0.584 6.251 7.815 9.348 11.3454 0.297 0.484 0.711 1.064 7.779 9.488 11.143 13.2775 0.554 0.831 1.145 1.610 9.236 11.070 12.832 15.0866 0.872 1.237 1.635 2.204 10.645 12.592 14.449 16.8127 1.239 1.690 2.167 2.833 12.017 14.067 16.013 18.4758 1.646 2.180 2.733 3.490 13.362 15.507 17.535 20.0909 2.088 2.700 3.325 4.168 14.684 16.919 19.023 21.66610 2.558 3.247 3.940 4.865 15.987 18.307 20.483 23.20911 3.053 3.816 4.575 5.578 17.275 19.675 21.920 24.72512 3.571 4.404 5.226 6.304 18.549 21.026 23.336 26.217We will use the symbol 2p,n to denote the number along the horizontal axis thatcuts off, to its left, an area ofp under the chi square distribution with n degrees offreedom. For example, from the fth row of the chi square table, we see the num-bers 1.145 and 15.086 under the column headings .05 and .99, respectively. It followsthatP_25 1.145_=0.05andP_25 15.086_=0.99(see Figure 7.5.1). In terms of the2p,nnotation,1.145 =2.05,5and15.086 =2.99,5.(The area to the right of 15.086, of course, must be 0.01.)412 Chapter 7Inferences Based on the Normal DistributionConstructing Condence Intervals for 2Since(n1)S22has a chi square distribution withn 1 degrees of freedom, we canwriteP_2/2,n1(n 1)S2221/2,n1_=1 (7.5.1)IfEquation7.5.1istheninvertedtoisolate2inthecenteroftheinequalities,the two endpoints will necessarily dene a 100(1 )% condence interval for thepopulation variance. The algebraic details will be left as an exercise.Theorem7.5.1Let s2denote the sample variance calculated from a random sample of n observationsdrawn from a normal distribution with mean and variance 2. Thena. a 100(1 )% condence interval for 2is the set of values_(n 1)s221/2,n1,(n 1)s22/2,n1_b. a 100(1 )% condence interval for is the set of values__(n 1)s221/2,n1,_(n 1)s22/2,n1_

Case Study 7.5.1Thechainofeventsthatdenethegeological evolutionoftheEarthbeganhundreds of millions of years ago. Fossils play a key role in documenting therelativetimesthoseeventsoccurred,buttoestablishanabsolutechronology,scientists rely primarily on radioactive decay.One of the newest dating techniques uses a rocks potassium-argon ratio.Almostallmineralscontainpotassium(K)aswellascertainofitsisotopes,including40K. The latter, though, is unstable and decays into isotopes of argonand calcium, 40Ar and 40Ca. By knowing the rates at which the various daughterproducts are formed and by measuring the amounts of40Ar and40K present ina specimen, geologists can estimate the objects age.Critical to the interpretation of any such dates, of course, is the precisionof the underlying procedure. One obvious way to estimate that precision is touse the technique on a sample of rocks known to have the same age. Whatevervariation occurs, then, from rock to rock is reecting the inherent precision (orlack of precision) of the procedure.Table7.5.1liststhepotassium-argonestimatedagesofnineteenmineralsamples, all takenfromtheBlackForest insoutheasternGermany(111).Assumethat theprocedures estimatedages arenormallydistributedwith(unknown) mean and (unknown) variance2. Construct a 95% condenceinterval for .(Continued on next page)7.5 Drawing Inferences About 2413Table 7.5.1Specimen Estimated Age (millions of years)1 2492 2543 2434 2685 2536 2697 2878 2419 27310 30611 30312 28013 26014 25615 27816 34417 30418 28319 310Here19

i =1yi =526119

i =1y2i =1,469,945so the sample variance is 733.4:s2= 19(1,469,945) (5261)219(18)=733.4Since n =19, the critical values appearing in the left-hand and right-hand limitsof the condence interval come from the chi square pdf with 18 df. Accordingto Appendix Table A.3,P_8.23 0 isP_Y 0S/n0at the =0.05 level of sig-nicance. Let n =20. In this case the test is to rejectH0if the test statisticy0s/nisgreater than t.05,19=1.7291. What will be the Type II error if the mean has shiftedby 0.5 standard deviation to the right of 0?Saying that the mean has shifted by 0.5 standard deviation to the right of0isequivalent tosetting10= 0.5. Inthat case, thenoncentralityparameteris =10/n =(0.5) 20 =2.236.The probability of a Type II error isP(T19,2.2361.7291)where T19,2.236 is a noncentral t variable with 19 degrees of freedom and noncentral-ity parameter 2.236.420 Chapter 7Inferences Based on the Normal DistributionTo calculate this quantity, we need the cdf of T19,2.236. Fortunately, many statis-tical software programs have this function. The Minitab commands for calculatingthe desired probability areMTB > CDF 1.7291;SUBC > T 19 2.236with outputCumulative Distribution FunctionStudents t distribution with 19 DF and noncentrality parameter 2.236x P(X set c1DATA > 2.5 3.2 0.5 0.4 0.3 0.1 0.1 0.2 7.4 8.6 0.2 0.1DATA > 0.4 1.8 0.3 1.3 1.4 11.2 2.1 10.1DATA > endMTB > describe c1Descriptive Statistics: C1Variable N N* Mean SE Mean StDev Minimum Q1 Median Q3 MaximumC1 20 0 2.610 0.809 3.617 0.100 0.225 0.900 3.025 11.200Here,N=sample sizeN*=number of observations missing from c1 (that is, thenumber of interior blanks)Mean=sample mean = ySE Mean=standard error of the mean=snStDev=sample standard deviation=sMinimum=smallest observationQ1=first quartile=25th percentileMedian=middle observation (in terms of magnitude), oraverage of the middle two if n is evenQ3=third quartile=75th percentileMaximum=largest observationDescribing Samples Using Minitab Windows1. Enter data under C1 in the WORKSHEET. Click on STAT, then on BASICSTATISTICS, then on DISPLAY DESCRIPTIVE STATISTICS.2. Type C1 in VARIABLES box; click on OK.Percentiles of chi square, t , andFdistributions canbeobtainedusingtheINVCDF command introduced in Appendix 3.A.1. Figure 7.A.1.2 shows the syntaxfor printing out 2.95,6(=12.5916) and F.01,4,7(=0.0667746).422 Chapter 7Inferences Based on the Normal DistributionFigure 7.A.1.2 MTB > invcdf 0.95;SUBC > chisq 6.Inverse Cumulative Distribution FunctionChi-Square with 6 DFP(X invcdf 0.01;SUBC> f 4 7.Inverse Cumulative Distribution FunctionF distribution with 4 DF in numerator and 7 DF in denominatorP(X invcdf 0.90;SUBC> t 13.Inverse Cumulative Distribution FunctionStudents t distribution with 13 DFP(X set c1DATA > 62 52 68 23 34 45 27 42 83 56 40DATA > endMTB > tinterval 0.95 c1One-Sample T: C1Variable N Mean StDev SE Mean 95% CIC1 11 48.36 18.08 5.45 (36.21, 60.51)Constructing Condence Intervals Using Minitab Windows1. Enter data under C1 in the WORKSHEET.2. Click on STAT, then on BASIC STATISTICS, then on 1-SAMPLE T.3. Enter C1 in the SAMPLES IN COLUMNS box, click on OPTIONS, and enterthe value of 100(1 ) in the CONFIDENCE LEVEL box.4. Click on OK. Click on OK.Figure 7.A.1.5 shows the input and output for doing a t test on the approval datagiven in Table 7.4.3. The basic command is TTEST X Y, where X is the value ofo and Yis the column where the data are stored. If no other punctuation is used,Appendix 7.A.2 Some Distribution Results for Y and S2423Figure 7.A.1.5MTB > set c1DATA > 59 65 69 53 60 53 58 64 46 67 51 59DATA > endMTB > ttest 62 c1One-Sample T: C1Test of mu = 62 vs not = 62Variable N Mean StDev SE Mean 95% CIT PC1 12 58.66 6.95 2.01 (54.25,63.08)-1.66 0.125the program automatically takesH1 to be two-sided. If a one-sided test to the rightis desired, we writeMTB > ttest X Y;SUBC > alternative +1.For a one-sided test to the left, the subcommand becomes alternative 1.Notice that no value for is entered, and that the conclusion is not phrased aseither Accept H0 or Reject H0. Rather, the analysis ends with the calculation ofthe datas P-value.Here,P-value = P(T111.66) + P(T111.66)=0.0626 +0.0626=0.125(recall Denition6.2.4). Sincethe P-valueexceedstheintended(= 0.05), theconclusion is Fail to reject H0.Testing H0: =o Using Minitab Windows1. Enter data under C1 in the WORKSHEET.2. Click on STAT, then on BASIC STATISTICS, then on 1-SAMPLE T.3. Type C1 in SAMPLES IN COLUMNS box; click on PERFORM HYPOTH-ESIS TEST and enter the value ofo. Click on OPTIONS, then choose NOTEQUAL.4. Click on OK; then click on OK.Appendix 7.A.2 Some Distribution Results for Y and S2Theorem7.A.2.1Let Y1, Y2, . . . , Yn be a random sample of size n from a normal distribution with mean and variance 2. DeneY = 1nn

i =1Yiand S2=1n 1n

i =1(Yi Y)2Thena. Yand S2are independent.b.(n1)S22has a chi square distribution with n 1 degrees of freedom.Proof The proof of this theorem relies on certain linear algebra techniques as wellas a change-of-variables formula for multiple integrals. Denition 7.A.2.1 and theLemma that follows review the necessary background results. For further details,see (44) or (213). 424 Chapter 7Inferences Based on the Normal DistributionDenition7.A.2.1.a. A matrixA is said to be orthogonal ifAAT= I .b. Let beanyn-dimensional vectoroverthereal numbers. Thatis, =(c1, c2, . . . , cn), whereeachcjisareal number. Thelengthof will bedened as =_c21+ +c2n_1/2(Note that 2=T.)Lemma a. A matrixA is orthogonal if and only if A = for each b. If a matrixA is orthogonal, then detA =1.c. Let g be a one-to-one continuous mapping on a subset, D, of n-space. Then_g(D)f (x1, . . . , xn) dx1 dxn=_Df [g(y1, . . . , yn)]detJ(g) dy1 dynwhereJ(g) is the Jacobian of the transformation.Set Xi =(Yi )/for i =1, 2, . . . , n. Then all the Xis are N(0, 1). LetA be ann n orthogonal matrix whose last row is _1n,1n, . . . ,1n_. Let

X =(X1, . . . , Xn)Tanddene

Z = (Z1, Z2, . . . , Zn)Tbythetransformation

Z = A

X.[Notethat Zn =_1n_X1+ +_1n_Xn=n X.]For any set D,P(

Z D) = P(A

X D) = P(

X A1D)=_A1DfX1,...,Xn(x1, . . . , xn) dx1 dxn=_DfX1,...,Xn[g(

z)] detJ(g) dz1 dzn=_DfX1,...,Xn(A1

z) 1 dz1 dznwhere g(

z) = A1

z. ButA1is orthogonal, so setting (x1, . . . , xn)T= A1z, we havethatx21 + +x2n =z21+ +z2nThusfX1,...,Xn(

x) =(2)n/2e(1/2)_x21++x2n_=(2)n/2e(1/2)_z21++z2n_From this we conclude thatP(

Z D) =_D(2)n/2e(n/2)_z21++z2n_dz1 dznimplying that the Zjs are independent standard normals.Appendix 7.A.3 A Proof that the One-Sample t Test is a GLRT 425Finally,n

j =1Z2j =n1

j =1Z2j +nX2=n

j =1X2j =n

j =1(Xj X)2+nX2Therefore,n1

j =1Z2j =n

j =1(Xj X)2and X2(and thus X) is independent ofn

j =1(Xi X)2, so the conclusion fol-lowsforstandardnormal variables. Also, sinceY = X + andn

i =1(Yi Y)2=2n

i =1(Xi X)2, the conclusion follows for N(, 2) variables.Comment As part of the proof just presented, we established a version of Fisherslemma:Let X1, X2, . . . , Xnbeindependent standardnormal randomvariablesandlet Abe an orthogonal matrix. Dene(Z1, . . . , Zn)T= A(X1, . . . , Xn)T. Then theZis areindependent standard normal random variables.Appendix 7.A.3 A Proof that the One-Sample t Test is a GLRTTheorem7.A.3.1The one-sample t test, as outlined in Theorem 7.4.2, is a GLRT.Proof Consider the test ofH0: =o versusH1: =o. The two parameter spacesrestricted to H0 and H0 H1that is, and , respectivelyare given by={(, 2): =0; 0 20.225 0.262 0.217 0.240 0.230 0.229 0.235 0.217DATA >endMTB >set c2DATA >0.209 0.205 0.196 0.210 0.202 0.207 0.224 0.223 0.220 0.201DATA >endMTB >name c1 X c2 YMTB >twosample c1 c2Two-Sample T-Test and CI: X, YTwo-sample T for X vs YN Mean StDev SE MeanX 8 0.2319 0.0146 0.0051Y 10 0.20970 0.00966 0.0031Difference = mu (X) - mu (Y)Estimate for difference: 0.0221795% CI for difference: (0.00900, 0.03535)T-Test of difference=0(vs not =):T-Value=3.70 P-Value=0.003 DF=11Testing H0:X =Y Using Minitab Windows1. Enter the two samples in C1 and C2, respectively.2. Click on STAT, then on BASIC STATISTICS, then on 2-SAMPLE t.3. ClickonSAMPLESINDIFFERENTCOLUMNS, andtypeC1inFIRST box and C2 in SECOND box.4. ClickonASSUMEEQUALVARIANCES(if a pooledt test isdesired).5. Click on OPTIONS.6. Enter value for 100 (1 ) in CONFIDENCE LEVEL box.7. Click on NOT EQUAL; then click on whichever H1 is desired.8. Click on OK; click on remaining OK.Chapt erGoodness-of-Fit Tests1010.1 Introduction10.2 The Multinomial Distribution10.3 Goodness-of-Fit Tests: All Parameters Known10.4 Goodness-of-Fit Tests: Parameters Unknown10.5 Contingency Tables10.6 Taking a Second Look at Statistics (Outliers)Appendix 10.A.1 Minitab ApplicationsCalled by some the founder of twentieth-century statistics, Pearson received hisuniversity education at Cambridge, concentrating on physics, philosophy, and law.He was called to the bar in 1881 but never practiced. In 1911 Pearson resigned hischair of applied mathematics and mechanics at University College, London, andbecame the first Galton Professor of Eugenics, as was Galtons wish. Together withWeldon, Pearson founded the prestigious journal Biometrika and served as itsprincipal editor from 1901 until his death.Karl Pearson (1857--1936)10.1IntroductionThe give-and-take between the mathematics of probability and the empiricism ofstatistics should be, by now, a comfortably familiar theme. Time and time again wehave seen repeated measurements, no matter their source, exhibiting a regularity ofpattern that can be well approximated by one or more of the handful of probabilityfunctions introduced in Chapter 4. Until now, all the inferences resulting from thisinterfacing have been parameter specic, a fact to which the many hypothesis testsabout means, variances, and binomial proportions paraded forth in Chapters 6, 7,and 9 bear ample testimony. Still, there are other situations where the basic formof pX(k)or fY(y),ratherthanthevalueofitsparameters,isthemostimportantquestion at issue. These situations are the focus of Chapter 10.A geneticist, for example, might want to know whether the inheritance of a cer-tainsetoftraitsfollowsthesamesetofratiosasthoseprescribedbyMendeliantheory. Theobjectiveofapsychologist, ontheotherhand, mightbetoconrmor refute a newly proposed model for cognitive serial learning. Probably the mosthabitual users of inference procedures directed at the entire pdf, though, are statis-ticians themselves: As a prelude to doing any sort of hypothesis test or condenceinterval, an attempt should be made, sample size permitting, to verify that the dataare, indeed, representative of whatever distribution that procedure presumes. Usu-ally, this will mean testing to see whether a set of yis might conceivably represent anormal distributio