DATA EVALUATION AND METHODS RESEARCH Series 2Number 65

Distributionof Variance

and PropertiesEstimators

for ComplexMultistageProbability SamplesAn Empirical Distribution

This report presents results from an empirical investigation of the behavior of thereplication and Iinearization techniques for measuring variance. The study utilizesdata collected in the Health Interview Survey. Most scientific sample surveys are

based on complex multistage probability designs, with design components that

include unequal probabilities of selection of elements in the population,stratification, and several stages of clustering. The estimation procedures usuallyinvolve nonresponse and poststratification adjustments. These actions causeconcern in the methods of estimation of standard errors and their subsequent use in

constructing confidence intervals and testing hypotheses. There are several ways tostudy the characteristics of variance estimates. The method in this report is an

empirical investigation. Data that were collected in a national sample surveybecome the universe and repeated samples are drawn from it. For each sample the

statistics under study were calculated and sampling distributions of the estimateswere generated. The material in this report was taken, in part, from Dr. Bean’sdoctoral dissertation.

DHEW Publication No. (HRA) 75-1339

U.S. DEPARTMENT OF HEALTH, EDUCATION, AND WELFAREPublic Health Service

Health Resources Administration

National Center for Health Statistics

Rockville, Md. March 1975

Library of Congress Cataloging in Publication Data

Bean, Judy A.

Distribution and properties of variance estimators for complex multistage probabilitysamples.

(Series 2: Data evaluation and methods research, no. 65) (DHEW publication no.(HRA) 75-1339)

“Final report.”Supt. of Dots. no.: HE 20.2210:2/651. Sampling (Statistics) 2. Estimation theory. 3. Monte Carlo method. I. Title. II.

Series: United States. National Center for Health Statistics. Vital and health statistics. Series2: Data evaluation and methods research no. 65. III. Series: United States. Dept. of Health,Education, and Welfare. [DNLM: 1. Population surveillance. 2. Probability. 3. Sociometrictechnics. W2A N148vb no. 65]

RA409.U45 no. 65 [QA276.6] 312’.07’23s [519.5’2]ISBN 0-8406-0029-1 74-16356

For sale by the Superintendent of Documents, U.S. Government Printing OfficeWashington, D.C. 20402- Price $1.10

NATIONAL CENTER FOR HEALTH STATISTICS

EDWARD B. PERRIN, Ph. D., Director

PHILIP S. LAWRENCE, SC.D., Deputy Director

GAIL F. FISHER, Associate Director for the Cooperative Health Statistics System

ELIJAH L. WHITE, Associate Director for Data Systems

IWAO M. MORIYAMA, Ph.D., Associate Director for International Statistics

EDWARD E. MINTY, Associate Director for Management

ROBERT A. ISRAEL, Associate Director for Operations

QUENTIN R. REMEIN, Associate Director for Program Development

ALICE HAYWOOD, Information Officer

OFFICE OF STATISTICAL RESEARCH

GEORGE A. SCHNACK, Chiej Research Coordination Staff

MONROE G. SIRKEN, Ph.D., Chiej Measurement Research Laboratory

Vital and Health Statistics-Series 2-No. 65

DHEW Publication No. (HRA) 75-1339

Library of Congress Card Number HRA 74-16356

CONTENTS

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .StrategyoftheStudy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

HighIightsoftheFindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ReviewofLiterature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Replication Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Linearization Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . .An Empirical Investigation of All Three General Variance

Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .DescriptionofthePopulation . . . . . . . . . . . . . . . . . . . . . . . . .Sample Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

EstimatorsandVarianceEstimators . . . . . . . . . . . . . . . . . . . . . . . .Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Variance Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .VarianceEstimatorsforEstimator 1 . . . . . . . . . . . . . . . . . . . . . .

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . ... .. .Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ComparisonofReplicationandLinearization . . . . . . . . . . . . . . . . .NormalityofStandardizedEstimators . . . . . . . . . . . . . . . . . . . . .InfluenceofPoststratification . . . . . . . . . . . . . . . . . . . . . . . . .

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......”.”Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Behavior of Linearization and Replication . . . . . . . . . . . . . . . . . . .NormalityofStandardizedEstimators . . . . . . . . . . . . . . . . . . . . .ComparisonofTwo Ratio Estimators . . . . . . . , . . . . . . . . . . . . . .Approximate Variance Estimators.. . . . . . . . . . . . . . . . . . . . . . .Influence of Poststratification . . . . . . . . . . . . . . . . . . . . . . . . . .

112

3

55

10

12

14141516

17171818

2020202426

27272929292930

. .Ill

CONTENTS–Con.

Page

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Relevancy to HIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Appendix I. Extent of Biasness of Two Ratio Testimators . . . . . . . . . . . . 34Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...34Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...34Comparison of Two Ratio Estimators . . . . . . . . . . . . . . . . . . . . . 34

Appendix 11.A Simpler Variance Fstimator . . . . . . . . . . . . . . . . . . . . 38Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...38Variance Estimators for Estimator 1 . . . . . . . . . . . . . .. . . . . . . . . 38Variance Estimators for Estimator 2 . . . . . . . . . . . . . . . . . . . . . . 40Approximate Variance Estimators . . . . . . . . . . . . . . . . . . . . . . . 41

SYMBOLS

Data not available----------------------------------- ---

Category not applicable .............................. . , .

Quantity zero ----------------------------------------- -

Quantity more than Obut less than 0.05---- 0.0

Figure does not meet standards ofreliabilityy or precision--------------— ------- *

iv

DISTRIBUTION AND PROPERTIESOF VARIANCE ESTIMATORS

FOR COMPLEX MULTISTAGE PROBABILITY SAMPLES

Judy A. Bean, Ph.D., University of Iowa

INTRODUCTION tion. Because of these features, the assumptions

Background

Since the early 1940’s, there has been a sub-stantial growth in the use of surveys of humanpopulations, which has occurred in alI researchareas, including the biological and health sci-ences, and particularly the social sciences. Alongwith the increasing utilization of surveys, thepurposes for which data are collected have ex-panded. Research workers first employed sur-veys to obtain specific information about groupsfor descriptive purposes only. Increasingly, how-ever, they have become more interested in mak-ing comparisons among subgroups of the popula-tion, in testing hypotheses about the population,or in disclosing complex relationships. Both thegreater use and the change in objectives havepromoted considerable theoretical and practicaldevelopment in sample design which, in turn,has resulted in decreased use of the traditionalmethod of random selection from human popu-lations.

Most scientific sample surveys today are basedon complex multistage probability samples, withdesign components often including unequalprobabilities of selection for different elementsin the population, stratification, and two ormore stages of clustering. The estimation proce-dure may involve adjustment for nonresponse,use of concomitant variables, and poststratifica-

.of independence and equal probability of selec-tion are not valid. Therefore, analytical ques-tions arise. Theory has not been developed suffi-ciently to overcome all the difficulties evokedby the correlation induced by cIusters of sampleunits.

One analytical problem not solved in theoryconcerns estimation of variances and standarderrors of parameter estimates. Since these areintegral components of the formulas for con-structing confidence intervals and testinghypotheses, standard errors are crucial in statisti-cal inference. When variance estimates areneeded for statistical inference, several choicesare available.

‘There is the option of designing and carryingout a random sample but, as mentioned pre-viously, this has become an uncommon design.The human populations to be studied today arelarge and widely dispersed, which makes listingand travel prohibitively expensive. However, it isfrequently possible to use a reasonable approxi-mation for estimating variances when the exactformula is not known. Care must be exercisedwhen selecting an adequate approximation. Forinstance, if the random sample formula PQJn issubstituted for the appropriate vfiance esti-mator for clustered samples, it can be shownthat the variance is usuaIIy underestimated.

The investigator can perhaps translate theproblem for which no estimates of standard

errors are obtainable into one for which esti-mates are available. For example, the 2 X 2 chi-square test can sometimes be translated into thedifference between two proportions. Inferencescan be made on statistics for which the true vari-ances cannot be calculated, by applying vari-ances of similar statistics. For clarification, con-sider the situation in which an analyst is inter-ested in testing whether or not k (k > 2) meansare from the same population. Since the variancefor the difference of two means is known, thevariance for several pairs or perhaps all pairs canbe computed. Then the ratio of these variancesto variances that would have been obtained ifthe design had been a simple random sample canbe calculated. The analyst then infers that thecomparison of the correct variance estimate ofthe k means to simple random sample variancewould yield a similar ratio. Thus, he can use thisratio as an adjustment factor to the usual F ratiofor testing such ahypothesis.l’2

Another approach to the problem of estimat-ing variances is to interweave within a sampledesign a small number of replications. Each ofthese replications produces an estimate of theparameter. Therefore, their comparison providesan estimate of variance for the sample. An exam-ple of this is to draw 10 independent sub samplesfrom the same population using the same proba-bility design. Let Xi denote the estimated meanof the ith subsample and

Then from the 10 independent estimates, thesimple variance of the mean 37for the entire sam-ple can be computed. This estimate is

This method is too expensive to employ for acomplex design.3

A similar procedure to using independent sub-samples is the random group method. To illus-trate the technique, suppose one has a simplerandom sample of n observations drawn with re-

placement. The observations are randomly dis-tributed among t groups consisting of n/tobservations each. The variance estimate is

where

~j = the mean for the jth random group, and

j= 1

This method is suitable for multistage surveysamples. If primary sampling units (PSU’S) aredrawn and then second-stage units are sampledwithin, all the second-stage units from a primaryunit constitute a single unit when forming therandom groups. If there are few PSU’S, this pro-cedure is not too useful. Moreover, there is a lossof information.4

The final choice is to compute the estimatesof standard errors by one of the three generalmethods: Taylor series expansion or lineariza-tion method, pseudoreplication or replicationmethod (originated from the methods of inde-pendent replication and random group), or jack-knife method. Unfortunately, the behavior of’the variance estimates produced by these meth-ods is not thoroughly understood.

Three major survey organizations–the ,.U.S.Bureau of the Census; the National Center forHealth Statistics (NCHS); and the Survey Re-search Center (SRC), University of Michigan—commonly use one of the three general methods,linearization, replication, or jackknifing, to cal-culate estimates of variances from the sampledata. It is, therefore, important to becomefamiliar with the properties of these methods.

Strategy of the Study

There are several possible ways to study thecharacteristics of variance estimates created bythe general methods. One possibility is the devel-opment of applicable theory by assuming a cer-tain distributional form and obtaining exactanalytical solutions. However, the mathematicsinvolved have so far proven to be intractable.

There is another drawback with this approachwith a stratified multistage design: two sourcesof sampling error are between the first-stageunits and within the first-stage units. When onlyone unit is chosen from a stratum, there is noconsistent way of estimating the variance be-tween the first-stage units from the sample itself.In practice, the designs seldom fulfill this simpIeassumption of the selection of two units. An-other avenue of study is Monte Carlo samplingfrom synthetic populations. Such populationare difficuIt to construct and are of questionablerepresentativeness. A third device is an empiricalinvestigation. By this approach, data that havebken collected in a survey become the universe,and repeated samples are drawn from it. Foreach sample, the statistics being studied are com-puted, and sampling distributions of the esti-mates are generated. Perhaps the most famoussuch study is the investigation by “Student” in1908,s which resulted in his derivation of the tdistribution.

In this report, the view is that by Monte Carlosampling of data collected in a nationwide sam-ple, variance estimators can be studied to gaininsight into their properties and behavior. Anempirical investigation of the behavior of thereplication and linearization variance estimationmethods from a viewpoint that is both broadand practical is undertaken. The study uses fivevariables collected on 131,575 people in theU.S. Health Interview Survey (HIS). The mainobjectives of the research were the following:

1. To investigate two general variance esti-mator methods,= linearization and replica-tion

2. To study the distribution of the ratio of anestimated mean minus its expected valuedivided by its standard error

3. To measure the impact of poststratificationon variance estimates

Other aspects examined in the investigation werethe extent of the biasness of two estimators andthe feasibility of using a simpler variance esti-mator as an approximation to the correct repli-

aSice the replication and linearization schemes are actuallybeing used by several samplers, the author chose to study onlythese two techniques in this study.

cation and linearization variance estimates (theresults are given in appendixes I and II).

When a study of this magnitude is describedand discussed, there is the danger that the readerwill lose sight of the overall objectives amidst allthe details of the anaIysis. To facilitate report-ing, the main features of the methodology in-volved in doing the research are listed, as fol-lows:

1. Sample des@z. By this term is meant the

2.

3.

It

com-ponents- of selection of the sampleunits. The design used includes stratifica-tion, sampling of the first-stage units withprobability proportional to size, the largestunits entering the sample with probability y1, and. finally, subsampling clusters ofhouseholds within the first-stage units,thereby embedding two stages of clusteringin the design.Estimation procedure. This refers to themethod for producing estimates of thehealth characteristics. Two different post-stratified estimators of means are used.Varz”ance calculations. This feature is thecomputation from the sample data of esti-mates of the variance of the estimators pro-duced in feature number 2. For each esti-mate of a characteristic, the variance isestimated in a number of different ways.

is appropriate to state the reasons why onlytwo out of the three general variance estimatorsare studied here. The intent of the research is toobserve the variance estimators produced by themethods in more depth than has been done pre-viously and to study the behavior of the esti-mators for types of designs and estimation pro-cedures being used today. Data handling, evenfor a small-scale study of a similar nature, isquite massive, and this problem is compoundedby the type of design and procedures selected.Therefore, the decision was made that only twoof the three variance estimator techniques wouldbe investigated.

HIGHLIGHTS OF THEFINDINGS

This study was undertaken to investigate howwell statistical methods important for making

inferences are valid when the data have been col-lected in a multistage probability sample surveyand subjected to a complex estimation proce-dure. The main points examined were the fol-lowing:

1. The behavior of the variance estimates pro-duced by the balanced half-sample replica-tion method and the linearization method

2. The distribution of the ratio “of a sampleestimate minus its expected value to itsestimated standard error

3. The impact of the estimation techniquepoststratification on variance estimatey

Since the mathematics have not been devel-oped to answer this question of validity, theapproach was to use Monte Carlo simulationfrom a specified universe and to determine theempirical results.

The sample design was a stratified two-stagecluster design with first-stage units beingselected with probability proportional to size.The estimation process involves weighting by thereciprocal of probability of selection and post-stratification. Five variables were used, and three

different samples sizes were studied with 900samples of each size drawn.

Because design III is the largest sample sizeand, thus, the one most comparable to real sur-veys, the highlights for only this design are pre-sented. Characteristics of the universe and atypical sample are:

Universe Average sample

Number of PSU’S 149 30Number of segments 7,768 600Number of persons 131,175 8,772 {approximate)

Table 1 gives the major results for the varianceof a ratio estimate produced by the replicationmethod, VAR(R 1S) and the linearization meth-od, VAR(LI).

The proportion of times the standardized vari-able fell within certain regions was calculated foreach of the five variables. These proportionswere averaged across the five variables and com-p?xed with the expected proportions of a t dis-tribution with 19 degrees of freedom, and thenormal distribution, in table 2.

Table 1. A summary of the major findings for design III

Finding

Population parameter /? . . . . . . . . . .

Average sample estimate /?l . . . . . . . .

sample variance of, R~ - .31R . . . . . . .

Average value of the varianc~ estimate

VAR(RIS) . . . . . . . . . . . . . . . .

Biasof VAR(R1.S) . . . . . . . . . . . . .

Variance of VAR(RIS) . . . . . . . . . . .

Relative bias of VAR(RIS) . . . . . . . .

Relative variance of VAR(RIS) . . . . . .

Averaga valua of the variance estimate

VAN .. . . . . . . . . . . . . . . .

Bias of VAR(Ll) . . . . . . . . . . . . . .

Variance of VAR(Ll) . . . . . . . . . . .

Relative biasof VAR(Ll) . . . . . , . . .

Ralative variance of VAR(L.1) . . . . . ,

Relative bias of VAR (R IS)Relative bias of VAR(LI) “ “ “ “ “ “ “ -

Relative variance of VAR (R IS)

Relative variance of VAR (L 1 ) “ “ “ “ “ “

Family income

6400.08392.8

27,447.7

26,554.8-890.9

10,360.0 X 104-0.0325

0.1375

26,174.7-1,271.0

10,480 X 104-0.0463

0.1391

0.7019

0.9885

Restricted

activity

days

14.671614.6595

0.9496

0.9206-0.0290

1.1370-0.0305

0.1519

0.8915-0.0581

0.1310-0.0612

0.1453

0.4984

1.0454

Variable

T

4.67584.65480.0168

0.01780.0010

0.8132 X 10 ‘40.05950.2882

0.0175

0.00070.7959 x 1o+

0.04170.2820

1.4269

1.0220

1.05571.0597

7.0474 x 10-3

7.0842 X 10-30.0368 X 10 “0.0129 X 10-=

0.00520.2596

6.7380 X 10 ‘3-0.3084 x 10-3

0.0113X 10-3-0.0439

0.2275

-0.1185

1.1411

Proportion seeingphysician

0.68420.6840

7.8942 X 10-’

8.3010 X 10-’0.4068 X 10-s0.1089 X 10-*

0.05150.1747

8.1135 X 10-50.2193 X 10-S0.1086 x 10-8

0.02780,1743

1.8525

1.0023

4

RI -“E(R, ) RI -’E(R1)

‘able 2“ ‘distributions ‘f VAR(RIS) and VAR(L1 )compared

with expected values under t, ~f and normal’

Standardizeddeviate Normal

‘l~f-

~‘The hypothesized degrees df frebdoin are the number of

strata 19.

RwiEw oF LITERATURE

Replication Method

Although, in the past three decades, methodsfor selection of sampling units and proceduresfor producing estimates from sdrvey data havebecome sophisticated, only in rekifively recentyears have the methods of tialyzing ddta forsuch complex surveys begun to be considered.The first articles in the literature on the topicdescribed and discussed a variety of techniques:random grotips, replicated samples, interpetie-,.tratifig samples, &d duplicate samples, whichare the forerunners of the ge,neral replicationmethod as applied in this study. Mce this in-vestigation is concerned with the technique in itspresent-day form, the previous methods will notbe reviewed here, but their essential features aredescribed elsewhere.6

Before giving a brief, account of the develop-ment and evidence of the reliability of the repli-cation estimator of variance, its main featuresare outline& The basic premise of the method isa very simple one. Th6 estimator X’ of a popula-tion parameter is calculated from the entirebody of sample data (parent sample). Anotherestimator Y’ of the same population parameteris made, using only the data from half the sam-ple, randomly selected. The quantity (Y’ - X)2is an estimate of the vaiiance based on one-halfsample (replicate). Howevet, this estimate, de-pending on only one differertce t&m, has a highvariance; repeated differences are required toproduce a stable variance estimate. The esti-mator of variance is simply the mean of thesehalf-sample estimates.

k

VAR(X’) = + ~ (~’ - X’)2.

Cl=l

where k is ‘the number of replicates used. Thereare three other rather similar forms of varianceestimators.

k

1. +~(Y: - X’)2a=l

tvhere Y’fa * the estimator of the parametermade from the complementary set of data forthe oih replication (the half of data not used inthe Y’a estimator).

The U.S. Bureau of the Census7S8 was thefirst major smvey organization to employ themethod. The technique was used to estimatesamplitig errors of ratio estimates produced fromthe Current Population Survey (CPS) from 1954through 1964.b Gurneyg worked on the theoret-ical development of the method for the U.S.Bureau of the Census.

The National Center for Health Statisticsdraws inferences from data gathered in complex@obdbility sample surveys. The Center uses thereplication method to provide variance estimatesfor the estimates produced from the HealthExaminatiori Survey (HES), which consists of adirect physical examination of a probabilitysample of approximately 7,000 noninstitutionalU.S. civiIians.l 0 Its purposes are to make preva-lence estimates for certain medical and dentalconditions and to determine the distributions ofmany physiological characteristics. The survey isconducted in cycles with different age groupssampled for each cycle.

bCPS is a monthly nationwide survey of sample householdsconducted to provide measurements of the labor force. Nationalestimates of cmploymen~ unemploymen~ and other labor forcecharzctcristics arc made.

5

The sample design of the survey includes thecomponents stratification, clustering, and twostages of sampling. Along with this highlysophisticated sample design, an equally refinedestimation process is used. Features of the pro-cedure are simple inflation of the basic data col-lected on each sample unit by the reciprocal ofthe probability of selecting that unit, adjustmentfor nonresponse, poststratification, and possiblyanother adjustment factor.

The National Center for Health Statistics, inits efforts to provide the best estimate of vari-ance for this ratio estimate, has used several ver-sions of the replication technique and, from thisexperience, has developed a routine computerprogram to provide the analyst with prevalenceestimates and their variances. The analyst, bysimply requesting the estimates for the total orspecified subgroups of the population, receivesan estimate of the aggregate value of the healthcharacteristic (numerator), an estimate of thepopulation of the total or subgroup (denomi-nator), and the ratio of the two values. In addi-tion, the analyst obtains an estimate of thestandard errors for these three quantities. Amore detailed description of the developmentand use of the replication method can be foundelsewhere.6 ~11 Y1z Based on research conductedby NCHS and the performance of the replicationvariance estimates, the technique appears notonly appropriate for the type of estimate pro-duced in its surveys but also useful for varianceestimates of other statistics such as regressioncoefficients, multiple correlation coefficients,and partial correlation coefficients; moreover,the technique has potential for being employedin other forms of statistical analysis. For in-stance, replication variance estimates are used ina modified sign test and in the calculation of apseudo-chi-square statistic, which provides a testof independence in a two-way table.

Valuable theoretical work has been done onthe replication method.6 If a subset of all thepossible half-samples that could be formed isselected randomly, the number necessary to pro-duce a stable variance estimate is large. By theuse of a stratified design with two independentselections made from each stratum and an esti-mate of the mean, it was demonstrated that thevariability among the half-sample estimates ofvariances comes from the between-strata contri-

butions to these estimates.~ It was then shownhow to eliminate these cross-product terms bychoosing a relatively small subset of half-samplesin a particular fashion that would yield an un-biased estimator of the true variance of thelinear estimator. Thk variance estimator, in fact,is equal to the value that would be obtained ifall possible half-samples were formed to calcu-late the variance estimate. The haJf-sampleschosen in this manner are said to be orthogon-ally balanced. This set of half-samples is referredto as an orthogonal set or a balanced set.

The steps involved in composing a balancedhalf-sample are explained. Assume a sample de-sign of two independently selected units from astratum. First, the digit 1 or 2 is arbitrarily as-signed to each sampled unit from a stratum.Then the number of replicates needed is speci-fied. This number, k, has the restriction that itmust be a multiple of 4, in order to obtain anorthogonal set. The value is equal to the numberof strata, L, plus either O, 1, 2, or 3, dependingon which integer added to L will yield a numberdivisible by 4. The next step is to form a k X Lmatrix with entries aij of either plus or minus 1such that the columns (strata) of the matrix areorthogonal to one another. This means

~ aajaaj) = O where j # j’

Plackett and Burmanl 3 describe the construc-tion of such a matrix. Finally, the cith half-sample is made by including the unit numbered1 if CZij = +1 and the unit numbered 2 if aij = -1.The names “balanced pseudorep~ication” and“balanced repeated replication” are given to themethod in which the minimum number of half-samples are needed. (Throughout this text, theterm “replication method” means the full ortho-gonal balanced version unless otherwise speci-fied.)

Also considered in the report6 was the situa-tion in which the number of strata was so largethat the number of half-samples necessary forfuIl orthogonal balancing would be too numer-ous to be feasible because of computer costs. Inthis case, a method is suggested whereby the

half-samples are partially balanced. To achievepartial balance, the L strata are divided into L/tgroups, where t is an arbitrary integer thatdivides L evenly. The steps ‘~ven for full orthog-onal balancing are carried out for one group ofstrata, which results in the elimination of thebetween-strata contributions for these strata.The matrix that is formed is applied to each ofthe remaining groups. Thus, within orthogonalsets there is no between-strata contribution, andthe cross-product ‘terms partiaIly disappear.However, the other cross-products do not dropout. In the case of a stratified random sampIeand an estimated mean, this subset of half-samples has a lower estimated relative variancethan a set of k randomly selected half-samples.

Leel 4 has recently published theoretical andempirical work on a method of selecting a par-tially balanced set of half-samples. He also con-sidered the problem of the best possible value oft. There will be a larger variance when the repli-cates are only partially bahuiced than when theyare fully balanced. However, this loss can be re-duced when the semiascending order arrange-ment (SAOA) of selecting half-samples is em-ployed. To accomplish an SAOA, the strata arefirst arranged in ascending order of the magni-tudes of their within-stratum weighted popula-tion variance. After this is done, the last L/2 or(L - 1)/2 (if L is odd) strata are reversed. Thismeans that these strata are put in descendingorder of magnitude of their within-stratum vari-ance, and the first L/t strata form the first groupand so on.

Another factor in reduction is to choose asmall t value, but the smaller the value oft, themore half-samples are needed. Therefore, whenthe investigator decides on the value of t, hemust weigh the loss in precision against the in-crease in computer costs due to a larger numberof half-samples.

The behavior of the replication method whena ratio estimator is used was also examined.15The findings were primarily based on empiricaldata from the first cycle of the HES, a sample ofapproximately 6,600 adults. Body measure-ments were analyzed from a subsample ofapproximately 3,000 U.S. adult males.16 Six-teen regression equations with age, weight, andheight as the independent variables and each ofthe other 16 anthropometric measurements in

turn as the dependent vafiable were tabulated.cA variance estimate for a regression coefficientcan be produced by the replication method.Using the data in a replicate, a regression equa-tion identical to the one computed for the par-ent sample can be calculated. Then an estimateof variance is obtained from the average of thedeviations of the half-sample regression coeffi-cient estimates from the parent sample estimate.Using a balanced set of 28 replicates, estimatesof variances were computed for two types ofestimates: ratio estimates of population meansfor the physicaI body measurements, and multi-ple regression and correlation coefficients.According to the study, the set of balanced half-samples gave the same results as would havebeen obtained by employing the full set of repli-cates. Also developed was an expression for therelationship between the replication variance ofthe estimator of the population means and thereplication variance of the average of the k half-sample estimated population means. Because theresults from a sample provide values for thequantities in the expression, an inference as towhether adequate variance estimates are beingobtained can be made. It was concluded thatthese data showed that the replication varianceestimates were sufficient.

Simmons and Baird,l Z following a suggestionby Kish at the Survey Research Center, Univer-sity of Michigan, investigated whether anapproximating variance formula requiring lesscomputer time could be used in place of theexact equation. In the replication method, thedata for a haIf-sample are subjected to the full

estimation procedure in exactly the same man-ner as the data from the parent sample. Thisoperation can use extensive computer time, es-pecially when steps such as ratio adjustment bya concomitant variable and poststratificationmust be carried out in each half-sample. How-ever, by ,weighting the individual sample observa-tions ,more simply than the usual way designatedby the sample design, the correct replicationvariances can be approximated. The gozd of thestudy was to weight the data in three ways toobserve whether the approximations could besubstituted for the correct variance estimate.

cThese calculations were provided by the Survey ResearchCenter, UNversity of Michigan, in accordance with NCHS speci-fications.

The data for the research were the same physicalmeasurements as those used elsewhere.1s

Operationally, when making the parent sam-ple ratio estimates, a ratio adjustment factor dueto the use of a concomitant variable is computedfor each of six residence-region classes. This wasdone only on the data coUected in the first cycleof the HES. A poststratification factor for eachof 60 age-sex-color classes is also calculated.Both adjustments are in the form of multipliersapplied to the weight (reciprocal of the selectionprobability) for each sampled person. In the rep-lication method, the replicate estimate of thepopulation parameter undergoes the ordinaryestimating process. Hence, the multiplicationfactors of these adjustments should be recalcu-lated in each half-sample to bring the distribu-tion of the sample into as close agreement aspossible with the distribution of the universe. Byusing the two multipliers from the parent sampleinstead of recalculating them in each half-sarnple, Simmons and Baird ,hoped that an ade-quate approximation involving less computertime could be made. The weighting schemeswere to assign each observation (1) a weight of 1as if the data were collected in a simple randomsample, (2) the weight it received when the par-ent sample estimate was constructed, and (3) theappropriate weight for the specified half-samplecontaining the observation. A brief summary ofthe conclusions is offered. The estimates of vari-ance produced from the unweighted data(scheme 1) were not acceptable as approxima-tions to the true variance. The ratios of the vari-ance estimates using weighting scheme 2 to thevariance estimates using weighting scheme 1were all greater than unity, indicating that thesimple random sample variance estimates areunderestimates. The variability of the ratios forthe different statistics was wide. Therefore, theapplication of a constant adjustment factor tosimple random sample variances is not feasible.Simmons and Baird then compared the varianceestimates resulting from weighting schemes 2and 3. The analyses consisted of computing theratio of replicate variances using the parent sam-ple adjustment factors to variances using uniqueadjustment factors. These ratios fluctuatedaround unity. The ratios for the ratio means ofthe physical body measurements and for thesimple correlation coefficients were slightly

greater than unity, but the ratios for the partialand multiple correlation coefficients were justbelow unity. The variability of these ratios wasnot negligible. Because of this, the investigatorsconcluded that analysis on similar variances formore statistics was required before making a de-cision about the use of the simple weighting pro-cedure (scheme 2) in place of scheme 3,,

The SRC has produced variance estimates bythe replication technique since 1957. Kish andFrankell T~1g presented the empiricaI evidenceon the reliability of the method from SRC’Sprojects and from their analysis of the NCHSbody measurement data for the HES. In thesestudies, the replication method was employed toprovide standard error estimates for regressioncoefficients, correlation coefficients, and partialcorrelation coefficients. The analyses examinedthe square roots of the ratios of the true stand-ard errors calculated by balanced repeated repli-cations to variances assuming a simple randomsarnple. These ratios are referred to as the designeffect (DEFF).

The first empirical evidence presented was theresults from 20 linear regression equations calcu-lated on data for a sub~oup of the populationsampled in three different surveys. The subgroupwas identified as husbands and wives aged 35-44years living together, with the head of the housein the labor force and having a family income of$3,000 or more. There were 1,853 such definedfamilies. Seven predictors used in various combi-nations one, two, three, or four at a time ,werethe basis of the regression equations, An esti-mate of variance for each of 60 regression coeffi-cients was made. The mean value of th,e squareroots of the 60 DEFF’s was slightly higher thanunity, indicating that simple random samplevariance estimates underestimate the true vari-ance.

A total of 1,111 people were interviewedtwice to gather data on the political party theysupported and their political attitudes. Then,using the political part y voted for as the depend-ent variable and four different attitude scales aspredictors, a regression equation was con-structed. The mean value of the square roots ofthese four DEFF’s was higher than unity, butlower than the value in the first study. In addi-tion, the mean of the square roots of theDEFF’s for the means, the mean value for corre-

8

Iation coefficients, and the mean value for par-tial correlation coefficients all fluctuated nearunity.

Data collected in 3,990 interviews conductedin three national household samples were em-ployed to form six dummy variable regressionequations. The predictor variables in this studywere education, type of occupation, family in-come, family reserves, race, and relationshipwith relatives and friends. For the square rootsof 64 DEFF’s, the mean value was again aboveunit y.

Kish and Frankel,l 7 under a contract withNCHS, estimated in the 16 linear regressiondquations based on the physiological measure-ments of 3,091 males in the HES the ratio of thereplication variance to simple random samplevariance. The mean value here for the regressioncoefficients was above 1 and higher than the pre-vious empiricaI values. The investigators sug-gested that this higher mean value resulted fromthe large clusters used in the HES.

The last set of empiriczd results examined thevariance estimates for statistics computed by thetechnique of multiple classification analyses.This technique is a multivariate analysis thatexamines the relationship of each predictor to adependent variable that must be an interval scaleor a dichotomy, with the other predictors heldconstant. The model is additive; and no assump-tions of equal cell frequencies, linearity of re-gression, and orthogonality are required. Foreach predictor, there are two estimated statis-tics, eta and beta coefficients. The squared etacoefficient is the proportion of variance of thedependent variable explained by unadjusted de-viations (cell means minus overall mean), andthe squared beta is the proportion of variance ofthe dependent variable explained by a given pre-dictor, holding the other predictors constant.Data for a sample of 2,214 family heads wereused in the multiple classification analyses equa-tion to fit a receptivity index to education ofhead of household, age of head, total family in-come, social participation, achievement orienta-tion, sex, and marital status. Because of the costinvolved, the replication variance estimates werecomputed from 12 partially balanced replicatesand the simple random sample variance esti-mates from 12 repetitions based on simple ran-dom splits of the data. For the six eta coeffi-

cients, the mean DEFF’swere the mean DEFF’sbetas.

were greater than 1, asfor the corresponding

By means of a Monte Carlo approach,’ Levyl gcompared the performance of the balan”cedreplication and jackknife methods for the ratioestimator when the sampIe design was stratified,with two PSU’S selected from each stratum.Within each of 16 strata, a synthetic normalpopulation with specified mean and variance wasgenerated for two variables Xh amd Y~. Theparameter to be estimated from the samples was

xx,R=~

p, ‘a ratio estimator. The sampling and estimationprocedure was to draw from each stratum twoindependent estimates of Xh and two independ-ent estimates of Y~. These estimates welefashioned into a combined ratio estimate, R.From a balanced set of 1Q half-samples, an esti-mate of the variance of R was computed. An-other estimate of this variance was calculated bythe jackknife method.

In the experiment, the parameters Rh wereallowed to vary in succession according to theratios 1.01, 1.05, and 1.10 for one value of thecorrelation between X and Y and one value ofthe relative variance of the population mean X.The correlation value used equzded 0.9, and therelative variance was set equal to 0.01.

For analytical purposes, the estimated vari-ances, biases, and mean square errors for boththe replication and the jackknife method wereexpressed relative to thg sampling variance ofthe 1,200 estimates of R. The relative varianceequaled the sampling variance of the varianceestimates for a method divided by the square ofthe sampling variance of R. The relative biassquared equaled the square of the mean of thevariance estimates -for a method minus the sam-pling variance of R djvided by the square of thesampling variance of R. The relative mean squareequaled the sum of the relative variance plus therelative bias squared.

The relative mean squaremethods increased slightly as

errors for boththe varying ratio

9

increased from 1.01 to 1.10. The jackknife rela-tive mean square error estimates were lower, thanthe estimates produced by replication for allthree cases. The jackknife estimates of relativevariance were also lower than replication esti-mates for all the varying ratios. When the R ‘h’svaried by the ratios 1.05 and 1.10, the replica-tion method had the lower relative bias. Theseresults indicate that further such studies usingdifferent parameter values are needed.

Several of the design and estimation compo-nents included in complex sample surveys areunequal sampling fractions, clustering, stratifica-tion, first-stage ratio adjustment, nonresponseadjustment, and poststratification. In an e~piri-cal investigation, Simmons and ‘Bean20 meas-ured the effect these components, alone and incombination, had on replication variance. Datafor the four health conditions–hypertensiveheart disease, myocardial infarction, angina pec-toris, and syphilis–from the HES were used.Using six estimation procedures, six estimates ofthe proportion of the total population or pro-portion of a subclass of the population having aspecified health condition were calculated. Toimplement these procedures, each examinedperson received a set of six weights. The weightsassigned were ( 1) a weight of 1, as if the datawere collected in a simple random sample; (2) aweight of 10,000; (3) the appropriate weight forthe HES sample design; (4) the weight in 3,multiplied by a nonresponse adjustment; (5) theweight in 4, multiplied by a first-stage ratioadjustment; and (6) the weight in 5, multipliedby a poststratification factor. A simple random-sampling variance estimate was calculated forthe estimate produced when the weight of 1 wasused; replication variances were tabulated forthe other five estimates.

Ratios of variances and ratios of mean squareerrors formed the basis of comparison. For theratios of variances, the numerator was thevariance of one of the estimates, and thedenominator was the variance of another esti-mate. For the ratios of variances, the numeratorwas the variance estimate produced by onemethod, and the denominator was the varianceestimate produced by one of the other methods.To take bias into account, the ratio of the meansquare error of one estimate to the mean squareerror of another was examined. The estimate

produced by the full design and estimationprocedure (the last weighting scheme) was sub-stituted for the true value. Thus, an estimate ofbias equaled the difference between this esti-mate and the estimate obtained for the methodin comparison.

The results of the comparison based on ratioof variances and the general conclusions arepresented. The use of unequal sampling fractionsalone showed a 27-percent increase in replica-tion variance over simple random sample vari-ance. The combined effect of unequal weighting,clustering, and stratification produced a 137-per-cent increase in replication variance over asimple random sample model. However, thevariance estimate for the estimate produced bythe full design and estimation procedure wasonly about double the simple random varianceestimate. Thus, the estimation steps-non-response, first-stage ratio adjustment, and post-stratification—improved the precision of theintimates. Simmons and Bean concluded thatnone of the design and estimation componentsinvestigated introduced large biases in the esti-mates and that these components have substan-tial impacts on variance estimates.

The strong point of the replication varianceestimator method is that measurement errors,design components, and estimation proceduresare all accounted for. The technique can beemployed to yield variance estimates for anystatistic and, thus, is flexible. It can also beadapted to a design that selects more than twoPSU’S from each stratum.

Linearization Method

The other general method for estimatingvariances was developed by Keyfitz,z 1 whooutlined a method for estimating the variance ofa poststrati~led estimator. This estimator isderived from the fact that the variance of a sumof two independent estimates of a parameterequals the expected value of the square of thedifference between them. This is

VAR(X~ +X;)= E(XI - Xi )2

where X’l and X’2 are estimates of the pmam-eter X made from two independent random

10

samples drawn with replacement. The keytheorems in developing the method will besketched without giving any proofs. This expres-sion can be extended to include estimates drawnfrom another stratum, assuming the selectionsamong the strata are independent.

The next term needed is the covariancebetween characteristics measured on the samesampled units within a stratum. Assume X’l andY’l are estimates from one random sample andX’2 and Y’2 are estimates from another randomsample. The two samples are drawn independ-ently with replacement and, thus, COV(X’l, Y’2 )= COV(X’2, Y’~ ) = O, E(X’l ) = E(X’2), andE( Y’l ) = E( Y’2 ). Then the covariance can beshown to equal

The key in the derivation of the linearizationmethod is the approximation of the relativevariance of a ratio estimator using the Taylorseries expansion. Defining the relative varianceof an estimator as the variance of the estimatordivided by the square of the expected value ofthe estimator, then relative variance (F) is

= W(x’)-1-P (Y’)- 2V(X’,Y’)

where

x’~ = the ratio estimate of the population

parameter +,

V(x’,Y’)=p(x’,Y’)Iqx’)V(Y’),

p(X’,Y’)= the correlation between X’

L’(Y’) = the coefficient of variation

(X’) = the square root of V2 (X’).

and Y’,

of Y’, and

A proof of this result is in Hansen, Hurwitz, andMadow.22

Assuming again independent random samplesdrawn with replacement and using the abovetheorems, substitution, and algebraical manipu-lations, it maybe shown that

If selections are independent among strata, theseresults can be generalized to any number ofstrata.

The last theorem gives the variance of apoststratified estimator. The sample design istwo random selections drawn independently andwith replacement from each stratum. The post-stratified estimator is

where

x“

x’hai

= the poststratified estimate of X charac-teristic,

= the ith estimate for the ath class of thehth stratum,

= the ith estimate of population for theath class of the hth stratum,

= precalculated total population in theath group.

The difficulty in determining the variance of X“is the lack of independence among the classes.The variables defining the classes are not used tostratify the population. Therefore, the covari-ance between the classes is not zero. Thismethod evaluates the covariance between thegroups separately in each stratum. These values

11

are then summed across the strata. The varianceof X“ is

VAR(X”) = E?

x

In this discussion, random sampling withreplacement has been assumed. If sampling iswithout replacement, the finite population cor-rection factor can be inserted into the formulas.From these equations, formulations applicableto complex sample designs have been developed.Tepping2s developed a more general procedurefor linear variance estimators using a Taylorseries expansion and showed how the methodcan be employed to provide variance estimatesfor more complicated statistics such as regres-sion coefficients. Bean presented the theory,proofs, and description of the form of thevariance estimator for the ratio estimates pro-duced by the HIS report.24 Shapiro25 andWa1tman26 outlined the adaptation of Key fitz’theorems to the ratio estimation proceduresemployed in the CPS. In essence, the lineariza-tion method fashions a variance estimator froma linear combination of sample totals for eachPsu.

Statisticians of the Bureau of the Census wereamong the first to apply the linearizationmethod. They investigated its use in the CPS’ssample design and estimation procedures andconstructed a computer program tabulating thevalues needed for each PSU. Banks and Shapiro8presented and discussed the empirical basis theBureau had acquired over the years from itsvariance estimates for ratio estimators. They alsoassembled the data on the replication varianceestimates. The methodology used in the analysiswas the comparison of design effects for un-biased estimates, ratio estimates, and compositeestimates for a variety of labor force variables.The values (true variance as computed bylinearization technique) for the unbiased esti-

mates and ratio estimates indicated that, with-out exception, the ratio estimate lowered thevariance. In fact, for the variables with largeparameter values, the decrease was substantial.When the comparison was made between theratio estimates and the composite estimates, thelatter reduced variance of some of the laborcharacteristics, but not all of them.

Data from 1964 on 13 variables, the relativevariances as estimated by linearization and repli-cation (not balanced) methods, were displayedin Banks and Shapiro. Both estimates wereconsistently of similar size, but there was morevariability among the monthly replication vari-ance estimates. The authors concluded that thelinearization estimates were more reliable.

Linearization is the device by which varianceestimates of the ratio-estimated characteristicsof the Canadian Labor Force Survey are calcu-Iated. Fellegi and Gray27 explained the applica-tion of this method to the survey with itsparticular sample design and estimation proce-dure and discussed how the analysts actuallymake use of the design effect in their analysis.The comparative findings of the design effectswere consistent with the data from the U.S.Bureau of the Census study.

The literature on Linearization is small com-pared to that on replication. There are probablytwo reasons for this. Until Tepping’s article,z 3this method was not available for the estimationof variances for statistics other than means, andthe method did not generate the interest of thereplication scheme. Furthermore, sam~plers mayhave used the technique without reporting itsgeneral behavior. One feature of linearization isthat the variance can be estimated for each stage‘of sampling, a fact that is important for design-ing surveys.

An Empirical Investigation of All ThreeGeneral Variance Estimators

In the articles reviewed, there are few com-parisons of the properties and behavior of theestimates produced by the two meth~ods whenthey are applied to the same data. The onlyextensive research along these lines has been thework of Frankel.28 Using repeated samplingfrom a universe modeled from real samplesurvey data, the validity of three fundamental

12

assumptions, important for purposes of statisti-cal inference, was assessed. The assumptionswere the following:

1. The sample estimate of the populationparameter is approximately unbiased.

2. An approximately unbiased estimate of thevariance of the estimate is computablefrom the sample.

3. The distribution of the ratio of the sampleestimate minus its expected value, to itsestimated standard error is reasonablyapproximated by the “Student” t withinsymmetric limits.d

Frankel regrouped the CPS’S March 1967 dataon noninstitutional families and individuals intoPSU’S consisting of three segments of approxi-mately five to six households each. Thesesegments were the basic sampling units of CPS.The PSU’S were stratified into 6, 12, and 30strata, and two PSU’S were selected independ-ently from each stratum for each design. Fordesign I (6 strata) and design II (12 strata), 300samples were drawn; for design III (30 strata),200 samples were chosen. Thus, the sampledesign consisted of one stage of sampling,stratification, and single-stage clustering.

The estimation procedure followed in eachsample was the fitting of two linear regressionequations. The total income of household headwas related to the number of persons in thehousehold under 18, the number of persons inthe household, and the sex of the householdhead. For the second equation, the total incomeof the household was the independent variable,and the dependent variables were number ofpersons in household in labor force, age ofhousehold head, and years of school completedby household head. Estimates were made ofsimple means, differences of means, correlationcoefficients, regression coefficients, partial cor-relation coefficients, and multiple correlationcoefficients. For each of these estimates, thevariance was calculated in nine different ways.The nine methods included all three of thegeneral variance estimators: Taylor expansion

(linearization), balanced repeated replication,and jackknife repe~ted replication.e A descrip-tion of the specific jackknife formulation used

can be found in Frankel’sz 8 work.Frankel compared the behavior of the esti-

mates of means, difference of means, regressioncoefficients, and simple, partial, and multiplecorrelations on the basis of relative bias. Relativebias is defined to be the difference between theestimate of the parameter and the parameterdivided by the parameter. The relative biases ofthe mean and differences of means were all lessthan 1 percent, except for one difference ofmeans that had a reIative bias of 3 percent. Theregression coefficients had higher relative biasesthan did the means. In design I, seven out of theeight regression coefficients showed relativebiases of less than 4 percent; in design II, onlyfive out of the eight were less than 4 percent; indesign III, the relative biases for seven of theregression coefficients were less than 4 percent.The correlation coefficients (simple, multiple,and partial) displayed the highest relative biases.Only for design III did the mean of the relativebiases for simpIe correlations drop below 5percent.

In analysis of assumption 2, the nine varianceestimators were also considered to be meansquare error estimators, and their performancesin that capacity were studied. The relative biaswas the main measure of evaluation. For meansand differences of means, none of the nine hadminimum reIative bias across aI1 the designs; forsimple correlations, one of the replication meth-ods emerged with lowest relative bias. However,for the estimates of regression coefficients, ajackknife technique displayed minimum relativebias for two designs and linearization behavedbest for the other design. In all three designs, thereplication estimate, involving the square of thedifferences between half-sample estimate andcomplement estimate, showed the lowest rela-tive bias for partial correlation coefficients.

The results provided evidence for the validityof assumption 2, but the data did not definitely

‘l’here were nine methods becauseeach of the last twotechniquesprovidefour different possiile schemes of variancecalculations (the four produced by replication me outlined in theReview of Literature section).‘Frankel,28 p. 2.

13

resolve the issue of which variance schemeproduced estimates closest to the true variance.The conjecture that the estimates produced bythe three general methods are really estimates ofmean square errors rather than variances wasstudied using another relative bias expression.For designs I and II, variance estimates formeans and differences were relatively closer tothe mean square error parameter than thevariance parameter. More research is requiredbefore a definite answer can be given as towhether the estimates produced by the generalmethods are mean square error estimates ratherthan variance estimates.

Probably the most important conclusionreached concerned assumption 3. The ratios ofthe sample estimate minus its expected valuedivided by each of its nine estimates of standarderrors were computed. The evaluation of theratios was based on the proportion of times theratios fell within symmetric intervals around theorigin. The intervals chosen were normal values.One-sided intervals were also computed, andexamination of the proportions within theseintervals revealed instances of asymmetry.Frankel concluded that these findings werepartly real and partly the result of selecting onlyseveral hundred samples rather than severalthousand samples. Because of this, he decidednot to compare the proportions with values inthe t table. To reduce variability of the ratios,proportions were averaged for similar statisticswithin a design. These proportions were c,om-pared to the value in the t table with degrees offreedom equal to the number of strata. Thefindings indicated that the t distribution withinsymmetric intervals could be used to makeinferences. Another vital observation was thatratios based on the replication scheme weremore likely to be in agreement with the propor-tions of that distribution than the proportionsbased on the estimated standard errors.

The Frankel investigation has contributedsignificantly to knowledge concerning the prop-erties of the variance estimates produced by thethree general variance estimators. The validity ofthe assumptions investigated needs to be con-firmed for different sample designs, for differentvariables, and for different estimators.

METHOD

Description of Population

Morbidity data collected by the HIS in 1969on 131,575 civilian, noninstitutional U.S. indi-viduals constitute the universe for this investiga-tion. The HIS29 is a highly stratified, multistageannual probability sample that provides esti-mates of disease incidence and prevalence andhealth characteristics.

The same basic sampling design has been usedsince the survey began in 1957. The :first-stageunit, PSU, ii a single county or several con-tiguous counties. The entire geographical terri-tory of the United States is classified into 1,900such units. The next step is the stratification ofthe 1,900 units into 357 strata, many of whichconsist of only one PSU. Geographically, thePSU’S are divided into subunits (each containingan expected six households) called segments.Sampling for the survey is done in two stages:one PSU is selected with probability propor-tional to population size from each of the 357strata. The PSU’S comprising an entire stratumare included with probability 1 and are referredto as self-representing PSU’S (SR PSU’S). Theother PSU’S are called non-self-representingPSU’S (NSR PSU’S). Within each seIected PSU,segments are interviewed. An interview scheduleon every individual residing in the household iscompleted. Thus, the design includes the effectsof stratification and two stages of clustering,providing the model for this study.

The features of the sample design employedin this empirical investigation were stratifyingthe PSU’S, sampling of two stages, and clusteringat two levels. The steps involved in implement-ing the design were to sample the first-stageunits with probability proportional to popula-tion, the largest units entering the sample withprobability 1, and then to subsample within theselected first-stage units. However, before any ofthe details of the sample design—e.g., the deci-sion on the sampling fraction to be used—couldbe determined, the HIS data had to be re-grouped.

The 357 selected PSU’S, each representing astratum in HIS, were taken as the total popula-

14

tion. An examination oforizinal Health Interview

the 1969 data for theSurvey PSU’S revealed

a ~ean of only 18 segments p& PSU, excludingfrom the average eight PSU’S having over 100segments. This average segment size was toosmall to support a two-stage design. Therefore,the 357 units were regrouped to form 149 PSU’Sfor the universe. Paralleling the HIS design, the149 PSU’S were classified into 19 strata, eightconsisting of only one PSU, grouped primarilyby geography and population size. The originalsegments were retained, except the real outlierp(segments with three or fewer persons) werecombined with other segments to reduce varia-bility of segment size within PSU.

The HIS observations were for persons, butsince in this investigation everyone within aselected segment was to be in the sample(eliminating the problem of adjusting for non-response), the person records were combined toform a single segment record. To recapitulatethe construction of the population, the 357PSU’S in the HIS sample were reorganized into149 units, the units were categorized into 19strata, the segments were edited for outliers, andthe data coIlected on persons in each segmentwere combined into a single segment record.

There are several reasons for using these dataas the population. The data had already beencollected, edited, and made available on mag-netic tape. However, even after the completionof these operations, further extensive manipula-tions were required to put the data into theform required to perform the necessary calcula-tions. One purpose of the project was tomeasure the effects of stratification and cluster-ing on the variance estimators studied. The datareflect the real homogeneity or clustering thatexists in human populations. The estimationcomponent, poststratification, which was to beinvestigated to ascertain whether the varianceestimators were sensitive to estimation proce-dures, could easily be performed on these data.Another basis for their use was the variablescollected in the survey. Evidence of the behaviorof the balanced half-sample replication and thelinearization method is needed for a variety ofvariables in order to determine if observeddifferences between the variance estimators pro-

duced by thenature of the

two methods are causedvariables themselves or

by theby the

methods. The variables available from HIS dataare primarily morbidity information based on anindividual’s own perception of his health.

Sample Design

As stated earlier, the population was dividedinto 19 strata, 8 self-representing and 11 non-self-representing. Characteristics of the universeare given in table 3. First-stage sampling con-sisted of the selection of the eight SR PSU’S andthe selection of two PSU’S from each of the 11NSR stratum.

The number of PSU’S in each NSR stratumwas small; thus, the finite population correctionfactor (fpc factor) would have a considerableimpact on the variances. The usual procedurewould have been to sample without replacementand then include the fpc factor in the formulasfor the variance estimator, but unfortunately,the theory for incorporating fpc factor in thesecalculations has not been developed. The solu-tion was to sample with replacement. Onedesirable feature of a design is for the sample to

Table 3. Characteristics of the population

Stratum

1 ..........2 . . . . . . . . . .3 . . . . . . . . . .4 . . . . . . . . . .5 . . . . . . . . . .6 . . . . . . . . . .7 . . . . . . . . . .8 . . . . . . . . . .9 . . . . . . . . . .

10 . . . . . . . . . .11 . . . . . . . . . .12 . . . . . . . . . .13 . . . . . . . . . .14 . . . . . . . . . .15 . . . . . . . . . .16 . . . . . . . . . .17 . . . . . . . . . .18 . . . . . . . . . .19 . . . . . . . . . .

Total . . . . . .

Number ofprimary

sampling units

11111111

1415131213

4111211131314

149

Number ofsegments

323324283257270191339271593592526482461449481446455502513

7,768

Populationsize

5,2195,2955,0874,6124,6493,3385,5194,517

10,23810,573

8,8157,6387,3327,8087,7857,8Ekl7,6798,8878,610

131,575

15

be self-weighting, which means that each ele-ment has the same probability of being selected.This feature can be included when the PSU issampled with probability proportional to size.The first stage of sampling included the selectionof two PSU’S independently from the 11 NSRstrata with probability proportional to size andwith replacement.

To determine whether sample size plays a partin the observed differences in the varianceestimates produced by the methods, three sam-ple designs were used. The number of PSU’Ssampled was constant, so the subsampling ratewas varied to achieve different overall samplesizes. To accomplish this, a uniform samplingfraction J equal to the product of the PSUselection probability of selecting the PSU timesthe subsample fraction within the PSU, was usedin each stratum. The formula is

f=h XJ$!

where

f= uniform sampling fraction,

fi = the sampling rate of first-stage units, and

jj = the sampling rate of second-stage units.

In design I, f equaled 1/75; in design II, 2/75;and in design III, 2/30. The fraction f2equaledf/fl.Thus, by setting f and knowing fl,the f2could be computed. Because two PSU’S werechosen in the NSR strata, the values 1/15 O,1/75, and 1/30 were used in the computations.The expected yields from the two fractions ofdesigns I and 111 were 4 segments and 20segments, which approach the two possibleextremes in sampling: simple random and ulti-mate cluster. The optimum design probably fallsbetw-ecn the two rates used. The second stage ofsampling was random subsampling with replace-ment of the segments at the rate f2 from thechosen PSU’S.

To clarify the sampling plan, consider designIII, which had an overall sampling fraction of 2in 30. First, the eight SR strata entered thesample with probability 1. Within each, seg-ments were randomly selected at the rate f2=

(2/30)/1 = 2/30. Thus, for stratum 1, approxi-mately 22 segments were drawn. Next, twoPSU’S were chosen with probability proportionalto population from each of the remmining 11strata. Within a selected PSU, the segments weresubsampled with the rate f2= (l/30)/fl.

For each design, 900 samples were independ-ent y drawn. In drawing these samples, not onlywas the sampling distribution of the varianceestimators being estimated, but also being esti-mated was ~he sampling distribution of the ratioestimators Ri, +1, “ . “ , 900. In repeated sam-pling, this ratio estimator will settle down to itsexpected value, and, thus, the simple samplevariance

900---& (R, - @2/899

will become fairly stable. The value 900 wasthought to be an adequate number to achievestability.

Variables

Five variables were selected from the HIS torepresent a variety of basic distributional forms.These are as follows:

Variablaf

Family incomeNumber of restricted activity days in past 2 weeksNumber of physical and dental visits in past 12 monthsNumber of days spent in short-stay hospitals in past 12

monthsWhether or not the person has seen a physician in last 12

months

Quantity estimated

Average income per personAverage number of restricted activity days per person per yearAverage number of visits per person per yearAverage number of short-stay hospital days per person per

yearProportion of population seeing a physician in last 12 months

Two criteria for selection of the variableswere that (1) they could be obtained from theperson-record HIS tapes and (2) they wouldhave a variety of distributions. Ideally, what is

.%ach sample person’s record contained a measure for each ofthe five variables; therefore, to obtain a segment record, thesemeasures were summed across all persons in the segment.

16

desired is for the computed variance estimates toreflect the differences of the variance estimators.Thus, the possibility that the variables them-selves cause the observed differences betweenthe variance estimates is investigated.

It is important to remember in consideringthe variables measured in this study that theultimate sampling unit is the segment and thatalthough several variables are converted to aper-person basis, no direct assessment of within-segment variability is availabIe. Thus, distribu-tions of variables in the study directly reflectsegment-to-segment variation.

The income variable is a family income figure.Thus, each person within a family is assigned thesame value. Income, then, was a highly clus-tered, continuous variable. However, in the HIS,income was reported only in group intervals,with the highest class being open ended. Toconvert from grouped data, midpoints of eachinterval, except for the open one, were thevalues given to individuals. Everyone having afamily income falling into the open intervaI wasassigned the lowest income of the class.

The proportion of persons in the populationvisiting a physician in the past 12 months wasselected as an exampIe of a variable distributedon the unit interval. The variable, number ofrestricted activity days, produces an incidencestatistic based on the respondent’s recall of thepast 2 weeks. The respondent’s recall for theremaining two variables is 12 months. Thevariable, number of days spent in short-stayhospitals in the past 12 months, has a J-shapeddistribution. Most individuals do not spend anydays in the hospital, but there are a few peoplewhose hospital experience lasts for months.These two extremes account for the J-shape.

These variables provide an example from eachtype and each range class of statistics estimatedin the HIS. The statistics produced by the HISare classified by the length of the recall periodand by the usual value of the measure of ahealth characteristic for an individual. Theclasses are (1) narrow range—the measure isusually O or 1 and occasionally 2, e.g., thenumber of days spent in short-stay hospitals; (2)medium range—the typical value for an individ-ual is from O to 5, e.g., the number of physicianand dental visits; and (3) wide range—the meas-

ure for an individual is usually greater than 5,e.g., the number of restricted activity days.

ESTIMATORS AND VARIANCEESTIMATORS

Estimators

Two estimating equations, each producingfrom the sample a ratio poststratified estimateof the population ratio parameter, wereemployed with the level at which poststratifica-tion was done as the differentiating characteris-tic. Each equation consisted of three basicoperations: inflation by the reciprocal of theprobability of selecting second-stage units, infla-tion by the reciprocal of the probability ofselecting first-stage units, and poststratification.

The purposes of stratification are to improveprecision of the overall population estimates andto insure that subgroups of the population(domains of study) are in the sample in the sameproportions as they are in the universe. Usually,the universe cannot be stratified into thesesubgroups before sampling. However, if thedistribution of the subb~oups in the universe isknown, the sampIe resuhs can be adjusted(poststratified) so that these domains of studyare appropriately represented and the accuracyof the estimate increased.

The subgroups in this study were the popula-tion in 24 ethnic-sex-age classes (white andnonwhite, male and female, ages O-4, 5-14,15-24, 25-44, 45-64, and 65+). This spreadresulted in small frequencies in most of thenonwhite cells, another reason for poststratify-ing. Typically, the distribution of the subgroupsis known only for the complete population, andadjustments must be made at this IeveL In thissituation, however, the universe was totallyspecified and poststratification could be carriedout either after combining the stratum estimatesfor an ethnic-sex-age class or earlier. Poststratifi-cation was applied at two different leveIs:universe and region.

Universe level meant that the sample esti-mates for a class were combined across all thestrata and then adjusted to the totaI populationin that group. Region-1evel poststratificationrefers to the adjustment of sample estimate for

17

the PSU’S within a region to the regionaldistribution of classes. The PSU’S belonged toone of two regions: the combined East andNorth Central regions or the combined Southand West regions.

The first estimator-of a statistic (see appendixI for the formula for R2 ) is

RI –poststratification at universe level

Y

the final poststratified ratio estimate ofthe population parameter R,

the poststratified estimatecharacteristic x,

the total population size,

of health

the inflated estimate combined acrossall PSU’S of health characteristic x forethnic-sex-age class a,

the known population in the ethnic-sex-age class a, and

the inflated estimate across all PSU’S ofthe population in class a.

Variance Estimators

The main objective of the study is to examinethe behavior of the variance estimators producedby two general variance estimation methods,replication and linearization. Ideally, the vari-ance of each estimator produced by one of thetwo estimating equations would be estimatedusing the correct application of both varianceformulas and each of the approximate formulas,but due to computer costs and volume ofnumbers, this was not feasible. For estimator 1,variance estimates were computed by the correct

replication and linearization methods and twoapproximate schemes, while for estimator 2,only the correct replication method was used.(See appendix II for the formulas for theapproximate variance estimates and for thecorrect replication variance estimator for R.z.)

An assumption of both general methods isthat two PSU’S are chosen independently fromeach stratum. The sample design for this studywith SR PSU’S violates the assumption givingrise to the problem of how to treat these PSU’S.The solution was to divide randomly the sam-pled segments of each SR PSU into two groupsand to assume that these two groups, pseudo-PSU’S, were two independent selections fromthe same stratum.

Variance Estimators for Estimator 1

The variance for the estimate R ~ was esti-mated in 10 different ways. Three replicationmethods, each yielding three estimators, and onelinearization method were used.

The process for computing the correct replica-tion variance estimator is as follows; Assumethat from each stratum one of the two sampledprimary units is selected. Nineteen PSI-Y’Sform ahalf-sample. The data from these 19 l?SU’S areinflated by the reciprocal of the probability ofselection to their stratum level. These estimatesfor a particular ethnic-sex-age class are combinedacross all the strata. Then, within the classes, theestimates are adjusted by poststratification fac-tors. These factors are calculated, using thepopulation estimates for the class produced byonly the PSU’S in the half-sample, on theuniverse level, in the same manner used tocalculate the factors for the parent sample. The19 PSU’S in the complement replicate undergoan identical process to yield a similar estimate ofthe parameter. This is repeated for 20 suchhalf-samples. The three variance estimates pro-duced for qach sample estimate obtained fromestimating R ~ are

20

VAR(R lH) = ~ ; (R[,a - RI )2a

20

18

and

VAR(R 1S) = ; [VAR(R lH) + VAR(R lC)]

where

& = the parent sample ratio estimate,

% = the cxth half-sample ratio estimate ofthe population parameter R by an iden-tical estimation equation to the onethat yielded R1, and

R~a = the cxth complement half-sample ratio,estimate.

The 20 half-samples are composed from theorthogonal pattern in table 4. This design wasconstructed by following the method describedin Plackett and Burman.l 3 The pattern isdetermined by the rotation of the first 19entries of the first column and by specifying the20th row to always be mifius.

The linearization formula empIoyed in thisstudy was derived from the theorems given byKeyfitz.21 The form of the method used herecan be described as foHows: Remembering thatR ~ = x’: /y, the formula for the aggregate X{ is

This equation can be rewritten as Iinear combi-nations of the simpIe inflated estimates of thePSU’S from each stratum. Expressed this way,the equation becomes

Replicate

1 ................2 . . . . . . . . . . . . . . . .

3 . . . . . . . . . . . . . . . .

4 . . . . . . . . . . . . . . . .

5 . . . . . . . . . . . . . . . .

6 . . . . . . . . . . . . . . . .

7 . . . . . . . . . . . . . . . .

8 . . . . . . . . . . . . . . . .

9 . . . . . . . . . . . . . . . .

lo . . . . . . . . . . . . . . . .

11 . . . . . . . . . . . . . . . .

12 . . . . . . . . . . . . . . . .

13 . . . . . . . . . . . . . . . .

14 . . . . . . . . . . . . . . . .

15 . . . . . . . . . . . . . . . .

16 . . . . . . . . . . . . . . . .

17 . . . . . . . . . . . . . . . .

18 . . . . . . . . . . . . . . . .

19 . . . . . . . . . . . . . . . .

20 . . . . . . . . . . . . . . . .

Table 4. Orthogonal pattern for 20 replicates

Stratum

SR PSU’S NSR PSU’S

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

?--F-

?---

-- +

i-+-

+++

+4-+’

-!- +--

+-i-

-!--

+-+

-1--

-F --

---

---

-- +

-F+

i--F-

+-+

i-+

--

+-E

+++

++-i-

++-

+-+

+-

i--+-

+-

+--

---

---

-- i-

— ++.

+i--

+-+

-1--1’

++-

+--

-- -!-

---

++-+--!-

+-

+-+

+-

+–-

-——

---

+

+--!-

+?--

‘!--+

++

++-

?---

-- +-

++

+-!-+

+++

-—

+

+

—i+

++

+,+++

+

+i-

++

+++i-

+

+

++

i-+

++++

i-

i-+

i-+

++++

+

+

++

++

+i-+i’

+

&

i-i-

i-i-

-t+++

+

+

+i-

++

++++

+

+

++

+-!-

+

+

+

+

+

+

—

+-+’

+ +

+-

+

+ +

+ +

+ -i-

+-

i-

i--

i-

-t-

i-

+ i-

Note.–SR PSU indicates self-representing primary sampling unit; NSR PSU indicates non-self-representing primary sampling

unit.1..+,, denotes +1 and “-” denotes -1.

19

where

X;ai = the inflated estimate of health charac-teristic x for class a of the ith PSU inthe h th stratum and

YL’ = the inflated population estimate, forclass a of the ith MU in the h th stratum.

From this expression, the linearization varianceestimator of R 1 is derived. The formula is

VAR(X:)VAR(L 1) =

Y2

where

X;i = the inflated estimate for the ith PSU ofthe h th stratum,

‘l,a = the final poststratified ratio estimate ofthe health characteristic x for the athethnic-sex-age class.

For more detailed formulas and discussion, seeBean.30

RESULTS

Introduction

The main objective of the investigation was todetermine if either or both of the generalvariance estimator methods, replication andlinearization, yields acceptable variance esti-mates for a complex ratio estimator employed inmultistage probability sample surveys. The prop-erties considered critical for acceptability are theextent of the bias of the variance estimates, thedegree of their variability, and the amount oftheir mean square error. Another very importantpoint is whether inferences can be made whenvariance estimates are used. The other resultpresented is the effect of the estimation tech-

nique poststratification on variances. The bias ofthe two ratio estimators employed and thesubstitution of approximate variance estimatesare discussed in appendixes I and II. Beans Opresents a more in-depth report on other proper-ties of. the ratio estimators, other approximatevariance estimate methods, and the results, usingseveral different procedures for estimating thevariance component from SR PSU’S.

Comparison of Replication and Linearization

The primary purpose of this investigation wasto compare replication and linearization meth-ods of variance estimation for a sample designthat includes stratification, two stages of cluster-ing, and poststrati~led ratio estimation. Thecomparisons are in terms of bias, variamce, andmean square error.

Only for the sample estimator R ~ were boththe appropriate replication and linearizationvariance estimates computed. “Appropriate”here refers to the best application of the theoryof the methods to the particular sample designand estimation process employed. For the repli-cation method, this means that the data for thePSU’S in a half-sample were (1) inflated to thePSU level, (2) inflated next to the stratum level,and (3) poststratified by adjustment factorscomputed from the half-sample itself. Thereforejthe half-sample estimate of the poptdationparameter contained all the elements of thesample design and estimation procedure. For thebest application of the linearization theory, thebasic equations by Key fitzz 1 were interpretedto include inflation to PSU and stratum levelsand to include an adjustment for poststratifica-tion. For each estimate of R ~, one linearizationand three replication variance estimates werecalculated. The replication estimates VAR(R 127), VAR(R 1C), and VAR(R 1S) all have thesame expected value. Because VAR(R 1S) is anaverage of the other two, its precision is ex-pected to be greater. Since the estimatorVAR(R lH) is the one most often used ‘by statis-ticians, the discussion will deal with it and withVAR(R 1S).

One important question is whether the meth-ods yield biased statistics. To compute true bias,the real variance of R ~ for this sample design isneeded. Unfortunately, this value was not

20

knoyn, but with 900 samples, the distributionof R 1 should become. quite stable; and thesample variance of 900 R 1 estimates shpuld be agood criterion. The sample variance of R ~ is

where

900=

‘L ZR’lj

‘1 – 900 j.1

By assuming s’fi ~ to be the true variance,

measures of bias can be calculated. These are

bias [VAR(R

where

VAR(R W

H)] = “Vmp7iq - s?RI

bias [VAR(RIS)] = VAR(RIS) - S2=

Table 5. The bias of the RIH, R’

Variance estimator

VAR(RIH) . . . . . . . . . . . . . . .VAR(RIS) . . . . . . . . . . . . . . .VAR(L1) . . . . . . . . . . . . . . . .

VAR(RIH) . . . . . . . . . . . . . . .VAR(RIS) . . . . . . . . . . . . . . .VAR(fl) . . . . . . . . . . . . . . . .

VAR(RIH) . . . . . . . . . . . . . . .VAR(/?lS) . . . . . . . . . . . . . . .VAR(L1) . . . . . . . . . . . . . . . .

K]

where

900

VAR(ll 1s) = & ~ [VAR(R 1s)] j,

and

bias [VAR(L 1)] = VAR(L 1) - s’.Rl

where

900

~ [VAR(L1)] jvAR(~I.) ‘ & j.1

Inspection of table 5 reveals that the bias ofthe R lH estimator decreases across the threedesigns for four of the five variables. Only forthe variable proportion seeing a physician doesthe bias, which goes from -6.083 X 10-6 indesign I to 1.614 X 10-5 in design II and thendown to 4.206 X 10-6 in design III, not followthe pattern. For physician visitsand hosp’~aldays; the bias is always positive, while for farndyincome and restri-cted activity days, it switchesfrom’ positive to negative.

, and L 1 variance astimatom of the sample estimator F, of the population parameter R

Variable

Family incomeRestricted

Physician visits Hospital daysProportion seeing

activity days a physician

3.857 X 10’4.103 x 10’

-3.191 x 10’

2.736 X 10-’2.783 X 10-’

-1.085 X 10-’

1.489 X 10’ -4.337,x 10-21.640X 103 -4.082 x 10-2

-3.211 X 102 -1.598 X 10 “

-8.772 X 102-8.809 X 10’-1.271 X 103

-2.934 X 10 ‘=-2.900 X 10 “-5.818 X 10 ‘z

Design I

2.751 X 10 ‘32.558 X 10 ‘3

-4.050 x 10-3

Design II

2.693 X 10 “2.668 X 10 ‘31.022 x 10-3

Dasign III

1.073 x 10-’1.042X 10-37.121 X 10-4

3.750 x 10-33.765 X 10 ‘3

-2.242 X 10 ‘3

1.928 x 10-31.785X 10-3

-6.848 x 10-5

-6.083 X 10-s-7.095 x 10-6-4.944 x 10 -s

1.614X 10-51.581 X 10-s6.845 X 10-s

21

For family income, restricted activity days,and hospital days, the bias of VAR(R 1S) alsodecreases with increasing sample size. For thefirst two variables mentioned, the bias is positivein design I, but in design III, it is negative. Forphysician visits, the bias in design II is slightlygreater than the bias in design I, while for designIII, it is at its smallest value. VAR(R 1S)’s biasshows a different trend for proportion seeing aphysician. The bias is -7.095 X 10-6 in design I,which increases to 1.581 X 10-T in design II andthen drops to 4.069 X 10-6 in design III. This isthe only example for VAR(R 1S) and VAR(R lH) for which the bias first is negative andthen becomes positive.

VAR(L 1) exhibits a different pattern. Thebiases for physician visits and proportion seeinga physician are negative for design I. Thesebiases decrease across the designs. However, themethod goes from underestimating to over-estimating the variance. The bias is the smallestin design 11 for family income and hospital days.

Comparing the biases of the three methodsdoes not yield a consistent pattern of one of themethods having less biaa than the other. Theimportant fact is that for all three estimators,the bias is small and tolerable. However, turningto the variances of the methods in table 6, more

of a trend is found. The variances of the varianceestimates are:

VAR[VAR(R lH)] = &

VAR[VAR(R 1S)] = &

VARIVAR(L1)] = &

900

x ~ {[vAR(~l)]j- rA~}2

The first point to consider when examiningthe variance of estimators is to check to makesure the variance does decrease with increasingsample size. This is true for all three methods.

Tab!e 6. The variance of the RIH, R1.$, and f. 1 variance estimators of the sample estimator ~1 of the population parameter R

VariableVarianceestimator

Family incomeRestricted

Physician visits Hospital daysProportion seeing

activity days a physician

Design I

VAR(RIH) . . . . . . . . . . . . . . . . . . . 1.144 x 109 2.052 2.859 X 10 “ 2.428 X 10-3 1.733X 10-8

VAR(RIS) . . . . . . . . . . . . . . . . . . . 1. II IX IO$’ 1.976 2.763 X 10 ‘3 1.902X 10-3 1.579X 10-S

VAN .. . . . . . . . . . . . . . . . . . . 1.062 X 109 1.487 2.380 X 10-3 2.833 X 10 “ 1.265 X 10-*

VAR(RIH) . . . . . . . . . . . . . . . . . . . 3.217 X 10*VAR(RIS) . . . . . . . . . . . . . . . . . . . 3.201 X 10’VAN .. . . . . . . . . . . . . . . . . . . 3.258 X 10’

Design I I

0.521 8.306 X 10-4 1.199 x 10-4 3.471 x 10-90.505 8.274 X 10-” 1.042X 10-’ 3.307 x 10-’0.460 8.250 X 10 ‘4 5.651 X 10-5 3.338 X 10-9

Design I I I

VAR(RIH) . . . . . . . . . . . . . . . . . . . 1.047 x 1Os 0.140 8.271 X 10-s 1.319X 10-5 1.I04X 10+

VAR(RIS) . . . . . . . . . . . . . . . . . . . 1.036 X 108 0.137 8.132X 10-s 1.288X 10-s 1.089X 10-’

VAN .. . . . . . . . . . . . . . . . . . . 1.048 X 108 0.131 7.959 x 10-’ 1.135X 10-s 1.D86X 10-9

22

The variance of the VAR(R lH) is always largerthan the variance of VAR(R 1S). This providesempirical evidence that R 1S is more precise thanthe other forms of replication variance esti-mators. With three exceptions—family income indesigns II and III and proportion seeing aphysician in design II–the variance of theVAR(L 1) method is less than the variance ofVAR(RIS).

It is preferable to judge the merits of thethree methods on the basis of the mean squareerror, which takes into consideration both prop-erties of bias and variance. The ideal estimatorwould be one that produces minimum meansquare error for all values of the parameterr32& . Perhaps such an estimator does not exist,

but ;he empirical results here provide evidenceon the behavior of the mean square error for theR Ml, R 1S, and L 1 estimator methods. Theformulas for the mean square errors (MSE) are

MSE [VAR(R lH)] = VAR[VAR(R I.H)]

MSE [VAR(R 1S)] = VAR[VAR(R 1S)]

+ biasz [VAR(R 1S)]

= VAR[VAR(R 1S)]

+ [VAR(RIS) - S;l] 2

and

MSEIVAR(L1)] = VAR[VAR(L 1)]

+ biasz [VAR(L 1)]

= VAR[VAR(L 1)]

+ [VAR(L1) - s;l]2

As can be observed in table 7, the mean+ biasz [VAR(R lH)] square errors for the methods for each variable

decrease with increasing sample size. When

= VAR[VAR(R lk?)] looking at the values for the procedures pair-wke, the results indicate that VAR(R 1S) zdways

+ [VAR(R Ill) - S;l] 2 has a smaHer mean square error than VAR(R lH), which is easily explained by the less

Table 7. The mean square error of the Rlf+, RIS, and L 1 variance estimators of the sample estimator ~1 of the population parameter R

Varianceestimator

VAR(RIH) . . . . . . . . . . . . . . . . . . .VAR(RIS) . . . . . . . . . . . . . . . . . . .VAN .. . . . . . . . . . . . . . . . . . .

VAR(RIH) . . . . . . . . . . . . . . . . . . .VAR(RIS) . . . . . . . . . . . . . . . . . . .VAN .. . . . . . . . . . . . . . . . . . .

VAR(J?IH) . . . . . . . . . . . . . . . . . . .VAR(RIS) . . . . . . . . . . . . . . . . . . .

VAR(.LI) . . . . . . . . . . . . . . . . . . .

Variable1 1 t I

Family incomeRestricted IPhysician visits IHospital days IProportion seeing

activity days a physicianI I 1 I

1.159X 1091.128X 10’1.072 X 109

3.239 X 1083.228 X 10s3.259 X 103

1.055 x 10’1.044 x 1081.064X 10*

2.1272.0531.499

0.5220.5070.485

0.1400.1380.135

Design 1

2.866 X 10-32.770 X 10-32.396 X 10 ‘3

Design I I

8.379 X 10 ‘“8.323 X 10 ‘47.653 X 10 ‘4

Design 1I I

8.387 X 10-s8.240 X 10-’8.OIOX 10-s

2.442 X TO“31.916X 10-32.883 X 10-4

1.236X 10-41.074X 10-45.651 x 10-s

1.320X 10-51.288X 10-’1.145X 10-s

1.736X 10-*1.584X 10-s1.510X 10-*

3.731 x 10-93.557 x 10-93.386 X 10-9

1.122X 10-91.I06X 10-91.091 x 10-9

23

variability of VAR(R 1S). The other finding isthat VAR(L 1) has a smaller mean square errorthan VAR(R 1S ) aside from the variable familyincome in designs II and III. The mean squareerror is less even though VAR(L 1) did notalways have the smaller bias. For the only twocases where the mean square error of VAR(L 1)is not lower, the variance of VAR(L 1) wasgreater. In the other situation, proportion seeinga physician in design II, of greater variance forthe L 1 method, the difference in the bias waslarge enough to cause VAR(L 1) to have thesmaller mean square error.

The behavior of the replication and lineariza-tion methods for the properties of bias, variance,and mean square error is very good, whichimplies the methods are yielding acceptablevariance estimates. However, there are otherproperties to consider when deciding on avariance method; one of these is examined next.

Normality of Standardized Estimators

As mentioned previously, scientists are in-creasingly concerned with making inferencesfrom data collected in scientific sample surveysrather than just calculating descriptive statistics.Ideally, an investigator would like to constructconfidence intervals for population parameters.The statement that the average annual incomeper person lies between $7,500 and $9,400 with95 percent confidence is more useful than apoint estimate of the average income. Such astatement using the normal distribution could bemade if one is willing to assume that the variableis sufficiently close tom-being normally distrib-uted and that the departure from randomsampling is not a severe limitation.

The objective here is to study empirically theratio estimate minus its expected value dividedby its estimated standard error. The analysisconsists of computing the proportion, of timesthe statistic falls into certain regions. By thisprocedure, the applicability of confidence state-ments based on the normal distribution can beapproached.

In this section, only the statistics for whichVAR(R lH), VAR(R 1S), and VAR(L 1) were

used as the variance estimates will be discussed.These ratios are

il - -E(iil)

i=mm ‘RI - E(iJ

@iii@iq ‘and

RI - E(iil)

@mqzij

where

Such ratios for each estimate in every sampleand the proportion of times the ratio fell withinthe regions–(- 1.000, 1.000), (- 1.000, O), (O,1.000), (-1.645, 1.645), (-1.645, O), (O, 1.645),(-1.960, 1.960), (-1.960, O), (1.960, O), and(-2.576, 2.576)–were computed. These regionswere chosen for purposes of comparison withthe normal distribution. The proportion of areaunder the normal curve for the four symmetricintervals are 0.6827, 0.90, 0.95, and 0.99,respectively.

The proportion of times the ratio fell withinthe limits is given in tables 8-10. Considering justthe symmetric intervals, one immediatelynotices that intervals based on this type ofstatistic usually have a lower confidence thanthe normal level, but the differences are gener-ally very small. There are instances in which theproportion of time the statistic based on thesedifferent variance estimates fell within the limitsexceeded the corresponding normal values.These results are good news, indeed, becausecoupled with the findings of Frankelz S theyjustify the type of inferences consumers of datacollected in multistage complex sample surveys

24

often wish to make. For example, a health indicate that such an inference based on theplanner concerned with the use or expansion of normal distribution is really not a bad approxi-health care facilities would most likely need to mation and could be used with reasonabledraw an inference from the estimated propor- confidence.tion of the population who saw a physician at A word of caution about these findings isleast once in the past 12 months. The data appropriate. Inspection of the one-sided inter-

Tabla 8. Proportion of timas the sample estimate minus its expected value divided by RIH estimate of standard error is within the

Variable

Familyincome .

Restrictedactivitydays . . .

Physicianvisits . . .

Hospitaldays . . .

Proportionseeing aphysician

Familyincome .

Restrictedactivitydays . . .

Physicianvisits . . .

Hospitaldays . . ,

Proportionseeing aphysician

Familyincome .

Restrictedactivitydays . . .

Physicianvisits . . .

Hospitaldays . . .

Proportionseeing aphysician

stated limits

Limits

*1.000 -1.000,0 0, 1.000 il .645 -1.645,0 0, 1.645 *I .960 -1.960,0 0, 1.960 *2.576

Design I

0.6789

0.6778

0.6444

0.6556

0.6567

0.6944

0.6356

0.6733

0.6722

0.6833

0.6656

0.6511

0.6844

0.6789

0.6900

0.3411

0.3322

0.2989

0.2933

0.3433

0.3567

0.2956

0.3244

0.3233

0.3489

0.3189

0.3322

0.3333

0.3422

0.3511

0.3378

0.3456

0.3456

0.3622

0.3133

0.3378

0.3400

0.3489

0.3489

0.3344

0.3467

0.3189

0.3511

0.3367

0.3389

0.8833

0.8811

0.8744

0.8622

0.8689

0.8833

0.8844

0.8967

0.8733

0.9089

0.8656

0.8700

0.8989

0.8789

0.8756

0.4500

0.4411

0.4300

0.4156

0.4400

0.4333

0.4400

0.4444

0.4467

0.4289

Design I I

0.4467

0.4356

0.4478

0.4367

0.4589

0.4367

0.4489

0.4489

0.4367

0.4500

Design I I I

0.4167

0.4456

0.4456

0.4467

0.4378

0.4489

0.4244

0.4533

0.4322

0.4378

0.9356

0.9367

0.9233

0.9122

0.9300

0.9389

0.9300

0.9367

0.9289

0.9522

0.9278

0.9244

0.9456

0.9244

0.9222

0.4744

0.4711

0.4633

0.4468

0.4633

0.4733

0.4622

0.4744

0.4722

0.4789

0.4522

0.4733

0.4678

0.4722

0.4556

0.4611

0.4656

0.4600

0.4633

0.4667

0.4656

0.4678

0.4622

0.4567

0.4733

0.4756

0.4511

0.4778

0.4522

0.4667

0.9800

0.9789

0.9822

0.9578

0.9811

0.9744

0.9778

0.9811

0.9767

0.9822

0.9767

0.9756

0.9822

0.9656

0.9833

25

Table 9. Proportion of times the sample estimate minus its expected value dividad by HIS estimate of standard error is within the

Variable

Family income

Restricted activ-

ity days . . .

Physician visits .

Hospital days . .

Proportion

seeing a

physician . .

Family income .

Restricted activ-

ity days . . .

Physician visits .

Hospital days . .

Proportion

seeing a

physician . .

Family income .

Restricted activ-

ity days . . .

Physician visits .

Hospital days . .

Proportion

seeing a

physician . .

stated limits

Limits

?1.000 -1.000, 0 0, 1.000 *1 ,645 -1.645,0 0, 1.645 *1.960 -1.960,0 0, 1.960 %2.576

0.6800

0.6722

0.6556

0.6611

0.6578

0.6911

0.6344

0.6722

0.6800

0.6767

0.6678

0.6511

0.6878

0.6789

0.6900

0.3456

0.3278

0.3067

0.2956

0.3433

0.3567

0.2956

0.3233

0.3244

0.3456

0.3200

0.3311

0.3356

0.3422

0.3511

0.3344

0.3444

0.3489

0.3656

0.3144

0.3344

0.3389

0.3489

0.3556

0.3311

0.3478

0.3200

0.3522

0.3367

0.3389

0.8822

0.8811

0.8733

0.8644

0.8656

0.8822

0.8833

0.8978

0.8744

0.9133

0.8667

0.8689

0.8978

0.8789

0.8778

vals indicates that they are not symmetric.Thus, any statements about one-sided intervalscannot be made, and confidence intervals com-puted are not of minimum length. An area offurther study could be the direct corroborationof the skewness and Ieptokurtosis noted in thetables by computing and examining the thirdand fourth sample moments.

1nfluence of Poststratification

Another purpose of the study is to determinethe effect the estimation component poststratifi-cation has on variances. The reason for adjustingis to lower the variance. Normally, to reducevariance a larger sample is drawn, but if the samething can be accomplished by the estimation

Design I

0.4478 0.4344

0.4356 0.4456

0.4300 0.4433

0.4178 0.4467

0.4356 0.4300

Design I I

0.4456

0.4333

0.4500

0.4389

0.4611

0.4367

0.4500

0.4478

0.4356

0.4522

Design III

0.4189 0.4478

0.4456 0.4233

0.4422 0.4556

0.4478 0.4311

0.4389 0.4389

0.9344

0.9378

0.9244

0.9156

0.9322

0.9378

0.9311

0.9367

0.9344

0.9522

0.9256

0.9222

0.9422

0.9267

0.9256

0.4733

0.4700

0.4622

0.4533

0.4667

0.4733

0.4644

0.4733

0.4744

0.4800

0.4500

0.4722

0.4656

0.4722

0,4589

0.4611

0.4678

0.4622

0.4622

0.4656

0.4644

0.4667

0.4633

0.4600

0.4722

0.4756

0.4500

0.4767

0.4544

0.4667

0.9789

0.9789

0.9778

0.9578

0.9811

0.9756

0.9756

0.9778

0.9767

0.9822

0.9778

0.9778

0.9822

0.9644

0.9833

procedure, there can be a savings in cost byusing a smaller sample size, unless, of course, thesavings in field costs are offset by costs ofimplementing the estimation procedure. Vari-ance reduction has been demonstrated primarilyusing simplified methods and theoretical analy-sis, while the studies of Simmons and Beanz 0and Banks and Shapiros gave empirical evidence.This investigation examines the effect for theparticular design used here.

The VAR(R lH) and VAR(R 1C) schemes callfor the poststratification factors to be computedwithin each half-sample and each complementhalf-sample, while for the methods VAR(R 2H)and VAR(R 2C), the parent sample adjustmentfactors are used. (See appendix II for :formulasfor VAR(R 227) and VAR(R 2C’).) The orthog-

26

Table 10. Proportion of times the semple estimate minus its expected value divided by L 1 estimate of standard error is within the

Varieble

Family income .

Restricted activ-

ity days . . .

Physician visits .

Hospital days . .

Proportion

seeing a

physicien . .

Family income .

Restricted ectiv-

ity days . . .

Physician visits .

Hospitel deys . .

Proportion

seeing a

physician . .

Family income .

Restricted activ-

ity days . . .

Physician visits .

Hospital days . .

Proportion

seeing a

physicien . .

stated I imits

Limits

*1.000 -1.000,0 0, 1.000 *1 .645 -1.645,0 0, 1.645 *I .960 -1.960,0 0, 1.960 t2.576

Design I

0.6733

0.6433

0.6211

0.6200

0.6189

0.6811

0.6178

0.6533

0.6511

0.6544

0.6644

0.6400

0.6811

0.6689

0.6867

0.3400

0.3189

0.2867

0.2844

0.3211

0.3522

0.2867

0.3167

0.3166

0.3378

0.3167

0.3256

0.3311

0.3322

0.3489

0.3333

0.3244

0.3344

0.3356

0.2978

0.3289

0.3311

0.3367

0.3356

0.3167

0.3478

0.3144

0.3500

0.3367

0.3378

0.8622

0.8611

0.8489

0.8344

0.8378

0.8833

0.8644

0.8900

0.8533

0.9022

0.8600

0.8611

0.8956

0.8733

0.8722

0.4378

0.4244

0.4222

0.4011

0.4244

0.4244

0.4367

0.4267

0.4333

0.4133

Design II

0.4444 0.4389

0.4211 0.4433

0.4467 0.4433

0.4256 0.4278

0.4522 0.4500

Design I I I

0.4133

0.4411

0.4444

0.4456

0.4367

0.4467

0.4200

0.4511

0.4278

0.4356

0.9211

0.9244

0.9044

0.8933

0.9022

0.9289

0.9211

0.9356

0.9200

0.9467

0.9256

0.9156

0.9378

0.9233

0.9244

0.4667

0.4667

0.4522

0.4411

0.4511

0.4667

0.4567

0.4744

0.4678

0.4744

0.4489

0.4700

0.4622

0.4744

0.4567

0.4544

0.4578

0.4522

0.4522

0.4511

0.4622

0.4644

0.4611

0.4522

0.4722

0.4767

0.4456

0.4756

0.4489

0.4689

0.9722

0.9711

0.9711

0.9600

0.9771

0.9744

0.9711

0.9767

0.9700

0.9800

0.9767

0.9756

0.9822

0.9644

0.9844

onal Pattern used for the methods was the Only the third replication scheme is discussed.same, - producing identiczd composition. Theratio of the variance estimator secured fromVAR(R lH) to the estimator produced byVAR(R ZH) does measure the relative impact ofpoststratification. The average of these ratioswas calculated. The ratios are defined to be

average ratio of variances ~ 900 [VAR(R 1~)] j

for method i = ~ ~ [VAR(R2~)l j

where

i = H,c,s.

Computing the poststratification weights in eachhalf-sample reduces variance estimates for familyincome and physician visits (see the ratio ofVAR(R 1S) to VAR(R2S) in table 11. This istrue for all three designs. A reduction is achievedfor all the remaining variables in design III.These data provide evidence that poststratifica-tion does improve precision.

SUMMARY

Introduction

Increasing use of scientific survey samplinghas led to the development of a wide variety oftechniques. These have been produced in re-

27

Tabla 11. The relative impact of poststratification on the variance estimates

Variable

Family income . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Restricted activitydays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Physician visits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Hospitaldays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Proportion seeingaphysician . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Family income . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Restricted activitydays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Physician visits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Hospital days . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Proportion seeingaphysician . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Family income . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Restricted activitydays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Physician visits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..$... . . . .

Hospitaldays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Proportion seeingaphysician . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Average value of the ratic) of

I IVAR(RIH) I VA R(RIC) I VARU?IS)

VA R(R2H) VAR(R2C) vAR(R2s)

0.833

1.030

1.001

1.035

1.034

0.840

1.002

0.973

1.015

0.985

0.851

0.980

0.961

0.967

0.975

Design I

0.833

1.030

1.000

1.036

1.026

Design II

0.844

1.005

0.973

1.004

0.984

Design I I I

0.850

0.982

0.961

0.966

0.973

0.833

1.031

1.000

1.035

1.030

0.842

1.003

0.973

1.009

0.984

0.850

0.981

0.961

0.967

0.974—

sponse to the need to sample populations that I. Behavior of the two general variance esti-are geographically scattered, requiring largePSU’S. Theuseof poststratificationt oadjustthedistribution of the sample for certain demo-graphic characteristics has produced furthermethodologic refinements. Such features justifythe generic term “complex multistage probabil-ity samples” to describe the methods. Becauseof the complexity of these samples, varianceformulas for estimators are themselves,compli-cated, and often only approximate expressionscan be obtained. Also, since the criteria ofsimple random sampling, independence, andnormality are not met, classical statistical proce-dures must be examined to determine theirapplicability in such an environment. This in-vestigation has employed an empirical approachto consider various problems of estimating vari-ances and making inferences for complex sam-ples.

By the procedure of Monte Carlo samplingfrom a completely specified universe, the follow-ing points were investigated:

mater methods: Iinea;ization and replica-tion

2. Distribution of the ratio of an estimatedmean minus its expected value divided byits standard error with the normal distribu-tion

3. Study of the bias of two estimators4.1nvestigation of a simpler variance estima-

tor as an approximation to the correctreplication and linearization procedures

5. Measurement of the impact of poststratifi-cation on variance estimates produced bythe two general methods

The study considered a particular sample designand estimation process. The design was complex,involving stratification and two stages of cluster-ing with poststratification. The results are for900 samples drawn for three different samplesizes and, thus, are based on a considerablebody of evidence.

28

Behavior of Linearization andReplication

The variance of a poststratified ratio esti-mator was estimated by the replication andlinearization methods. Three different varianceestimators were computed from the replicationmethod. For each of 900 samples drawn forthree sample sizes, two ratio estimates and theirvariances were computed. The variance estimatesproduced by these techniques VAR(R I-H),VAR(R 1S), and VAR(L 1) were compared onthe bases of bias, variance, and mean squareerror.

For the sample design and estimation proce-dure employed here, both the replication andlinearization methods provide generally ade-quate variance estimates. Almost no severeabnormalities were observed. The biases of themethods are satisfactory, and the measures oftotal mean square error for the methods areadequately small with one exception. The vari-ance of the VAR(L 1) method is generally lessthan the variance of VAR(R 1S’), especially forthe variable hospital days.

Normality of Standardized Estimators

The distribution of the ratio

sample estimate -E(sample estimate)

~ VAR(sample estimate)

was studied. When the sample design and estima-tion procedure result in a complex survey, theexact distribution of this ratio is unknown. Forthis study, such ratios were computed for eachestimator, VAR(R W) , VAR(R 1S), andVAR(L 1). To examine confidence interval state-ments, the proportion of times the ratio fellwithin stated limits was computed. The sym-metric limits chosen were the normal deviatesfor the 99th, 95th, 90th, and 68.27th quartilesof a normal distribution. Also calculated was theproportion of times the ratio fell in the one-sided intervals (- 1.000, O) (O, +1.000), (- 1.645,O) (O, 1.645), (- 1.960, O), and (O, 1.960).

Proportions for the symmetric intervals withVAR(R 1S), VAR(R lH), and VAR(L 1) as thevariance estimator were exceedingly close to

corresponding normalthe proportions for

values. For the most part,the three methods were

smal~er ~han the proportions for the normaldistribution. Examination of the one-sided inter-vals showed that the proportions on either sideof zero are not the same. This fact and theindication of Ieptokurtosis provided evidencethat the distributions of the ratios are notsymmetric.

This body of empirical data indicates thatapproximate interval estimates based on normaldistribution theory can be made. Other impor-tant points are that (1) the true confidence levelis somewhat lower than the nominal IeveI of thenormal, (2) one-sided intervals cannot be con-structed, and (3) because of asymmetry, theseintervals are not minimum Iength.

Comparison of Two Ratio Estimators

Sampling theory states that ratio estimatorsare biased.z 24 In the present study, two esti-mators, distinguished by the-Ievel of poststratifi-cation, were investigated: R 1 was a~justed tothe universe totals of a ckiss, and R2 was acombined ratio estimator with the poststratifica-tion adjustment at the region level. Theseestimators were czdculated for each of the fivevariabIes for every seIected sample.

The bias and relative bias were reported. Thedifferential] bias between 71 (average of rqplicateand complement estimators of R) and R 1 wasalso determined using the first replication equa-tion. None of the relative biases was greater than1 percent. Thus, these ratio estimators areessentially unbiased for the sample sizes studied.

Approximate Variance Estimators

Two computationally simpler methods wereincluded in the study to determine whether oneof them could be use~ to produce approximatevariance estimates of R 1. The fkst method was areplication method in which the parent sampleadjustment factors were used in place of sepa-rate factors for each half-sample. The secondmethgd produced an overall variance estimatefor Rl by fi~st estimating the variances of

fiEN c,1 and RS w,1 and then combining thetwo correctly. Included in this discussion were

29

the results of estimating variances correctly forR2.

Only the first approximation VAR(R2S) canbe seriously considered for possible use. Thismethod was more biased than the correctprocedure, but the mean square errors werewithin tolerable bounds. The proportions pro-duced when this variance estimator was used donot have large deviations from the normalvalues. However, the direction of the deviationsvaried more than for the VAR(R 1S) method.

The behavior of VAR(R4S), which is a linearcombination of subuniverse estimators, was ade-quate. The difference between this method andthe approximate one, VAR(R 3S), is that VAR(R4S) is the appropriate variance estimato~ forthe ratio estimator R2, The ratio estimator R2 ispoststratified to the regional distribution. There-fore, the variances are weighted averages of thevariance estimators of the region estimators. Thebias, variance, and mean square error were foundto be acceptable and not to be extremely large.The findings on the ratio of the sample esti-matm- minus its expected value divided by itsstandard error were that for VAR(R4S) theempirical proportions for the symmetric inter-vals compared favorably to the proportions forthe normal distributions.

Influence of Poststratification

Theory states that the benefits derived fromadjusting sample results to population totals forselected characteristics lie in improvement of theprecision of the ratio estimator by insuring thatdomains of study have the same distribution inthe sample as in the population. Assessment ofthe effect of poststratification on variances wasa natural byproduct of the calculations done forthe other methods. The estimates of variancesproduced by the first replication method werecompared with those secured from the secondequation. In the first scheme, poststratificationfactors were computed within each half-sampleand each complement using half the sampledata, while the parent sample factors wereapplied when making these estimates in thesecond scheme.

VAR(R 1S) showed a reduction in variance forfamily income in design I; for family income,

physician visits, and proportion seeing a physi-cian in design II; and for all the variables indesign III.

Conclusions

The outstanding finding was that both thereplication method and the linearization methodare highly satisfactory methods for estimatingvariances of poststratified ratio estimators ob-tained from complex probability surveys of thetype studied here. Features of the sample designemployed were unequal sampling fractions,stratification, and two stages of clustering. Sincethe methods yield adequate variance estimates,they provide the means for drawing valid infer-ences with known precision from such surveys,For these data, the linearization variance estima-tion method had slightly lower variance andmean square error than the replication method.Neither method showed substantial bias.

The proportions based on the ratios of thesample estimate minus its expected value dividedby its standard error as estimated. by thereplication method were closer to the corre-sponding normal values than the proportionswith the linearization method as the varianceestimator. However, the ratios with the L 1

variance estimator had a more consistent patternthan the ratios with the standard error estimatedby replication method. The consistency wasevidenced by the difference between empiricalproportions and the corresponding ones for thenormal distribution getting smaller as samplesize increased.

The applicability of confidence intervals byusing normal distribution theory has been shownto be adequate. The true level of confidence isusually lower than the confidence stated, whichshould be brought out when making inferences.Another item that should be mentioned is thatthe distributions of the ratios are not symmetric.Thus, one-sided intervals are not valid and theintervals constructed are not of minimumlength.

The biases of the ratio poststratified estima-tors were negligible for the sample sizes studied.

The VAR(R 2S) estimator proved to be anexcellent approximation, and its use is recom-mended. However, there is a tradeoff involved

30

when using an approximation. The gain insimpler formulas may be offset by larger biases.Also, the proportions produced when theVAR(R2S) was used to estimate standard errorsvaried widely in the direction of its deviationfrom the proportions of the normal distribution.The investigator should consider that consumersof the statistics may make such inferenceswhether or not he does. Then each must weighthe losses and the consequences when decidingabout a variance estimation method.

The effect of the estimation componentpoststratification was to lower variance andmean square error. This reduction is importantin making inferences from complex samplesurveys and can be converted into cost savings.

Relevancy to the H IS

The striking feature of this investigation hasbeen to provide empirical evidence that datacollected in complex multistage probability sur-veys such as the HIS can be used for analyticalpurposes. Variance estimates, which are essentialin analyses, can be calculated by using either thereplication or the linearization variance esti-

mator method. Having adequate variances opensavenues of further research in developing testingtechniques and other refined analyses, perhapsalong the Iines of multivariate procedures, com-parable to classicaI ones.

Equally rewarding as finding that suitablevariance estimates can be computed was theresult that tests and confidence intervals can beapproximated by normaI distribution theory.This allows one to draw inferences about themorbidity estimates published by the HIS.

The HIS has now a choice of either of the twomethods to use in computations. The variabilityof the linearization method was slightly lowerthan that of the replication method, but neitheris very biased. The Linearization equations caneasily be interpreted to obtain estimates ofcomponents of variance, which are crucial inplanning such surveys. Another factor to con-sider when choosing a variance estimatormethod is that the replication technique is easiIyapplicable to the estimation of variances formany types of estimators without deriving newtheory or new computer programs for eachestimator. This is not true for the linearizationmethod.

000

31

1Kis.h, Leslie: Confidence intervals for clustered samples. AmSot. Rev. 22(2): 154-165, Apr. 1957.

2Kish, Leslie: Survey Sampling. New York. John Wdey &Sons, Inc., 1965.

3Deming, WNiam E.: Some Theory of Sampling. New York.John Wdey & Sons, Inc., 1950.

4Hansen, M. H., Hurwitz, W. N., and Madow, W. G.: SampleSuwey Methods and Theory. Vol. 1. New York. John Wiley &Sons, Inc., 1953.

5“Student.” (W. S. Gossett): The probable error of a mean.Biometrika 6(Pt. 1):1-25, Mar. 1908.

6National Center for Health Statistics: Replication: Anapproach to the analysis of data from complex surveys. vital andHealth Statistics. Series 2, No. 14. DHEW Pub. No. (HSM)73-1269. Health Services and Mental Health Administration.Washington. U.S. Government Printing Office, Apr. 1966.

7U.S. Bureau of the Census: The Current Population Survey:A report on methodology. Tech. Paper No. 7. Washington. U.S.Government Printing Office, 1963.

‘Banks, Martha J., and Shapiro, Gary M.: Variances of theCurrent Population Survey, including within- and between-PSUcomponents and the effect of the different stages of estimation.Proc. Sot. Statkt. Sec. Am. Statist. A., 1971,40-49.

‘Gurney, Margaret: The variance of the replication methodfor estimating variances for the CPS sample design. Washington.U.S. Bureau of the Census, 1962. (Dittoed memorandum.)

1‘National Center for Health Statistics: Cycle I of the HealthExamination Survey: Sample and R&.ponse, United States,1960-1962. Vital and Health Statistics. PHS Pub. No.1000-Senes 1l-No. 1. Public Health Service. Washington. U.S.Government Printing Office, 1964.

1 lMcCarthy, Philip J., Simmons, Walt R., and Losee, GarrieJ.: Replication techniques for estimating the variances fromcomplex surveys. Paper presented at the Joint Session of theJoint Epidemiology and Statistics Section of the AmericanPublic Health Association, Chicago, Ill., Oct. 18, 1965.

12simmon5, w~t FL, and Baird, James T., Jr.: Pseudo-replication in the NCHS Health Examination Survey. Proc. Sot.Statist. Sec. Am. Statist. A., 1968, 19-30.

13Plackett, R. L., and Burrnan, J. P.: The design of optimummultifactorial experiments. Biornetrika 33 (Pt. IV):3 05-325, June1946.

14Lee, Kok-huat: Partially balanced designs for half-samplereplication method of variance estimation. J. Am. Statist. A.67(338):324-334, June 1972.

15National Center for Health Statistics: Pseudo-replication:Further evaluation and application of the balanced half-sampletechnique. Vital and Health Statistics. Series 2, No. 31. DHEWPub. No. (HSM) 73-1270. Health Services and Mental Health

Administration. Washington. U.S. Government Printing Office,Jan. 1969.

16McCarthy, Phdip J.: Pseudo-replication: HaIf samples. Rev.International Statkt. Inst., 37:239-264, 1969.

17Kish, Leslie, and Frankel,, Martin: Balanced repeatedreplications for analytical statistics. Proc. Sot. Statist. Sec. Am.Statirt. A., 1968, 2-10.

18Kish, Leslie, and Frankel, Martin R.: Balanced repeatedreplication for standard errors. J. Arm Statist. A., 65:1071-1094,1970.

19Levy, Paul S.: A comparison of balanced half-samplereplication and jackknife estimatom of the variances of ratioestimates in complex sampling with two primary sampling unitsper stratum. Rockville, Md. National Center for Health Statistics,1971. (Dittoed report.)

20Simmons, Walt R., and Bean, Judy A.: Impact of designand estimation components on inference. New Developments inSurvey Sampling. (N. L. Johnson and H. Smith, Jr., eds.) NewYork. Wdey-Interscience, 1969, pp. 601-628.

21 Keyfitz, Nathan: Estimates of sampling variance wheretwo units are selected from each stratum. j. Am. Statist. A.,52(280):503-510, Dec. 1957.

22Hansen, M. H., Hurwitz, W. N., and Madow, W. G.: SampleSuruey Methods and Theory. Vol. 2. New York. John W]ley &Sons, Inc., 1953.

23Tepping, Benjamin J.: Variance estimation in complexsurveys. Proc. Sot. Statirt. Sec. Am. Statirt. A., 1968, 11-18.

24National Center for Health Statistics: Estimaticm andsampling variance in the Health Interview Survey. Vital andHealth Statistics. Series 2, No. 38. DHEW Pub. No. (HRA)74-1288. Health Resources Administration. Washington. U.S.Government Printing Office, June 1970.

25Shapiro, Gary M.: Keyfitz method of estimating varianceand its application to the Current Population Survey. Washing-ton. U.S. Bureau of the Census, 1966. (Dittoed report.)

26 Waltman, Henry: Variance estimation formulas to be usedin computing totsd NSR PSU variances for the 449-area design.Washington. U.S. Bureau of the Census, 1968. (Dittoed report.)

27 Fellegi, I. P., and Gray, G. B.: Sampling errors in periodicsurveys. Pro.. Sot. Statist. Sec. Am. Statist. A., 1971, 28-35.

28 Frankel, Martin R.: Inference from Suroey Samples: AnEmpirical Investigation. Ann Arbor. Institute for Social Re-search, University of Michigan, 1971.

29U.5. National Health Survey: The statistical design of theHealth Household-Interview Survey. Hea[th StatsMics. PHS Pub.No. 584-A2. Public Health Service. Washington. U.S. Govern-ment Printing Office, July 1958.

30Bem, Judy A.: Behatior of replication and linearizationvariance estimators for complex multistage probability samples.Unpublished doctoral dissertation. University of Texas, 1973.

33

APPENDIX I

EXTENT OF BIASNESS OF

Introduction

Another objective of the study was todetermine the extent of biasness of two ratioestimators. The formulas used and the resultsfollow.

Estimators

The two estimators of a statistic follow:

1. R ~–poststratification at universe level. Theformula is given in the section Estimatorsand Vaiiance Estimators.

2. R2 –poststratification at region level

‘:NC,2 + X;w,2R2 =

Y

where

ENC = East and North Central regions,

SW = South and West regions,

Rz = poststratified ratio estimate of the pop-ulation parameter R,

=E YENC,.‘:NC,2 ‘~N C,a t@1

)’ENC,a‘

TWO RATIO ESTIMATORS

x~Nc,a = the sample estimate for class a in theENC Region,

yEI’Jc,a = the population size in class a in theENC Region,

=E YSW,.%i’W,2 U=l‘iW,a t

Ysw,a

‘kW,a = the sample estimate far class a in theSW Region,

y~w,a = the population size in class a in theSW Region,

Y; W,a = the sample estimate of population sizefor class a in the SW Region.

Comparison of Two Ratio Estimators

Ratio estimators are commonly employed incomplex multistage sample surveys andusually contain the features of unequal sam-pling fractions and poststratification. Thedistributions of these ratio estimators are un-known. In this investigation, two such ratioestimators were employed in order to study theextent of their biasness.

Each estimation procedure included inflationby the reciprocal of the selection probabilityand poststratification by 24 ethnic-sex-ageclasses. This probability of selection is theproduct of the probabilities of selection fromeach step of selection: PSU and segment. The

34

level at which poststratification wzy performeddifferentiates the estimators. Thee R ~ estimatorwas adjusted to a universe level; R2 was adjustedto the distribution of subgroups within the ENCRegion and within the SW Region.

Ratio estimators are biased estimators. Theextent of this bias has been studied boththeoretically and empirically. Since these ratioestimates were calculated for each of fllve varia-bles for 900 samples for three sample sizes,sample empirical evidence was available to in-vestigate the magnitude of the bias of thepoststratified ratio estimators employed here.The universe was completely specified, so theparameter R was known. A measure of the biasis

design I, with the exception of jhe variableproportion seeing a physician, R ~ has thesmaHer value. For the other two designs, R ~does not compare as well, and in fact, onlyexhibits the smaller bias for the variable propor-tion seeing a physician in design II and forhospital days in 111.

The relative bias of the estimators used foranother evaluation of the estimators is definedto be

Ri-Rrelative bias (fii) = ~

where

R = the population parameter,bias (Ri) = ii - R

fii = the sample mean of the estimates pro-duced by ratio estimator i,

900

= ~+ ~R~j, i=l,2j

where

R = parameter value,900

TL X R;, +1,2‘i = 900 i=l

Table II shows the values for the two esti-mates for each variable and design. The relativebias is always less than 1 percent for any of the

Table I shows that neither of the two esti-mators consistently had the smaller bias. In

Table 1. The bias of two ratio estimators Table 11. The relative bias of two ratio estimators

Variable VariableEstimatorEstimator

Design I Design I

-0.0006 -0.00150.0031 0.0014

-0.0029 -0.00400.0083 0.00780.0009 -0.0004

Design II

-0.0047 “ -0.01240.0049 0.0021

-0.0135 -0.01970.0097 0.00820.0059 -0.0035

Family income . . . . . . . . . . . . . . . . . . .Restricted activity days . . . . . . . . . . . .Physician visits . . . . . . . . . . . . . . . . . . .Hospital days . . . . . . . . . . . . . . . . . . . .Proportion seeing a physician . . . . . . . .

Family income . . . . . . . . . . . . . . . . . .Restricted activity days . . . . . . . . . . . .Physician visits . . . . . . . . . . . . . . . . . .Hospital days . . . . . . . . . . . . . . . . . . . .Proportion seeing a physician . . . . . . .

Design II

0.0055-0.0008-0.0119

0.00160.0035

Desi

0.0047-0.0004-0.0112

0.00090.0041

1 Ill

Family income . . . . . . . . . . . . . . . . . . .Restricted activity days . . . . . . . . . . . .Physician visits . . . . . . . . . . . . . . . . . . .Hospital days . . . . . . . . . . . . . . . . . . . .Proportion seeing a physician . . . . . . . .

Family income . . . . . . . . . . . . . . . . . . .Restricted activity days . . . . . . . . . . . .Physician visits . . . . . . . . . . . . . . . . . . .Hospital days . . . . . . . . . . . . . . . . . . . .Proportion seeing a physician . . . . . . . .

0.0007-0.0006-0.0025

0.00150.0006

Desi!

0.0006-0.0CS03-0.0Q240.00080.0006

I Ill

Family income . . . . . . . . . . . . . . . . . . .Restricted activity days . . . . . . . . . . . .Physician visits . . . . . . . . . . . . . . . . . . .Hospital days . . . . . . . . . . . . . . . . . . . .Proportion seeing a physician . . . . . . . .

-0.0082-0.0012-0.0210

0.0040

-0.0064-0.0010-0.0198

0.0045

Family income . . . . . . . . . . . . . . . . . . .Restricted activity days . . . . . . . . . . . .Physician visits . . . . . . . . . . . . . . . . . . .Hospital days . . . . . . . . . . . . . . . . . . . .Proportion seeing a physician . . . . . . . .

-0.0010-0.0008-0.0045

0.0038-0.0003

-0.0008-0.0007-0.0042

0.00430.00010.0021 I 0.0007

35

Table I I 1. Distribution of differential bias of ;I and Al as a fraction

Lessthan –.30 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

–.30to–.25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.25to-.20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

–.20to–.15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.15to–.lo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

–.loto–.05 . . . . . . . . . . . . ’. . . . . . . . . . . . . . . . . . . . .

-.05to-.oo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.ooto .05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.05to.lo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.loto .15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.15to .20 . . . . ..’...... . . . . . . . . . . . . . . . . . . . . . . .

.20to .25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.25to .30 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Greaterthan ,30 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Lessthan-.3O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.30to–.25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.25to-.20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.20to–.15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.15to-.lo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.loto–.05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.05to.oo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.ooto .05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.05to.lo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .,

.loto .15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.15to .20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.20to .25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.25to .30 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Greaterthan .30 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Lessthan -.30 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.30 to-.25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.25to-.20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

–.20to–.15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

–.15to-.lo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.loto–.05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-.05to.oo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.ooto .05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.05to.lo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.loto .15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.15to .20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.20to .25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ,.

.25to .30 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Greaterthan ,30 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .,

theestimated standard error of ~ for replication method 1

Variable

RestrictedFamily Physician Hospitel

Proportion

activityincome

seeing a

daysvisits days

physician

4

4

5

18

45

142

345

249

66

17

2

3

0

0

0

0

0

0

1

24

327

438

100

7

3

0

0

0

0

0

0

0

0

4

231

578

82

5

0

0

0

0

4

6

8

29

49

139

265

257

98

30

12

2

1

0

0

0

0

2

9

89

333

363

93

7

4

0

0

0

0

0

0

0

0

30

394

437

39

0

0

0

0

0

Design I

5

5

2

13

64

129

304

267

86

20

3

1

1

0

Design II

o003

7

42

314

423

100

11

0

0

0

0

Design Ill

o00006

270

577

47

0

0

0

0

0

7

3

9

10

42

115

291

264

113

27

10

6

3

0

1

0

1

3

13

78

318

350

111

17

7

1

0

0

0

0

0

0

1

25

328

467

68

9

1

0

0

0

18

12

22

40

97

178

254

i 73

74

22

6

1

2

1

0

0

0

2

1

57

344

369

115

10

2

0

0

0

0

0

0

0

1

13

310

486

82

7

1

0

0

0

36

estimates and only approaches 1 percent for R 1in design I for hospital days. As the sample sizeincreases, the biases should decrease. However,here the relative biases are very small and exhibitno consistent decrease. Also, the relative biasesare not always positive or negative, but vary.Neither of the two estimators employed yieldsthe smallest value consistently across the statis-tics and designs.

McCarthyl 5 developed an analytical tech-nique for checking on the bias of the ratioestimate using a byproduct of the replicationvariance estimation method. This method wasemployed to provide a further study of the biasof RI, since it is the common estimator incomplex sample surveys. The replicate estimateR’l~ and the complement estimate R *1,aandthei’r average 71 are each estimates of R securedfrom half the data rather than the entire sample.Since the bias decreases with increasing samplesize, ~he estimate 71 should have greater biasth+ RI.Another measure of the absolute biasof R ~ is the differential bias between 71 and R 1.If values R ~ and 71 for a set of sample data arenearly the same, it is reasonable to beIieve thedifferential bias is small. Thus, the absolute biasof either estimator is also smalI.

This approach was used to examine the biasof the ratio estimator R 1. The half-sampleestimates R‘1~ computed in the VAR(R lEZ)scheme, and the estimates R *1,calculated in theVAR(R 1C) scheme, and the average of the twoestimates across all the half-samples were used.The differential bias of ?1 and R ~wasexpressed

as a fraction of the estimated standard error ofi=l. The formula is

7.. - RI

J VAR(~ )

where

( \F1=+j:R;a+&a,;Ci=’ ~=1

~~ = the ath half-sample estimate of RR,computed in the R lkl variancescheme,

R~a = the ath complement half-sampleestimate of R computed in theR lC variance scheme, and

As can be seen in table III, biases appear to besmalI and to decrease with sample size.

These data indicate that the bias of the twopoststratified ratio estimators used in this inves-tigation is small and is not a factor to beconcerned about. In the properties studied, noneof the two-ratio estimators clearly dominated.However, R ~, with the exc:ption of familyincome, is perhaps better than R2.

000

37

Introduction

The purpose ofwas to ascertainestimator can be

A SIMPLER

APPENDIX II

VARIANCE ESTIMATOR

this phase of the investigationwhether a simpler variance

used as an approximation tothe correct replication and lineariza~ion varianceestimates of the ratio estimate R 1. Anotheraspect explored was to examine an alternativeratio estimator for which correct variance esti-mates are easier to compute.

Variance Estimators for Estimator 1

The following two replication methqds areapproximate estimates of the variance of R 1.

1. VAR(R2H), VAR(R2C), and VAR(R2S)

Here, instead of calculating a new poststratifica-tion factor for each half-sample and each com-plement replicate, the multiplication factor de-rived in the parent sample was used. Again, theorthogonal pattern in table 4 was used, whichmeans that these replicates were the same as theones in the variance schemes R lH, R 1C, andR 1S. The formulas are

20

VAR(R2H) = +~(R;,& - RI )2II

20

VAR(R 2C)= & ~l(R.fa - RI )2a’

VAR(R2S)=+ [VAR(R2H)+VAR.(R2C)]”

where

%2= the ath replicate estimate of the param-eter with parent poststratification fac-tors,

R.$a = the cxth complement estimate with par-ent poststratification factors.

2. VAR(R2H), VAR(R3C), and VAR(R3S)

The variance estimates for the region ratioestimates were calculated and then combined togive another approximation to the variance ofRI. When the number of strata in a sampledesign is too large, it may not be feasible toconsider employing an orthogonal pattern ofsize k X k, where k is the necessary :number ofhalf-samples required. An illustration of thismight be a design which has, say, 400 strata. Inthis case, an approximation or alternative proce-dure could be substituted for the correctformula. By computing the estimate in thismanner, two orthogonal patterns, one for theENC Region and another for the SW Region,each requiring less than k replicates, can beemployed rather than ‘a single pattern of size k Xk.

The ratio estimator R 1 can be =pr~ssed =

‘ENC,l =YENC~ENC,I + Ysw&w,I

Yand

38

where

‘#NC,l‘ENC,l = YENC

$X;NC .?‘ Ya.

YENC,where the sum is over

PSU’S in the ENC Region,

&!Nc, ~ = the poststratified ratio estimate ofhealth characteristic x for the ENCRegion,

4’W ,1%w,l – ysw

2xiW,a~~=1. ,where the sum is over

Ysw

PSU’S in the SW Region,

4’W ,1 = the poststratified ratio estimate ofhealth characteristic x for the SWRegion.

Then the variance estimator is

VAR(fil) =Y#NCv~R(fiENC, 1)+Ygwv~R(&w,l)

Y*

The two orthogonal patterns used are in tablesIV and V. Another feature of this approxima-tion is the level at which poststratification wasdone for ~he half-sample estimators. Since theestimator R 1 is poststratified at a universe leveLthese factors should be computed within eachhalf-sample at a universe level, in order for thehalf-sample estimates to be equivalent. However,this cannot be done when two different-sizedpatterns are used. The solution was to apply theparent factors. Formulas for these variances are

1 f(R~Nc .- fiENc,I )2VAR(~ENc, I :~) = fia=l ,

1 &R#NC,. - ‘ENC,1)2vAR(kENc,l :C) = fi~=l

VAR(fiENc,l:S)= ~[VAR(kENc,l :~) + VJW~ENC,I :C)l

ijR&v,. - &l,,)2VAR(R5W,1:C) ‘&=l

VAR(&w,l :s) = ~[VAR(&w,l :W + vAR(~sw,l :C)]

VAR(R3H) = y~Nc [VAR(kE~c,l :~~)] + Y;w [vAR(~sw,I :~

VAR(R3C) = y~Nc [vAR(~ENc,l :C)l +Y~w [vAR(~sw,I :Cl

39

Table IV. Orthogonal pettern for ENC Region variance estimates

Strata in the ENC RegionReplicate

1 2 34 5 9 10 11 12 13

1 ......2 . . . . . .3 . . . . . .4 . . . . . .5 . . . . . .6 . . . . . .7 . . . . . .8 . . . . . .9 . . . . . .

lo . . . . . .11 ......12, . . . . .

?- +-- +-++--- - ++-- +-i-+--- --1--

+++---+- +

● F-F-I----?=- + +

++---+-+- +-

+----+-++ - -+--- +- i-+-+ +-- +-4-+--++ +

+-- +-+-+.+ +-+-++-+++ --

++--4-++- --

--- --- -- --

1 ,~+~~denotes +1 and “-” denotes -1. i

Table V. Orthogonal pattern for SW Region variance estimates

Strata in the SW RegionReplicate

6 7 8 ’14 15 16 17 18 19

1 ........2 . . . . . . . .3 . . . . . . . .4 . . ...4..5 . . . . . . . .6 . . . . . . . .7 . . . . . . . .8 . . . . . . . .

9 . . . . . . . .lo . . . . . . . .11 . . . . . . . .12 . . . . . . . .

+---+++- -+--+----4- + i-

++ -+--- + +

++ -+--- +-+- ++-+-- --i- i--++--+ --+-l-+-++- +-

+++-+-i- +-- +++-++---- ++ +-+ ++---+++ - i---- --- ---

“’+r’denotes+l and’’-’’ denoles-l

and

VAR(R3S) = ~[VAk(R3H) + VAR(R3C)]

where

R’ENC,a = the cxth replicate ratio estimate ofhealth characteristic x for the ENCRegion with parent sample post-stratification factors used,

R’SW,a = the ath replicate ratio estimate ofhealth characteristic x for the SWRegion with parent sample post-stratification factors applied.

Variance Estimators for Estimator 2

The variances for the estimates ~2 wereestimated by the correct replicatilcm methodonly.

For R2, the variance estimators areVAR(R4H), VAR(R4C), and VAR(R4S). Thisscheme consists of estimating the va~iances forthe regional estimates RE N~,2 and R~ w,2 andi~en combining them for a variance estimate ofR2. This process is appropriate since, for theparent sample estimate, poststratification was atthe regional level. For the half-sample and thecomplement half-sample estimates, the regionalpoststratification factors were calculated eachtime. The orthogonal patterns in tables IV and Vwere used in producing the variances for

fiE N c,2 and&w, z. The formulas are

-~ = YENCRENC,2 + Y~wRsw,2

2 Y

where

RENCz = the ratio estimate for the ENC Re-9

gion with poststratification at a re-gional level,

Rswz = the SW Region ratio estimate with9

poststratification at the regionallevel,

VAR(&2 ) =

Y~Nc [VAR(.ENC,J1 + Ysw [vAR@SW,2)~

Y2

1SR’ ‘ENC,2)2,‘AR(RENC,2 ‘H) = ~Q=l( ENC,. - R

40

-& R* -~ENC,,)2‘AR(RENC,2: c, = Iza=l ( ENC,ct

VAR(fi~NC,2:S)=~[VAR(fiENc,2:H)+vAR(~ENc,2:c)l

VAR(RSW,2:S)=;[VAR(fisw,2:H)+ v@~sw,2: c)]

{VAR.(A!4H)= y$Nc[VAlZ(~Nc,2:.H)]+Y~w [VAR(RSW,2:H)I}-$

VAR(R4C)= {Y:NC[vAR(&J@ c)] +Y:w [vAR@sw,z’ c)]};

vAR(R4.s)=~[vAR(R@ +vAR(R4c)]L,

where

R’ENC,CY = the ath half-sampIe ratio estimatefor the ENC Region adjusted to re-gion leveI,

R’Sw,ci = the ath half-sample ratio estimatefor the SW Region adjusted to re-gion level.

Approximate Variance Estimators

Investigators would like to obtain estimates ofvariance by simple methods. The results of twoapproximate methods thought to be solutions tothis problem are discussed in this report.g Oneof the methods, VAR(R 21Y), has been used bothby the NCHS and the University of MichiganSRC, The discussion is based on the results ofthe third replication form.

For this evaluation, bias, variance, and meansquare error were computed. The definitions of

gTW~more ~pproximations of the correct VzUkiC;oCStimdCS

were also investigated. These results are given in Bean.

these measures are the same as given in theResults section. Inspection of table VI shows thebiases for VAR(R 3S) for every variable decreas-ing with increasing sample size. The VAR(R2S)method displays a different pattern. For familyincome and restricted activity days, the biases ofVAR(R 2S) reduce as the sample size increases;for the other three variables, the values increase,going from design I to design II, but then dropto their lowest in design III. VAR(R2S) dpessometimes underestimate the variance of R 1.This occurs in design I for the variable propor-tion seeing a physician and in designs H and IIIfor the variable restricted activity days. Regard-less that the biases for three variables increase,going from design I to design II, the R2Smethod behaves better than the R 3S approxima-tion. It always has the lowest bias.

Both the quantities variance and mean squareerror in tables VII and VIII indicate that theR2S method outperforms VAR(R 3S) withoutany question. This procedure always has thesmaller value of variance or mean square errorjust as in the base of bias.

A possible explanation for why VAR(R3S) isnot a good approximate variance estimator isoffered. VAR(R 3S) is a variance estimator for

41

Table VI. The bias of the approximate variance estimators of the sample estimator ~1

Variance estimator

VAR(R2S) . . . . . . . . . . . . . . . . . . . .VAR(R3S) . . . . . . . . . . . . . . . . . . . .

VAR(R2S) . . . . . . . . . . . . . . . . . . . .VAR(R3S) . . . . . . . . . . . . . . . . . . . .

VAR(R2S) . . . . . . . . . . . . . . . . . . . .VAR(R3S) . . . . . . . . . . . . . . . . . . . .

Variable

Family Restricted Physician Hospital Proportion

income activity days visits days seeing aphysician

2.469X 10”1.222X 10s

1.219X 1045.893X 10’

Design 1

2.164X 10-’ I 3.677X 10-35.885X 10-’ 3.498X 10-2

Designll

–2.705X 10-= I 3.984x 10-31.334X 10-1 1.884X 10-2

Design Ill

1.621X 10-33.647X 10-3

1.656X 10-32.489X 10-3

-1.078 X 10-56.536 X 10 “

2.027 X 10-s3.344 x 10-4

4.173 x 103 I -2.912 X 10-3 I 1.097 x 10-3 I 3.593 x 1o+ I 6.765 X 10-$2.461 X 104 6.099 X 10-2 8,263 X 10-3 7.021 X 10-4 1.437X 10+

Table VI 1. The variance of the approximate variance estimators of the sample estimator ~j

Variance estimator

VAR(R2S) . . . . . . . . . . . . . . . . . . . .VAR(R3S) . . . . . . . . . . . . . . . . . . . .

VAR(R2S) . . . . . . . . . . . . . . . . . . . .VAR(R3S) . . . . . . . . . . . . . . . . . . . .

VAR(R2S) . . . . . . . . . . . . . . . . . . . .VA R(R3S) . . . . . . . . . . . . . . . . . . .

Variable

Family Restricted Physician HospitalProportion

income activity daysseeing a

visits daysphysician

1.594X 1092.678X 10]0

4.800X 10s4.633X 109

Design I

0.190X 10I

2.989X 10-3

0.244X 10 5.020X 10-3

Design I I

5.320X 10-’ 9.071X 10-45.901 x 10-1 1.418X 10-3

5.298 X 10 ‘45.904 x 10-4

8.409 X 10 ‘s8.606 x 10-’

1.541 x 10-*9.937 x 10-7

3.729 X 10-’1.810x 10-’

Design I I 1

1.364 X 10s 1,495X 10-1 8.956 X 10-s ‘ 1.373X 10-’ 1.I09X 10-91. I06X 10$’ 1.594X 10-’ 1.807 X 10-4 1.447X 10-5 4.402 X 10-8

R ~, the parent sample estimator, obtained froma weighted average of variance estimators of

RENC,l and Rsw,l. The estimating equationpoststratifies the estimators to universe levelsand, thus, these subuniverse estimators are ad-justed to an overall distribution of ethnic-sex-age, rather than to subuniverse distribution.Questions that should be asked are Wha~ is themagnitude of the bias in RENC,l and Rsw,l ?and If the bias is not negligible, what is the

effect on the variance estimator? These ques-tions cannot be directly answered here becausethe biases of the subuniverse estimators were notcalculated. However, VAR(R 4S) is the appropri-ate weighted variance estimator for a ratioestimator poststratified to regional levels, and itsperformance can be studied.

The analysis was based on comparison of bias,variance, and mean square error. The formulasfor these measures follow:

42

bias [VAR(R4S)] = VAR(R4S) - s?R2

VAR[VAR(R4S)] =

@ {[vWR45)]j - VAR(R4S)}2

MSE[VAR(R4S)] = VAR[VAR(R4S)]

+ bias’ ~AR(R4S)]

where

900

J-z vAR(R4s)]j.VAR(R4S) = 900 j=~ [

The biases, variances, and mean square errorsfor each variable and design are given in tableIX. For design I, the R4S method has a range of4.813 X 10-5 for proportion seeing a physicianto 9.182 X 10-3 for family income. In designs IIand III, the smallest value is again for proportionseeing a physician and the largest bias for familyincome. Unlike the appropriate variance esti-mator VAR(R 1S) for the ratio estimator R 1, thebiases here decrease with increasing sample size,which is a desirable property.

The range for the variance of the varianceestimator R 4S is 1.191 X 109 for family income

to 2.077 X 10-8 for proportion seeing aphysician in design I; in design II, the range is3.297 X 108 to 3.748 X 10-9; and in design III,it is 1.008 X 108 for the variable family incometo 1.072 X 10-9 for proportion seeing aphysician. The mean square errors exhibit asimilar pattern.

Another consideration is the distribution ofthe sample estimate minus its expected valuedivided by its estimated standard error. Theratio computed was

R2 - E(~2)

Wmm

where

900-E(R2) = i2 ‘&~R2ja

The proportion of times these ratios fell withinstated limits of the normal distribution aredisplayed in table X. Only 12 times out of apossible 60 did the empirical proportions forVAR(R4S) surpass the normal values. The mag-nitudes of VAR(R4S)’S deviations are compara-ble to the amounts for VAR(R 1S) with nodiscernible tendency for the differences to be-come smaller across the designs. The total errorof the variance method is satisfactory, but there

Table VI 11. The mean square error of the approximate variance estimators of the sample estimator ;,—,-

i Variable

Variance astimatorFamily Restricted Physician Hospital

Proportion

incomeseeing a

activity days visits daysphysician

I Design I

VAR(R2S) . . . . . . . . . . . . . . . . . . . . 2.204 x 109I

0.195X 10I

3.003 x 10-3

I

5.324 X 10-4

I

1.552X 10-$

VAR(R3S) . . . . . . . . . . . . . . . . . . . . 3.763 x 10’0 0.278 X 10 6.245 X 10 ‘3 6.037 X 10-4 1.421 X 10-’

Design I I

VAR(R2S) . . . . . . . . . . . . . . . . . . . . 6.289 X 10s I 5.328X 10-’ 9.229 X 10-4 I 8.684 X 10-’ I 4.140X 10-9

VAR(R3S) . . . . . . . . . . . . . . . . . . . . 8.IIOX 109 6.079 X 10-’ 1.774X 10-3 9.226 X 10-s 2.930 X 10-’

Design 11I

VAR(R2S) . . . . . . . . . . . . . . . . . . . . 1.539 x 108 1.495X 10-’ 9.320 X 10 “ 1.386X 10-5 1.155X 10-9

VAR(R3S) . . . . . . . . . . . . . . . . . . . . 1,713X 10’ 1,631 X 10-’ 2.490 X 10-’ 1.496X 10-s 6.470 X 10-8

43

Table IX. The bias, variance, and mean square error of R4S variance estimator of Rz

Variable

Family income . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Restricted activity days . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Physician visits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Hospital days . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Proportion seeingaphysician . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Family income . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Restricted activitydays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Physician visits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Hospital days . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Proportion seeingaphysician . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Family income . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Restricted activitydays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Physician visits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Hospital days . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Proportion seeingaphysician . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

are indications that the size of the bias is aproblem. Although it would be difficult to stateany general rules about drawing inferences whenthe stated confidence level could be in error ineither direction, constructing such intervals isconceivable.

The method VAR(R2S) can offer a savingsincomputer time, since the process of calculatingnew poststratification weights within each half-sample is eliminated. Because the comparablemeasures used show this method to be a goodapproximation, it is also compared with VAR(RIS). The biasof VAR(R2S) islessin thethreedesigns for restricted activity daysand in designsI and II for the variable hospital days. In thecase of variance, VAR(R2S) has the lower valuefor variables restricted activity days, hospitaldays, and proportion seeing a physician in d=iw

I and also for hospital days in design II. Themean square errors of VAR(R 2S) are lower forthe same designs and variables as the variances.

For a better understanding of the comparisonbetween the two methods, the relative biaseswithin a design are averaged over the variables.The relative bias is defined as

(

(

i

T

I

Bias I Variance I Mean squara

error

9.182X 103

3.944 x 10-’4.517 x 10-3

3.398 X 10-3

4.813 X 10-s

2.425 X 103

4.057 x 10-’

3.626 X 10-3

2.759 X 10-3

2.476 X 10-$

-8.598 X 102

-1.357 x 10-21.214X 10-3

3.161 X 10-’

5.098 X 10-6

Design I

1.191 x 1090.208 x 103.173X 10-31.961 X 10-32.077 X 10-s

Dasign I I

3.297 X 10“5.433 x 10-’9.455 x 10-”2.095 X 10 ‘43.748 X 10-9

Design I I I

1.257 X 1090.224 X 10:3.193 x 10-3‘i.973 x 10-32.308 x 10-s

3.356 x 108!5.450 X 10-*9.587 X 10-4:2.171 X 10-44.362 X 10-9

1.008 X 10a 1.015X 1081.405X 10-1 ‘1.407X 10-’8.227 X 10 ‘s 8.374 x 10-*1.594X 10-s 1.604X 10-s1.072 X 10-3 ‘1,098 X 10-9

V-4R(R 1S) -<RI

relative bias [VAR(R 1S)] = ~ -.‘1

VAR(RI!S) - #‘1

relative bias [VAR(R2S)] = ~-‘1

The. average relative biases for VAR(R 1S) forthe three designs are 0.0782, 0.0656, and0.0363 as compared to 0.0862, 0.1231, and0.3150, the averages for VAR(R 2S). Thus, theseaverages show that as the sample size increases,the magnitude of the bias in VAR(R 2S) in-:reases, whereas for VAR(R 1S) it decreases.

To complete this discussion, the proportionof times the ratio of the sample estimate minusits expected value divided by the R 2S estimatedvariance is within certain limits is in table XI.rhe observation made is that 16 out of 60empirical proportions are larger than the sameproportions for the normal distribution. Con-

44

Table X. Proportion of times the sample estimate minus its expected value divided by R4S estimate of standard error is within the

Variable

Family income .Restricted activ-

ity days . . .Physician visits .Hospital days . .Proportion

seeing aphysician . .

Family income .Restricted activ-

ity days . . .Physician visits .Hospital days . .Proportion

seeing a

physician . .

Family income .Restricted activ-

ity days . . .Physician visits .Hospital days . .Proportion

seaing aphysician . .

stated limits

Limits

*1.000 -1.000,0 0, 1.000 *I .645 -1.645,0 0, 1.645 21.860 -1.960, 0 0,1.960 *2.576

Design I

0.6878

0.67330.65780.6722

0.6911

0.6922

0.65560.68780.6789

0.7011

0.6689

0.65780.68670.6856

0.6822

0.3478

0.33000.31440.3211

0.3522

0.3489

0.30440.34000.3189

0.3644

0.3211

0.33560.33440.3367

0.3378

0.3400

0.34330.34330.3611

0.3389

0.3433

0.35110.34780.3600

0.3367

0.3478

0.32220.35220.3489

0.3444

0.8867

0.89670.87220.8478

0.9011

0.8833

0.88560.89670.8833

0.9167

0.8667

0.87000.90330.8811

0.8856

0.4533

0.44560.42890.4222

0.4456

0.4433

Q.46110.44330.4256

0.4556

Design I I

0.4333 0.4500

0.4256 0.46000.4578 0.43890.4356 Q.4478

0.4678 0.4489

Design III

Q.4167

0.44560.44670.4456

0.4378

0.4500

0.42440.45870.4356

0.4478

0.9422

0.84440.93660.9033

0.8478

0.9344

0.93440.94220.9367

0.9544

0.9233

0.92560.84440.9278

0.9322

0.4766

0.47670.47000.4644

0.4678

0.4633

0.45670.48330.4711

ID.4867

0.4456

0.47330.46890.4744

0.4588

0.4667

0.46780.46560.4389

0.4800

0.4711

0.47780.45890.4656

0.4678

0.4778

0.45220.47580.4533

0.4733

0.9844

0.97890.98000.9556

0.9811

0.9778

0.97780.98000.9766

0.9856

0.9800

0.97440.98440.9667

0.9867

trustingly, the VAR(R 1S) estimates exceeded deviations are smaller for VAR(R 2S). Thesethe normal proportions only six times. This average differences for each statistic across thecomparison is done because, ideally, when an five limits and three designs are +.0145, -.0197,analyst uses such an approximation, he would -.0081, -.0015, and - .0097. The average dif-like to know in which direction he could be ferences of VAR(R 1S) are -.0168, -.0200, andmaking an error. A somewhat unexpected -.134. This body of empirical evidence supportsobservation is that even though the differences the proposal of using VAR(R2S) as an approxi-are not in the same direction, the average mation.

45

Table X 1. Proportion of times the sample estimate minus its expected value divided by /?2S estimate of standard error is within the

Variable

Family income .

Restricted activ-

ity days . .

Physician visits .

Hospital days . .

Proportion

seeing a

physician . .

Family income .

Restricted activ-

ity days . . .

Physician visits .

Hospital days .

Proportion

seeing a

physician . .

Family income .

Restricted activ-

ity days . . .

Physician visits .

Hospital days . .

Proportion

seeing a

physician .

stated limits

Limits

*I .000 –1.000,0 0,1.000 +1.645 -1.645, 0 0, 1.645 * 1.960 -1,960,0 0, 1.960 *2.576

Design I

0.7433

0.6689

0.6567

0.6689

0.6489

0.7256

0.6433

0.6811

0.6722

0,6900

0.7033

0.6533

0.7011

0.6933

0.7011

0.3811

0.3289

0.3100

0.3022

0.3400

0.3711

0.3000

0.3311

0.3267

0.3533

0.3378

0.3344

0.3411

0.3500

0.3556

0.3622

0.3400

0.3467

0.3667

0.3089

0.3544

0.3433

0.3500

0.3456

0.3367

0.3656

0.3189

0.3600

0.3433

0.3456

0.9122

0.8844

0.8744

0.8644

0.8544

0.9133

0.8878

0.9011

0.8800

0.9144

0.4644

0.4367

0.4333

0.4200

0.4311

0.4478

0.4478

0.4411

0.4444

0.4233

Design I I

0.4611

0.4411

0.4522

0.4389

0.4622

0.4522

0.4467

0.4489

0.4411

0.4522

0.9544

0.9333

0.9244

0.9200

0.9256

0.9500

0.9333

0.9378

0.9344

0.9522

Design II I

0.9011 I 0.4344 0.4667 0.9489

0.8744

0.9056

0.8889

0.4478

0.4433

0.4533

0.4267 0.9289

0.4622 0.9489

0.4356 0.9322

0.8822 0.4400 0.4422 0.9300

0.4822

0.4656

0.4644

0.4556

0.4667

0.4800

0.4611

0.4744

0.4767

0.4800

0.4722

0.4678

0.4600

0.4644

0.4589

0.4700

0.4722

0.4633

0.4578

0.4722

0.4611 I 0.4878

0.4733

0.4700

0.4789

0.4556

0.4789

0.4533

0.4589 0.4711

0.9933

0.9767

0.9756

0.9589

0.9833

0.9844

0.9733

0.9853

0.9756

0.9833

0.9900

0.9767

0.9833

0,9689

0.9844

000

46 * U. S. GOVEIUJMENT PRSNTLSG OFFICE :1975 684-528/37

series 1.

Series 2.

Sm”es 3.

Series 4.

Series 10.

Sen”es11.

Sm”es 12.

Sw”es 13.

Series 14.

Series 20.

Series21.

Sen”es 22.

-., -. . . . . . -. -.-— .—— —VITAL ANb tltALlll STATISTICS PUBLICATION SERIES

Formerly Public HaaJth Sewvica Publication No. 1000

Progmms and collection procedures.— Reports which describe the general pragrams of the NationalCenter for Health Statistics and its offkes and divisions, &ta collection methods used, definitions,and other material necessary for understanding the data.

~ta evaluation and methods research. —Studies of new statistical methodology including: experi-mental tests of new survey methods, studies of vital ststiatics collection methods, new analyticaltechniques, objective evaluations of reliability of collected data, contributions to statistical theory.

Analytical s$udies. —Reports presenting analytical or interpretive studies based on vital and healthstatistics, carryfng the analysis further than the expository types of reports in the other series.

Documents and committee reports. - Final reports of major committees concerned with vital andhealth statistics, and documents such as recommended model vital registration laws and revfsedbirth and dqath certificates.

Dato from the Health Interw”ew Survev.— Statistics on illness, accidental injuries, disability, useof hospital, medical, dental, and other services, and other health-related topics, based on datacollected in a continuing national household interview survey.

Data from the Haulth Ezaminatwn Survey. —Data from direct examination, testing, and measure-ment of nafional samples of the civilian, noninstitutional population provide the basis for two typesof reportsi (1) estimates of the medically defined prevalence of specific diseases in the UnftedStates and the distributions of the population with respect to physical, physiological, and psycho-logical characteristics; and (2) analysis of relationships among the various measurements withoutreference to an explicit finite universe of persons.

lkta from the Institutional Population Surveys. —Statistics relating to the health characteristics ofpersons in institutions, and their medical, nursing, and personal care received, based on nationalsamples of establishments providfng these services and samples of the residenta or patienta.

Data from the Hospital Discharge Survey. —Statistics relating to discharged patients in short-stayhospitals, based on a sample of patient records in a national sample of hospitals.

Data an health resources: manpower and facilities. —Statistics on the numbers, geographic distri-bution, and characteristics of health resources including physician, dentists, nurses, other healthoccupations, hospitals, nursing homes, and outpatient facilities.

Data on mortality. —Various statistics on mortality other than as included in regular annual ormonthly reportS —special analyses by cause of death, age, and other demographic variables, also

geographic and time series analyses.

Data on nafality, mawiage, and divorce. —Various statistics on natslity, marriage, and divorceother than as included in regular annual or monthly reports-special analyses by demographicvariables, also geographic and time series analyses, studies of fertility.

Data from the National Natality and Mortality Survevs. — Statistics on characteristics of birthsand deaths not available from the vital records, based on sample surveys stemrnfng ftom theserecords, including such topics as mortality by socioeconomic class, hospital experience in the

r last year of life, medfcal care during pregnancy, health insurance coverage, etc.

For a list of titles of reports published in these series, write to: Office of InformationNational Center for Health StatisticsPublic Health Service, HRA

Rockville, Md 20852