Journal of Epidemiology and Community Health 1990; 44: 179-186

REVIEW ARTICLE: Research methods in epidemiology, V

Bias in case-control studies. A review

Jacek A Kopec, John M Esdaile

It has been widely accepted that one reason forinconsistent or contradictory results ofepidemiologic studies is bias. Therefore, anappreciation of potential sources of bias hasbecome a critical issue in epidemiology. A largenumber of different sources and possiblemechanisms of bias have been described but aconsistent terminology has not evolved. Differentauthors have used different terms when referringto essentially the same type of bias. For example,Kleinbaum et al' and Rothinan2 classify biasesinto selection bias, information bias andconfounding. Sackett offered a more detailedclassification, with five major types of samplingbias (eg, admission rate bias, membership bias)and four types of measurement bias (eg, diagnosticsuspicion bias, recall bias).3 Feinsteindistinguishes, inter alia, among susceptibility bias,performance bias, transfer bias and detection bias.4Last's Dictionary of epidemiology5 givesdefinitions of 26 biases but fails to mention manyof the terms proposed by other authors.

It is not our objective to provide acomprehensive list of biases, or to propose a new,improved taxonomy. Our purpose is to review themost common threats to validity in the design ofcase-control studies-commonly regarded asbeing particularly susceptible to bias-and todiscuss some useful strategies in dealing withthese problems. Certain issues will not beaddressed, notably methods of controllingconfounding, and the statistical analysis of case-control studies.

Greenland and Morgenstern, case-control studiesare distinguished from cohort studies by their useof "outcome-selective sampling".1o Miettinenrejects the case-control versus cohort dualityentirely. In his view, the "case-referent strategy"consists in: (1) identifying and classifying all ofthecases in the "study base"a, defined as the"population experience captured in a study", and(2) drawing a sample of the base to obtaininformation on the exposure. 12 A good illustrationof this concept is a "nested", or "within cohort"case-control study, which can be regarded as anefficient form of the cohort design, rather than anentirely different type of study.'3

It may be noted that the two aspects (or "axes")of the design, ie, order of measurement andmethod of sampling, are interdependent. Thereason for determining exposure status for allsubjects in most cohort studies is to obtain as validinformation as possible for the relatively fewcases-to-be. Therefore, in cohort studies allpotential cases are classified as exposed orunexposed. The logic behind the case-controlmethod is to make the design more efficient byclassifying only the actual cases, since informationabout the source population can be obtained bysampling. This, however, implies a particularorder of measurement (directionality), namelyascertaining the outcome first. Measuringexposure for all members of the study populationafter the cases have been ascertained isunnecessary and impractical, if the disease is rare.

Department ofEpidemiology andBiostatistics, McGiilUniversity, Montreal,CanadaJ A KopecDepartment ofMedicine, MontrealGeneral Hospital,McGill University,1650 Cedar Avenue,Montreal, CanadaH3G 1A4J M Esdaile

Correspondence to:Dr Esdaile

Accepted for publicationApril 1990

Defining a case-control studyTraditionally, a case-control (case-referent) studyhas been defined as an investigation into arelationship between a given disease and one ormore causal or preventive factors, in whichpersons selected because they have the disease(the cases) and suitably selected persons who donot have the disease (the controls) are compared interms of their exposure to the factors understudy.'7 Case-control studies are usuallyconsidered to be an alternative to cohort studies,from which they are distinguished by twofeatures: (1) sampling by disease as opposed tosampling by exposure, and (2) investigativemovement from effect to cause as opposed to fromcause to effect. 1 6-8 Recently, Kramer and Boivin9have postulated that the order of measurement("directionality"), rather than the method ofsampling, provides an "unconfounded" criterionfor distinguishing between the two designs. Thisapproach is not shared by other authors. For

The RCT paradigmThere seems to be current agreement that themethods of the randomised clinical trials mayserve as a useful paradigm in designing case-control studies. Yet for many clinicians, theconceptual links between the randomised clinicaltrial model and the case-control design may not beobvious. As an illustration, consider the famousstudy of cholera in London, conducted by JohnSnow in the mid-nineteenth century. 4 The studyhas often been referred to as a "naturalexperiment",2 and was practically as valid as arandomised controlled trial. However, its designresembled a modem, nested case-control ("case-cohort") study, rather than a randomisedcontrolled trial. Snow did not follow each memberof the study cohort individually and, in fact, did

a The term "study base" actually refers to population-time. 1Some authors also distinguish between a source population,which includes prevalent cases, and a population at risk. Theterm "study subjects" is used here to refer to the actual cases andcontrols.

on 6 May 2018 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.44.3.179 on 1 Septem

ber 1990. Dow

nloaded from

80acek A Kopec, John M Esdaile

not know the source of drinking water for eachhousehold. In order to ascertain the exposureretrospectively, he visited those families in whichfatal cases of cholera had occurred. AlthoughSnow obtained the overall number of householdssupplied by each water company directly fromtheir records, the proportion exposed in the studypopulation could have been estimated bymeasuring the quality ofwater in a random sampleof all households. This example illustrates thatunder optimal conditions, neither "backward"directionality nor sampling of the base make thecase-control method inherently less valid, orscientifically inferior to other types of research inepidemiology.The idea of using the experimental approach as

a paradigm for observational research wasstrongly advocated by Hill"5 and, more recently,by Horwitz and Feinstein.4 '6`9 Exactly how thisparadigm should be implemented remainsproblematic. According to Feinstein, special carein the conduct ofobservational research should bedevoted to the selection of subjects according tothe same eligibility criteria that are commonlyapplied in randomised controlled trials (eg,indications, contraindications), and the avoidanceof susceptibility bias (confounding bysusceptibility), detection bias in ascertaining thecases, protopathic bias (confusing cause andeffect) and "transfer bias".4 Esdaile andHorwitz20 used the randomised controlled trialframework to develop a list of principles thatcould be used as a check list for a scientific qualityofan observational study. Miettinen21 pointed outthat the traditional outlook in case-control studiesis incompatible with the randomised controlledtrial paradigm, where the concern is always tocompare exposed and unexposed, rather thancases and "non-cases". He also emphasised theneed to define appropriately not only exposurebut also "non-exposure" (the reference level ofexposure), and the importance of restricting thestudy population, in order to obtain a desireddistribution of exposure levels.The usefulness of the randomised controlled

trial paradigm in elucidating potential sources ofbias stems primarily from its conceptualsimplicity as a model for cause-effect research,and the relatively advanced knowledge of themethodological principles involved. In thefollowing sections of the paper we will apply thisparadigm to examine common sources of bias incase-control research. Three broad issues will bediscussed: (1) problems related to the definition ofthe study population; (2) measurement relatedbiases; and (3) bias due to non-participation.

Study populationPOPULATION BASED STUDIESExperimental populations are "fixed" cohorts,with membership defined automatically by thefact of "enrolment" into the study. In "case-control within cohort" studies the populationwhich gives rise to the cases is an observationalcohort, and is defined in a similar way. Most ofcase-control studies are conducted within socalled dynamic (open) populations, characterisedby a constant influx and outflow of subjects.Nevertheless, a dynamic population can be

precisely defined, eg, as the population of ageographical area, employees in an industrialplant etc, over a specified time period.Once the study population (study "base") has

been defined, the validity of the study depends onthe investigator's ability to obtain accurateinformation on the outcome for all its members (asin a randomised controlled trial), and informationon the exposure and other relevant variables(confounders, effect modifiers) for all of the cases,and a random sample of the base." It may benoted that in a randomised controlled trial,exposure information is available for allindividuals. The selection of controls (referents)in population based studies should follow theprinciples of random sampling. Various methodsof sampling have been proposed: (1) traditional"cumulative" sampling, at the end of theobservation period; (2) "case-base" or "case-cohort" sampling, performed at the start of theobservation period; (3) "incidence-density"sampling, conducted throughout the period offollow up.22 23 In order to adjust for confoundingfactors, the sampling is usually performed withinstrata, formed by the categories of potentialconfounders. A useful strategy, especially whenthe study population is "unstable", is to considertime as one of the stratification variables.24 25

HOSPITAL BASED STUDIESIn so called hospital based, or "case initiated"case-control studies,26 the investigator starts witha case series, usually selected from a medical carefacility, and compares it (in terms of exposurehistory) to an arbitrary control group, taken eitherfrom the hospital (patients with other diseases) orfrom the community (eg, neighbours or relativesof the cases). The cases and controls are usuallymatched for a number of potentially confoundingvariables. The source population in these studiesis often described vaguely as those subjects whoare "at risk" ofboth exposure and disease,7 but noeffort is made to delineate it with any rigour. As aresult, two types of problems arise. First, theinvestigator has to accept that the cases studiedare only a fraction of all cases that might be"eligible" for the study. A substantial andpotentially differential underdetection cannot beruled out, but its extent can hardly be evaluated.Second, there is no "roster" from which to select apopulation based control series. The resultantproblem ofpotential "selection" biasa can be seenas one of "incoherence" of the cases and controlswith respect to the population at risk theyrepresent.6 12

SECONDARY BASE

Conceptually, some improvement in the design ofcase initiated studies is possible by "turning theargument around" and defining the studypopulation secondarily, as all those subjects whowould have been included in the case series, hadthey developed the disease of interest." Full caseascertainment is then achieved simply bydefinition, and the previously mentionedselection bias that arises in the case series isapparently eliminated. However, since the

a We try to avoid the term "selection bias" because of itsambiguity and because it has been used in many differentcontexts.

180

on 6 May 2018 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.44.3.179 on 1 Septem

ber 1990. Dow

nloaded from

Bias in case-control studies. A review

membership in the population at risk is nowdefined only conditionally on a hypotheticalevent, it cannot be determined with certainty.Thus the problem of unbiased sampling ofcontrols remains unresolved.The choice of an appropriate control group in

case initiated (hospital based) studies has been thesubject of much controversy. One proposedsolution is to take cases ofother, carefully selecteddiseases, from the same "source" as the originalcase series. To avoid bias, the control disease mustnot be related to the exposure at issue, and shouldbe subject to exactly the same surveillance anddetection procedures as the disease of interest.For many diseases, and some common exposures,these conditions seem to be very difficult tosatisfy, and may actually conflict with eachother.7 27 28 Such factors as care seekingbehaviour, diagnostic accuracy, referral patternsor hospital catchment areas are likely to differ fordifferent diseases.4 Using a "diagnostic roster",(eg, a chest x ray file) as a source of both cases andcontrols does not seem to be an improvement.Although the "medical itinerary" of the cases (eg,cases of lung cancer) and the controls (other lungdiseases, etc) may be similar, they may often berelated to the same exposure studied (eg,smoking). The same objection applies to anextreme version of the above strategy, namely tocompare different histological subtypes of thesame disease.29 Another common option inselecting the control group is to use "communitycontrols". This approach can seldom betheoretically justified in case initiated studies.6 12The use of neighbours, siblings or friends of thecases has more to do with controlling confounding(through matching), than it does with unbiasedsampling of the base. The amount of bias willcertainly be study specific, and it is difficult toestimate. Only limited attempts to study theproblem empirically have been made.30 In onestudy in which friends ofthe cases have been usedas controls,3' 32 the "friendly control bias"apparently increased the observed effect. In otherstudies it may act in the opposite direction.As an efficient alternative to the case initiated

study some authors employed a "controlinitiated" design,33 34 whereby the investigatoruses the same control series more than once, tostudy different diseases and different exposures.This design has been used infrequently, and itdoes not offer any advantage over the caseinitiated design in terms of validity.26Because of these difficulties, many

epidemiologists (including the authors of thisreview) believe that validity ofthe control group ismost likely to be achieved, and easiest to evaluate,when the case series can be linked to anidentifiable source population. Unfortunately,population based studies are expensive because ofthe difficulty in ascertaining all the cases. Thevalidity of a hospital (registry, etc) control groupdepends on the similarity between the controldisease and the disease under investigation interms of all factors bearing on the appearance ofthe cases in a particular "source", and the lack ofassociation between the control disease and theexposure at issue. Empirical data concerningthese properties of different case-control-exposure combinations are scant.

ELIGIBILITY CRITERIAAn internally valid clinical trial can be carried outin a very restricted population, eg, amongvolunteers, or patients in a single hospital ward.Moreover, the choice of appropriate "eligibility"criteria may significantly improve the study'svalidity, for example by increasing compliance,minimising drop out rate, or enhancing thequality of measurement.35 The same principlesapply in case-control studies. Failure to applycertain restrictions on admissibility may seriouslylimit the investigator's ability to control bias.'62'The specific admission criteria will usually differconsiderably in studies of therapy (most of whichare experimental) and studies ofaetiology (most ofwhich are observational). For example, in case-control studies on side effects of drugs and othertherapeutic manoeuvres, it does not matterwhether the subject had or did not haveindications for the therapy used, except when agiven indication can be considered a confoundingvariable or an effect modifier.2' Similarly, it isirrelevant whether the subjects had the"opportunity" for exposure or not.36Measurement accuracy is usually a greaterproblem in case-control studies than inrandomised controlled trials and, as will bediscussed, restricting the source population is animportant strategy in dealing with this problem.In both types ofstudies it is reasonable to excludethose subjects whose risk of developing thedisease of interest is known to be zero.' 1 25

MeasurementMEASUREMENT OF OUTCOME (CASEASCERTAINMENT)By analogy with experimental research, it seemsuseful to approach the problem of caseidentification in case-control studies from ameasurement point of view. In a large clinicaltrial, one would have an elaborate system of activesurveillance for all subjects in the studypopulation, perhaps involving periodic follow upexaminations. The out migrants would be traced,the subjects' personal physicians might becontacted, and the records reviewed. Thediagnosis in each case would be verified byindependent, "blinded" reviewers, according tostandard, pre-established criteria, and using allpertinent information available.35

Unfortunately, maintaining such a system for along period of time, even in a moderately largepopulation, may be expensive and difficultlogistically, as the experience of some clinicaltrials and observational cohort studies suggest.The alternative approach, commonly employed incase-control studies of rare diseases, is to replacethese costly activities with routine recording ofcontacts between patients and the health system(including deaths) as the primary source of dataon the outcome. The measurement (caseidentification) thus becomes a multistage process,largely beyond the investigator's control. For acase to be properly identified, the followingactivities must take place: (1) a person mustreceive medical attention; (2) he/she must beproperly diagnosed as having the disease ofinterest; and (3) the record of the diagnosis mustappear in the "source" of the case series (eg, a

181

on 6 May 2018 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.44.3.179 on 1 Septem

ber 1990. Dow

nloaded from

2acek A Kopec, Jrohn M Esdaile

registry, a hospital, etc). The remaining membersof the base are, by default, regarded as "non-cases". Since each missed case represents aninstance ofmeasurement error (misclassification),it becomes clear that the objective should alwaysbe to ascertain all incident cases that occur in thestudy population. For the purpose of analysis, arandom sample of the cases may sometimes besufficient. In some studies there will also bepersons without the disease, erroneouslydiagnosed as cases ("false positives").Misclassification of subjects in terms of theirdisease status, even if "non-differential" withrespect to the exposure, will usually produce abias toward the null in the estimate of causaleffect. (An important exception from this rule willbe discussed later.) Differential misclassificationmay produce bias in either direction.The objective of complete ascertainment of

cases in the study base may be unrealistic. Formany conditions the accuracy of medicaldiagnosis is imperfect. Increasingly, sophisticatedand specialised care is necessary to make thediagnosis. Even for conditions that almostinvariably lead to serious symptoms and hospitaladmission, such as many cancers, thecompleteness of reporting seldom exceeds 95%,and may be much lower.37 Many relatively milddisorders are never recorded simply becausemedical attention is not sought. The incidentcases of some chronic age related disorders, suchas osteoarthritis, are very difficult to ascertainfrom medical records, even though theseconditions produce significant morbidity anddisability. In addition, hospital catchment areasand referral patterns are often difficult to specifywith precision. Thus, for most diseases, the task ofidentifying all cases in any large population mayappear insurmountable.There are several strategies in the study design

that the investigator may employ to reducemisclassification of the outcome status and avertbias. The diagnosis can often be verified in thecases, through the use of various auxiliary datasources, thus eliminating "false positives". In thisway, misclassification is reduced to under-ascertainment of the cases, a situation that may beeasier to handle, as discussed below. Examiningthe controls to detect "false negatives" isrecommended by some authors,4 but may bedifficult, especially when sophisticatedtechnology, invasive tests or unpleasantprocedures are required to rule out a givendiagnosis.When information is gathered prospectively,

the investigator may consider various forms ofactive case ascertainment, similar to those used inlarge experimental or observational cohorts. Insome cases, a periodic survey ofall members ofthebase may be an option, especially in studies ofrelatively mild but selfevident illnesses (eg, minoraccidents). In retrospective studies, multiple datasources should be used to improve detection. Forexample, to identify all cancers in a givenpopulation over a specified time period, theinvestigator may review the death certificates,hospital records, and pathology reports, inaddition to the data from a tumour registry.37Although diagnosis verification, active case

ascertainment, and the use of all available data

sources may significantly improve validity,substantial and potentially differentialunderdetection may still remain. It is importantto recognise that non-differential under-ascertainment of the cases will not bias theestimate of the rate ratio.38 Thus another strategyis to ensure, to the extent possible, that allactivities associated with case ascertainment areperformed with the same vigour among theexposed and unexposed, and follow the same,standard protocol. In randomised controlled trialsthis is achieved through blinding of themanoeuvre (eg, a placebo) and blind assessment ofthe outcome. In case-control studies, exposurestatus is often unknown to the physician makingthe diagnosis, and "unrealised" by the patient,which would make differential bias unlikely. Thissituation may be changing, however, asinformation about newly discovered, or onlysuspected health hazards is increasinglyconsidered a "hot topic" by the media. Whenreviewing medical records or verifying thediagnoses, the evaluators should be blind as towhether the subject is exposed or not.Another useful strategy is to restrict the study

population to those who, because of theirdemographic profile (age, sex, ethnicity,education, occupation, income) or some othercharacteristic specific to the problem studied (eg,being covered by a particular health plan, livingclose to the study hospital), have a reasonablechance of appearing in the case series should theydevelop the disease under investigation. Then, ofcourse, those cases that do not meet theprespecified criteria must be excluded. As part ofthe criteria for admission, it will often benecessary to modify the definition of a case. Forexample, only serious cases with overt symptomsare accepted. The study population may have tobe redefined once all potential sources of caseshave been determined and it turns out that certainsegments ofthe population are not covered. Savitzand Pierce39 illustrate how these problems mightbe approached in the case of male infertility, acondition with a very high degree ofunderascertainment. The study populationshould be restricted to those men who (1) want tohave children and (2) are willing to seek medicalcare when this desire is unfulfilled. Then, giventhat appropriate care is devoted to theidentification of cases, a high degree ofascertainment may.be achieved. At the same time,the selection ofa representative control group willbe possible, provided that such attributes as"child seeking behaviour" can be validlymeasured (eg, by means of a questionnaire). Inpractice, one could initially select a larger pool ofpotential controls from a "candidate" populationand then eliminate those who do not qualify asmembers of the base ("child seekers"). With theabove approach, the distinction between"primary base" and "secondary base" becomessomewhat artificial.39 The whole process ofdesigning a case-control study is in a senseiterative. The challenge is to define the studypopulation in such a way that all its members(over a specified time period) can be correctlyclassified as cases or "non-cases", using availablecase detection procedures.

182

on 6 May 2018 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.44.3.179 on 1 Septem

ber 1990. Dow

nloaded from

Bias in case-control studies. A review

MEASUREMENT OF EXPOSURE

In a laboratory experiment or, to a somewhatlesser degree, in a randomised controlled trial, theinvestigator has almost full control over thespecific type ofagent being investigated, as well asthe dose, timing and duration of exposure. Noneof these elements is controlled by the investigatorin a case-control study, in which the estimate ofexposure is usually based on past records, madefor other purposes, or recall of past events by thesubjects. Even the very definition of exposuremay be a problem, let alone the measurement ofthe dose.40 Ofcourse, the specific techniques usedto obtain exposure data will depend on the type ofexposure studied. These issues are too complex tobe discussed here, but excellent references are

available (eg41).In some case-control studies exposure is

recorded before the onset of disease, or after theonset of disease but before the assessment ofdisease status. However, it is ascertained by theinvestigator after the disease status has beenassessed. Thus the possibility of differentialmisclassification of exposure, often referred to as

"recall bias" when the assessment is based on selfreport, should always be taken into account.Another form of measurement related bias incase-control studies is protopathic (reversecausality) bias. Here, the ascertainment ofexposure is affected by disease related changes inthe level of exposure.42

Perhaps the most common method of datacollection in case-control studies is self report.The accuracy of recall in an interview has beenstudied extensively for a number of common

exposures, for example smoking, medicationassociated with female reproduction, past historyofchronic illness and hospital admission, diet, andmany occupational exposures. A common findingis that the quality of information depends on boththe type of data sought and the methodologicalaspects of the assessment process.43-46 Forexample, memory aids, such as pictorial displays,food models, or wall calendars have been founduseful in a number of studies.43 45 47 In a series ofexperiments on survey techniques, substantialimprovement in reporting, particularly for"sensitive" topics or "socially undesirable"behaviours, was observed when the followingtechniques were used: (1) a mixture of long andshort questions; (2) detailed respondentinstructions; (3) feedback (following eachquestion), informing the respondent how well heis performing; and (4) commitment technique,consisting in asking the respondent to sign an

agreement to respond conscientiously.48 The so

called cognitive approach to questionnaire designhas been extensively studied at the NationalCentre for Health Statistics, and the preliminaryresults are encouraging.49

In many situations, despite the use of state ofthe art measurement techniques, significant biasor imprecision in the estimate of exposure isdifficult to avoid. If this occurs, an importantconsideration is whether these measurementerrors are likely to be different (in terms offrequency, magnitude or direction) among thecases and the controls. Differential errors are

usually considered a greater threat to the study'svalidity than non-differential ones, since in the

former the direction of bias in the estimate ofeffect may be away from the null and may bedifficult to predict.50 Differential report (recall)bias has been implicated mainly in studies onpregnancy outcomes.5" There are also examples ofrecall bias in psychiatric publications.52 Thesuggestion that the apparent relationship of softtissue sarcoma and malignant lymphoma tophenoxy acids may be due to recall bias has notbeen confirmed.53 Overall, the empirical evidenceconcerning the extent and frequency of recall biasis scant.54 Of two recently reported studies, onefailed to provide any evidence of biased reportingof exposure during pregnancy,55 whereas thesecond study (using a somewhat different design)suggested the existence of substantial recall biasfor some, but not all, exposure factors.56The most effective device the investigator can

employ to ensure comparability of measurementis blind assessment of exposure. That is, neitherthe subject nor the investigator (or interviewer)should know who is a case and who is a control.This is analogous to double blinding, routinelyemployed in randomised controlled trials.Despite its strong appeal this obvious precautionis sometimes ignored in case-control studies. Ifblinding is not feasible, it is essential that the samestandard measurement conditions and proceduresare used for all subjects. In some instances thisapproach may lead to a conflict between theprinciple of maximum accuracy and that ofmaximum comparability of measurement. Forexample, if data for the cases have been obtainedthrough proxies, it may be advisable to obtain datafor the controls in the same way, even if it maycause greater bias (but hopefully similar to that inthe cases) or less precision than questioning thecontrols directly.12 On the other hand, there is noevidence to support the need for using deadcontrols for dead cases.57Although the use of some equivalent to a

placebo to simulate the disease (or death) amongthe controls is not feasible in most case-controlstudies, the investigator may select the controlsfrom subjects with other disorder(s). This methodis commonly used in hospital based studies.Disease controls may also be used as a secondcontrol group in population based studies, ifcomparability of information appears to be asignificant problem.

It should be emphasised that the methods ofcontrolling measurement bias in the analysis arevery limited. Predictors of measurement errorscannot, in general, be treated in the same way asconfounders in the analysis, and improvedmeasurement quality is always preferable toanalytical adjustments.58 Nevertheless, it maysometimes be possible to correct algebraically formeasurement bias after the data have beencollected, if a validation substudy has beenperformed. Even then, for a wide range ofconditions, it may be more efficient to use theoptimal (but more expensive) method ofmeasurement in all subjects, and enrol fewersubjects into the study.59 It has been alsosuggested that a validity subscale be incorporatedinto the questionnaire, using "fake" exposures,whose relation to the disease studied has beenruled out.5' Alternatively, an additional,"objective" source of information, such as

183

on 6 May 2018 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.44.3.179 on 1 Septem

ber 1990. Dow

nloaded from

84acek A Kopec, John M Esdaile

medical records, may be available for someexposures, but not for others; ifthe two exposuresare comparable in terms of validity of recall, themagnitude and direction of bias can be estimated,and an appropriate correction factor can beincorporated into the analysis. This approachhas a serious limitation as there is someempirical evidence that different exposures tendto behave differently with respect to reportingbias44 45 56

Sometimes, the best tactic to obtain goodquality data is to restrict eligibility to those forwhom valid information can be collected. This issomewhat analogous to checking the complianceof potential candidates for a randomisedcontrolled trial. Such restrictions should be madea priori (or, at least, without reference to theoutcome data collected in the study), and mayinvolve age, ability to communicate in a givenlanguage, mental abilities, educational level,occupational group, etc. For example, whenoccupation is used as a surrogate measure ofexposure, the researcher conducting a case-control study may select the occupational groupswith definitely high or low (plus, perhaps,intermediate) exposure levels, and exclude allcases and controls whose occupation is not a goodindicator of exposure. Such an approachresembles in some respects a cohort study, and isconsistent with the randomised controlled trialparadigm.

ParticipationIn randomised clinical trials, any confoundingpotentially introduced by arbitrary exclusions/inclusions, refusal to participate, or other forcesoperating prior to the moment ofrandomisation iseffectively prevented. However, exclusions,withdrawals or losses to follow up occurring afterrandomisation are a potential source of"selection" bias.35 60 Differential losses can oftenbe prevented by true blinding with respect to theexposure. If blinding is not possible, the successwith which the follow up is achieved becomes thebasic measure of the quality of the study.8 Yet,relatively little attention has been paid to highparticipation rate in evaluating the quality ofcase-control studies.6 Even ifwe assumed that noconfounding by unknown factors was initiallypresent in the source population (as ifrandomisation had been performed), thepossibility of "selection" bias due to missing data,non-response, incompleteness ofmedical records,obvious errors in measurement, and otheruncontrollable factors should always be taken intoaccount. For example, cancer patients who areaware of having been exposed to a widelypublicised risk factor may be more likely toparticipate than healthy or unexposedindividuals. A positive effect may then be foundeven if in fact there is no association between thedisease and the exposure. This type of bias mayoccur if the prospective participants (members ofthe source population) are aware of the exposureand disease being investigated, and the responserate is low. Exposure differential non-participation among the controls will bias theestimate of the proportion exposed in the sourcepopulation.

Participation bias should not occur if therelevant information is obtained for all subjectsinitially selected for the study, and all exclusionsare based on a priori defined criteria (definition ofthe base). In practice, one should probably strivefor a participation level similar to that required inother types of studies, ie, 80-900/ in both casesand controls. This will sometimes call for the useof special techniques, developed by socialscientists and large survey organisations. It hasbeen found, for example, that availability ofpotential respondents in a population surveydepends on the timing and number of calls madeby the interviewer, whereas refusals to participateare related to, among other things, fear of crimeand concerns about confidentiality.6' When dataare collected by means of a self administeredquestionnaire, the method known as "total designmethod" may considerably improve the responserate.62 In some studies, the low response rate wasdue to such simple omissions in the planning of astudy as the investigator's failure to print asufficient number of questionnaires for a secondand third mailing.62 Whenever feasible, thesubjects should be kept blind as to the hypothesisunder investigation. Restricting the studypopulation to those subjects who are likely toparticipate in any study, regardless of thehypothesis, may be a very useful strategy andshould be considered whenever such generalpredictors of cooperation are known beforehandand are easy to measure.

Conclusions and recommendationsIt seems reasonable to argue that under optimalconditions, a case-control study is as valid as arandomised clinical trial. For many problems, anested study provides evidence as strong as acorresponding cohort study, but may be muchmore efficient. This is particularly true when theinformation on exposure is obtained from pastrecords. Ofcourse, confounding remains a criticalissue in observational research and neither carefulsubject selection nor measuring knownconfounders, followed by adjustment in theanalysis, will ever provide the level ofcomparability of the exposed and unexposedgroups that is attainable through randomisation.This weakness of the observational design instudying certain behavioural interventions hasbeen elegantly demonstrated by Grey-Donaldand Kramer in their study of the relationshipbetween formula supplementation in hospital andduration of breast feeding.63

In this review we have deliberately omitted thetopic of confounding, and focused on othersources ofbias, particularly relevant in the contextof case-control studies. The ideas presented hereare not new, and have been discussed in a numberof recent texts and articles on methods inepidemiology. Rather than emphasise differencesbetween the authors, we have attempted toreconcile and bring together various seeminglydivergent points of view. Our point of departure,and a unifying theme, was the conceptual linkbetween a case-control study and a randomisedclinical trial. A number of bias minimisingstrategies have been discussed and these aresummarised below.

184

on 6 May 2018 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.44.3.179 on 1 Septem

ber 1990. Dow

nloaded from

Bias in case-control studies. A review

(1) In all case-control studies, an attempt shouldbe made to define explicitly the source populationfor the cases.

(2) If the case series can be linked to anidentifiable source population, a valid controlgroup can be obtained by random sampling.

(3) If hospital (registry, etc) controls are used,the control disease(s) should be subject to thesame case detection mechanisms/procedures asthe disease ofinterest, and should not be related tothe exposure at issue.

(4) In population based studies, completeascertainment of all cases should be sought. Thiscan be facilitated by: (a) a suitable definition oftheoutcome; (b) an extensive use of multiple datasources and active case detection; and (c)restricting the source population to thosepresumably covered by the availableascertainment systems/procedures.

(5) Additional eligibility criteria can be applied,in order to improve the quality ofinformation andprevent participation bias.

(6) State of the art methods of measurementshould be used, including sophisticatedquestionnaire designs.

(7) Blind assessment is the best way to ensurecomparability of information in the cases and inthe controls. If blinding is not feasible,standardised, uniform data collection proceduresmust be used in all subjects.

(8) The subjects and interviewers should not beinformed about the hypothesis under study.Diagnosis verification should be done "blindly".

(9) High response rate among both cases andcontrols should be obtained.

The authors are grateful to Drs J-F Boivin, R I Horwitzand A B Miller for their constructive criticism andhelpful comments on preliminary drafts of thismanuscript. However, the views expressed in the articleare those ofthe authors and do not necessarily reflect theviews of Drs Boivin, Horwitz, or Miller.Dr Kopec is the recipient of a Fellowship from the

National Health Research and Development Program ofCanada. Dr Esdaile is a Senior Research Scholar of theFonds de la recherche en sante du Quebec. Supported inpart by grants from the Arthritis Society of Canada andthe Institut de recherche en sante et securite du travail.

1 Kleinbaum DG, Kupper LL, Morgenstern H. Epidemiologicresearch: principles and quantitative methods. Belmont:Wadsworth, 1982.

2 Rothman KJ. Modern epidemiology. New York: Little,Brown, 1986.

3 Sackett DL. Bias in analytic research. Y Chron Dis 1979; 32:51-63.

4 Feinstein AR. Clinical epidemiology. The architecture ofclinical research. Philadelphia: WB Saunders, 1985.

5 Last JM, ed. A dictionary of epidemiology, 2nd ed. Oxford:Oxford University Press, 1988.

6 Cole P. Introduction. In: Breslow NE, Day NE. Statisticalmethods in cancer research. Vol 1. The design and analysis ofcase-control studies. Lyon: IARC, 1980.

7 Schlesselman JJ. Case-control studies: design, conduct,analysis. New York: Oxford University Press, 1982.

8 Breslow NE, Day NE. Statistical methods in cancer research.Vol 2. The design and analysis ofcohort studies. Lyon: IARC,1987.

9 Kramer MS, Boivin J-F. Toward an unconfoundedclassification ofepidemiologic research design.J Chron Dis1987; 40: 683-8.

10 Greenland S, Morgenstern H. Classification schemes forepidemiologic research design. J Clin Epidemiol 1988; 41:715-6.

11 Miettinen OS. Theoretical epidemiology. Principles ofoccurrence research in medicine. New York: John Wiley &Sons, 1985.

12 Miettinen OS. The case-control study: valid selection ofsubjects. J Chron Dis 1985; 38: 543-8.

13 Mantel N. Synthetic retrospective studies and related topics.Biometrics 1973; 29: 479-86.

14 Snow J. On the mode of communication of cholera. 2nd ed.London: John Churchill, 1960.

15 Hill AB. Observation and experiment. N EnglI Med 1953;248: 995.

16 Horwitz RI, Feinstein AR. The application of therapeutic-trial principles to improve the design of epidemiologicresearch: a case-control study suggesting thatanticoagulants reduce mortality in patients with myocardialinfarction. J Chron Dis 1981; 34: 575-83.

17 Feinstein AR. Experimental requirements and scientificprinciples in case-control studies. J Chron Dis 1985; 38:127-33.

18 Feinstein AR. Epidemiologic analyses of causation: theunlearned scientific lessons of randomized trials. J ClinEpidemiol 1989; 42: 481-9.

19 Horwitz RI. The experimental paradigm and observationalstudies of cause-effect relationship in clinical medicine. JChron Dis 1987; 40: 91-9.

20 Esdaile JM, Horwitz RI. Observational studies of cause-effect relationships: an analysis of methodologic problemsas illustrated by the conflicting data for the role of oralcontraceptives in the etiology of rheumatoid arthritis. JChron Dis 1986; 39: 841-52.

21 Miettinen OS. The clinical trial as a paradigm forepidemiologic research. J Clin Epidemiol 1989; 42: 491-6.

22 Kupper LL. McMichael AJ, Spirtas R. A hybridepidemiologic study design useful in estimating relativerisk. J Am Stat Assoc 1975; 70: 524-8.

23 Miettinen OS. Design options in epidemiologic research. Anupdate. Scand J Work Environ Health 1982; 8 (Suppl):7-14.

24 Breslow N. Design and analysis of case-control studies.Annu Rev Public Health 1982; 3: 29-54.

25 Greenland S, Thomas DC. On the need for the rare diseaseassumption in case-control studies. Am J Epidemiol 1982;116: 547-53.

26 Greenland S. Control-initiated case-control studies. Int JEpidemiol 1985; 14: 130-4.

27 Axelsson 0. Comment. The "case-control" study: validselection of subjects. J Chron Dis 1985; 38: 553-5.

28 Axelsson 0, Flodin U, Hardell L. A comment on thereference series with regard to multiple exposureevaluations in a case-reference study. Scand J WorkEnviron Health 1982; 8 (Suppl): 15-19.

29 Knottnerus JA. Letter to the Editors. Subject selection inhospital-based case-control studies. J Chron Dis 1987; 40:183-5.

30 Stavraky KM, Clarke EA. Hospital or population controls?An unanswered question. J Chron Dis 1983; 36: 301-7.

31 Siemiatycki J, Colle S, Campbell S, Devar R, BelmonteMM. Case control study of insulin dependent (type I)diabetes mellitus. Diabetes Care 1989; 12: 209-16.

32 Siemiatycki J. Friendly control bias.J Clin Epidemiol 1989;42: 687-8.

33 Weiss NS, Daling JR, Chow WH. Incidence ofcancer of thelarge bowel in women in relation to reproductive andhormonal factors. I Natl Cancer Inst 1981; 67: 57-60.

34 Harris NV, Weiss NS, Francis AM, Polissar L. Breastcancer in relation to patterns oforal contraceptive use. AmJEpidemiol 1982; 116: 643-51.

35 Peto R, Pike MC, Armnitage P, et al. Design and analysis ofrandomized clinical trials requiring prolonged observationof each patient. I. Introduction and design. Br J Cancer1976; 34: 585-612.

36 Poole C. Exposure opportunity in case-control studies. AmjEpidemiol 1986; 123: 352-8.

37 Robles SC, Marrett LD, Clarke EA, Rish HA. Anapplication ofcapture-recapture methods to the estimationof completeness of cancer registration. J. Clin Epidemiol1988; 41: 495-501.

38 Poole C. Exceptions to the rule about nondifferentialmisclassification (Abstract). AmJEpidemiol 1985; 122: 508.

39 Savitz DA, Pearce N. Control selection with incomplete caseascertainment. AmJ Epidemiol 1988; 127: 1109-17.

40 Matanoski GM. Issues in the measurement of exposure. In:L Gordis, ed. Epidemiology and health risk assessment.Oxford: Oxford University Press, 1988.

41 Gordis L, ed. Epidemiology and health risk assessment.Oxford: Oxford University Press, 1988.

42 Horwitz RI, Feinstein AR. Methodologic standards andcontradictory results in case-control studies. Am J Med1979; 66: 556-64.

43 Mitchell AA, Cottler AB, Shapiro S. Effect of questionnairedesign on recall of drug exposure in pregnancy. Am JEpidemiol 1986; 123: 670-6.

44 Harlow SD. Agreement between questionnaire data andmedical records. The evidence for accuracy of recall. AmJ

Epidemiol 1989; 129: 233-48.45 Lee-Han H, McGuire V, Boyd NF. A review ofthe methods

used by studies of dietary measurement. J Clin Epidemiol1989; 42: 269-78.

185

on 6 May 2018 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.44.3.179 on 1 Septem

ber 1990. Dow

nloaded from

Jacek A Kopec, John M Esdaile

46 Jain M, Howe GR, Harrison L, Miller AB. A study ofrepeatability of dietary data over a seven year period. AmJEpidemiol 1989; 129: 422-9.

47 Beresford SAA, Coker AL. Pictorially assisted recall of pasthormone use in case-control studies. Am3J Epidemiol 1989;130: 202-5.

48 Cannel CF, Miller PV, Oksenberg L. Research oninterviewing techniques. In: Lainhard S, ed. Sociologicalmethodology. San Francisco: Jossey-Blass, 1981.

49 Jobe JB, Mingay DJ. Cognitive research improvesquestionnaires. Am J Public Health 1989; 79: 1053-5.

50 Copeland KT, Checkoway H, McMichael AJ, HolbrookRH. Bias due to misclassification in the estimation ofrelative risk. Am J Epidemiol 1977; 105: 488-95.

51 Raphael K. Recall bias: a proposal for assessment andcontrol. Int J Epidemiol 1987; 16: 167-70.

52 Wolkind S. Koleman EZ. Adult psychiatric disorder andchildhood experiences: the validity ofretrospective data. BrJ Psychiatry 1983; 143: 188-91.

53 Hardell L. Relation of soft tissue sarcoma, malignantlymphoma and colon cancer to phenoxy acids,chlorophenols and other agents. Scand Work EnvironHealth 1981; 7: 119-30.

54 Lippman A, MacKenzie SG. What is recall bias and does itexist? In: Marois M, ed. Progress in clinical and biologicalresearch, vol 163. New York: Alan R Liss, 1985.

55 MacKenzie SG, Lippman A. An investigation of report biasin a case-control study of pregnancy outcome. AmEpidemiol 1989; 129: 65-75.

56 Werler MM, Prober BR, Nelson K, Holmes LB. Reportingaccuracy among mothers of malformed and nonmalformedinfants. Am I Epidemiol 1989; 129: 415-21.

57 Gordis L. Should dead cases be matched to dead controls?AmJEpidemiol 1982; 115: 1-5.

58 Greenland S. Robins JM. Confounding andmisclassification. Am J Epidemiol 1985; 122: 496-506.

59 Greenland S. Statistical uncertainty due to misclassification:implications for validity substudies. J Clin Epidemiol 1988;41: 1167-74.

60 Armitage P. Exclusions, losses to follow-up, andwithdrawals in clinical trials. In: Shapiro SH, Louis TA,eds. Clinical trials. Issues and approaches. New York:Marcel Dekker, 1983.

61 Hawkins DF. Nonresponse in Detroit area study surveys: aten year analysis. Working papers in methodology No 8.Chapel Hill, N C: Institute for Research in Social Science,1977.

62 Dillman DA. Mail and other self-administeredquestionnaires. In: Rossi PH, Wright JD, Anderson AB.Handbook of survey research. New York: Academic Press,1983.

63 Grey-Donald K, Kramer MS. Causality inference inobservational vs. experimental studies. An empiricalcomparison. Am J Epidemiol 1988; 127: 885-92.

186

on 6 May 2018 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.44.3.179 on 1 Septem

ber 1990. Dow

nloaded from