Validity Issues in CB Formulation

G. H. Mumma: Validity Issues in Cognitive-Behavioral Case FormulationEuropean Journal of PsychologicalA ssessment 2011; Vol. 27(1):29–49© 2011 Hogrefe Publishing

Special Section: Clinical Case Formulation

Validity Issues in Cognitive-Behavioral Case Formulation

Gregory H. Mumma

Texas Tech University, Lubbock, TX, USA

Abstract. Despite considerable interest and growth in methods to develop or generate cognitive behavioral case formulations (CBCFs),relatively little conceptual and empirical work has focused on the validation or testing of these formulations. A case formulation can beregarded as an idiographic theory of the person and his or her life situation. This complex set of clinical judgments consists of a mea-surement model including the behavior problems or distress constructs and how they are measured; and a causal model involving variablessuch as thoughts or beliefs hypothesized to trigger and maintain this person’s distress or dysfunction. This article describes four types ofvalidity issues in CBCF and how these validity issues can be evaluated using person-specific, intraindividual data collected daily ormultiple times a day. Specific topics include the evaluation of content and construct (convergent and discriminant) validity for themeasurement model, and the evaluation of predictive and treatment-related validity for the causal model. One goal of the person-specificevaluation of CBCF validity is to develop an intraindividual statistical prediction model that has the advantages of actuarial predictionyet is fine-grained and tailored to the specific issues and life circumstances of greatest relevance for a particular individual. Greaterattention to evaluation of validity issues in CBCF is important for future research comparing formulation-based to manualized treatment.Implications and applications to clinical practice and training are discussed.

Keywords: cognitive case formulation, clinical case formulation, idiographic assessment, clinical vs. statistical (actuarial) prediction,person-specific validation

A cognitive behavioral case formulation (CBCF) can bedefined as an idiographic theory of the person and his orher life situation (the person-situation), which includesproblems as well as triggering and maintaining variables,including cognitions (thoughts and beliefs), that have rele-vance for treatment planning for a particular individual.Growing interest in CBCFs is apparent in the increasingnumber of manuals describing how the clinician can orshould develop a case formulation. These manuals eitherfocus on CBCF generally (e.g., Kuyken, Padesky, & Dud-ley, 2009; Persons, 1989, 2008), combine this with a focuson specific disorders (e.g., Needleman, 1999; Nezu, Nezu,& Lombardo, 2004; Persons, Davidson, & Tompkins,2001), or develop templates to use as a starting point for adisorder-specific CBCF (Tarrier, 2006; Wells, 2006; Zay-fert & Becker, 2007).

Case formulation (CF) can be regarded as a complexinterrelated set of clinical judgments and inferences havingtreatment relevance (Haynes & O’Brien, 2000). But thesejudgments can be influenced by information processingand judgmental biases that can decrease their validity(Garb, 1998; Grove, Zald, Lebow, Snitz, & Nelson, 2000).Although increasingly structured and specific manuals onCBCF can help to minimize such biases, these manualshave typically provided only limited guidance on how tovalidate or test the CBCF. A basic premise of this article is

that, once developed (even if in just a preliminary form),the validity of the CBCF should be evaluated to increasethe likelihood that it is accurate and assists in treatmentplanning.

Indeed, one implication of regarding a case formulationas an idiographic theory of the person-situation that focuseson the person’s distress and dysfunction and the relevanttriggering and maintaining variables (Korchin, 1976;Sundberg & Tyler, 1962) is that the clinician needs to func-tion as a pragmatic, idiographic clinical scientist who notonly develops or generates this idiographic person-specifictheory, but also empirically evaluates the validity of theconstructs and empirically tests the hypothesized relation-ships between outcome and causal variables.

Validity Evaluation and the Efficacy ofFormulation-Based Treatment

Surprisingly, with the exception of a few studies examininginterrater reliability or treatment utility of CBCFs, fewstudies have tested or evaluated the validity of CBCFs orhave proposed methods to do so. This is surprising for sev-eral reasons: (a) Validity issues have historically been a

DOI: 10.1027/1015-5759/a000054© 2011 Hogrefe Publishing European Journal of Psychological Assessment 2011; Vol. 27(1):29–49

major focus in the development of constructs and measuresof constructs in nomothetic psychological science. Giventhat psychology is an empirical science, a psychologicaltheory must be repeatedly tested empirically and subjectedto risk of disconfirmation to be considered valid (Anderson& Gerbing, 1988; Shadish, Cook, & Campbell, 2002). (b)Cognitive behavior therapy (CBT) has developed a strongempirical base for its efficacy with randomized controlledclinical trials (and, more recently, some effectiveness) stud-ies (Chambless, 1996; Wilson, 1996). Some degree of caseformulation is often present in many manualized treatments(Wilson, 1996); yet, again, methods to validate the formu-lation are generally not described. (c) Behavioral case for-mulations developed using a functional analysis are fre-quently evaluated and experimentally tested using manip-ulations of the stimuli and/or contingencies either by theclinician or an agent such as a teacher or parent who cancontrol these (e.g., Iwata et al., 1994; Sturmey, 2008). (d)Certain psychodynamic approaches have developed re-search-based methods to evaluate the validity of their CFs(e.g., Horowitz & Eells, 2007; Luborsky, Crits-Christoph,& Mellon, 1986). Despite all of the above, validity issueshave received relatively scant attention in CBCF on theconceptual/theoretical, methodological, and empirical lev-els (Bieling & Kuyken, 2003; Kuyken, 2006; Kuyken etal., 2009).

The case formulation is used to guide formulation-basedtreatment – treatment tailored to the unique and specificneeds of a particular client. But are formulation-basedtreatments more effective? A small number of randomizedcontrolled clinical trials have compared formulation-basedtreatment with manualized treatment (e.g., Emmelkamp,Bouman, & Blaauw, 1994; Ghaderi, 2006; Jacobson et al.,1989; Schulte, Kunzel, Pepping, & Schulte-Bahrenberg,1992). Although several reviews discussed important is-sues in these studies (e.g., Bieling & Kuyken, 2003; Kuy-ken, 2006; Kuyken et al., 2009; Persons et al., 2001), someadditional concerns that bear on validity issues in CBCFhave not been discussed. First, in these studies, the validityor accuracy of the CF in the formulation-based treatmentcondition was not evaluated; rather, it was left as an un-measured random variable. This is a particular concern be-cause the CFs in many of these studies were developedprior to the more detailed description of procedures avail-able in more recent manuals (e.g., Needleman, 1999; Nezuet al., 2004; Persons, 2008; Tarrier, 2006), so there mightbe more variability in the quality of the CF (see Eells, Lom-bart, & Kendjelic, 2005; Kuyken, Fothergill, Musa, &Chadwick, 2005, for CF quality criteria) as well as in thevalidity of the CF (as evaluated using methods describedin this article). Second, issues of treatment adherence for

the formulation-based treatments have generally not beenaddressed (Aston, 2009; Mumma, 1998). Third, these stud-ies generally used standardized measures of target con-structs. This is an important limitation because the case for-mulation can include idiosyncratic and unique aspects ofthe person’s distress, dysfunction, and treatment goals, allof which can be assessed idiographically and then targetedfor intervention (Haynes & O’Brien, 2000; Nelson-Gray,2003). Thus, these studies may have lacked sensitivity todetect change in areas of particular relevance to each par-ticipant. Finally, these studies generally focused on clinicalpopulations with predominantly single disorders (phobias,obsessive-compulsive disorder, bulimia), for which rela-tively effective manualized treatments exist. However, theutility of formulation-based CBT may primarily lie in thetreatment of complex, comorbid, and complicated cases aswell as for disorders or problems for which there are noempirically supported treatments (Haynes & O’Brien,2000; Mumma, 2001, 2004; Persons, 2008; Persons, Rob-erts, Zalecki, & Brechwald, 2006). In short, there has beena mismatch between the populations for which CBCF-based treatment may be most beneficial and those used instudies comparing manualized with formulation-basedtreatment. Overall, the jury is still out on the treatment util-ity of formulation-based versus manualized treatments.Clearly, though, evaluating and thus improving the validityof CBCFs should enable more valid comparisons of formu-lation-based versus manualized treatments in the future.1

Before discussing validity issues and evaluation inCBCF, we describe the person-specific approach briefly.

The Person-Specific Approach:Intraindividual Relationships andChange

The importance of quantitatively examining variables andrelationships between variables within a specific individ-ual has been suggested for a number of years. This ap-proach is closely related to idiographic assessment (de-scribed below), which focuses on increasing the relevanceand sensitivity of measurement for the particular person(e.g., Allport, 1962; Haynes & O’Brien, 2000). The basisand justification for the person-specific approach derivesfrom the notion that patterns of correlations between itemson a questionnaire may be different, depending on wheth-er data are collected from many persons or from one per-son across many times.2 This notion is probably not wide-ly known to clinicians and investigators alike, and is based

30 G. H. Mumma: Validity Issues in Cognitive-Behavioral Case Formulation

European Journal of Psychological Assessment 2011; Vol. 27(1):29–49 © 2011 Hogrefe Publishing

� The validity of CFs can potentially be improved by methods other than those discussed in the present article. For example, evaluating andimproving the validity of standardized measures used to test components of the CF and to evaluate change and response to treatment maycontribute to improving the validity of a CF. Likewise, research into reducing biases in clinical judgment (Garb, 1998; Mumma & Wilson,1995) may increase CF validity. Approaches such as these are beyond the scope of this article.

� My thanks to Steven Haynes for suggesting this simple but elegant way of summarizing this issue.

on some rather complex mathematical principles (seeMolenaar, 2004; Molenaar & Campbell, 2008; Molenaar& Valsiner, 2005). Because of this issue, however, an in-strument used repeatedly over time within an individualneeds to have its dimensional structure and validity coef-ficients evaluated using data collected repeatedly overtime from that specific person. This person-specific vali-dation must be done for any instrument used to assess con-structs repeatedly over time for that person, regardless ofwhether the instrument is standardized (and thus validatedusing between-persons designs) or idiographic (and thusdeveloped solely for that particular person.)3 Person-spe-cific validation of measures uses specialized factor anal-ysis methods developed to handle analysis issues (such astoday’s score being influenced by yesterday’s score) oftenencountered in time-series data. (See Ferrer & Nessel-roade, 2003; Luborsky, 1995; Molenaar, 1985; Mumma,2004; Nesselroade, McArdle, Aggen, & Meyers, 2002;Wood & Brown, 1994, for more information on dynamicfactor analysis and related methods.)

The same notion, that patterns of correlations betweenvariables may be different depending on whether data arecollected from many persons or from one person acrossmany occasions, also pertains to relationships betweenvariables. For example, studies of the specificity of cog-nitive content to distress have found that more depressedindividuals are more likely to report thoughts or beliefsinvolving personal failure, loss, or inadequacy, whereasmore highly anxious individuals are more likely to havethoughts regarding threat and danger (Clark & Beck, 1999,2010). However, unless certain restrictive assumptions aremet, these results cannot be assumed to apply to relation-ships between variables within a specific individual. Forexample, we cannot simply assume that experiencingmore frequent thoughts of failure on some days predictsgreater depression on those days for a particular person,whereas experiencing more threat-related thoughts pre-dicts greater anxiety for that person. The appropriate per-son-specific, intraindividual analyses need to be conduct-ed on data collected over time from that person, to seewhether these relationships are indeed present for this par-ticular person (Molenaar, 2004). Specialized regressionanalysis methods (e.g., dynamic time series regression)are needed to test such CBCF hypotheses, as well as ad-ditional case formulation hypotheses involving lagged re-lationships. For example, do more frequent thoughts offailure today predict a higher level of depression tomor-row? Testing hypotheses about lagged relationships maybe particularly important because temporal precedence is

important in evaluating mediation effects (Cole & Max-well, 2003) as well as causal effects more generally(Haynes & O’Brien, 2000). (See Cook & Campbell, 1979;Hokanson, Tate, Niu, Stader, & Flynn, 1994; Mumma,2004; Pankratz, 1991; Tabachnick & Fidell, 2007, formore information on dynamic time series regression andrelated methods.)

Validity in CBCF: General Issues4

Several general issues relevant to validity evaluation inCBCF are briefly discussed next.

Incremental Validity

Basically, the CBCF must be worth the additional effort,cost, and resulting complexity over and above assessmentwith standardized measures and use of standardizedtreatment manuals. The CBCF should yield explanations,predictions, or interventions that are incrementally bet-ter compared to these baselines (see Haynes & Lench,2003).

Approach-Specific Aspects of Validity

Different approaches to CBCF may emphasize differenttypes of information to be assessed, different types of vari-ables and relationships between these variables to evalu-ate, and different structures for the case formulation (seeapproaches of J. Beck, 1995 versus Persons, 2008 or Per-sons et al., 2001). Thus, certain validity issues may be spe-cific to each approach, but these lie beyond the scope ofthis article. Other approaches to CF are transtheoretical inthat they can incorporate cognitive variables in the assess-ment and formulation, though the focus is on the structureand processes of case formulation. Examples include theproblem-solving CF approach of Nezu, Nezu, and Lom-bardo (2004), the functional analytic clinical case modelof Haynes and O’Brien (1990, 2000), and the intra- andinterpersonal approach of Jose and Goldfried (2008). Thebehavioral-assessment-based model of Haynes andO’Brien is used as a transtheoretical model for the presentarticle.

G. H. Mumma: Validity Issues in Cognitive-Behavioral Case Formulation 31

© 2011 Hogrefe Publishing European Journal of Psychological Assessment 2011; Vol. 27(1):29–49

� Note that this issue differs from the one examined in studies evaluating changes in the psychometric structure of well-established standard-ized symptom inventories when administered repeatedly to the same sample of individuals. Such studies evaluate changes in the between-persons psychometric structure due to repeated administrations, whereas person-specific validation pertains to patterns of correlations forscores collected over time within a single person.

� This article does not discuss reliability issues in CBCF. Some of these issues (particularly internal consistency reliability) are discussed asan aspect of convergent validity, and the small number of studies examining interrater or interformulator reliability in CBCF have beenreviewed and critiqued elsewhere (Kuyken, 2006; Kuyken et al., 2009; Mumma, in press). Validity issues in CBCF are largely unaddressedand are the focus of this article.

Types of Validity and Validity Issuesin CBCF

This section describes four types of validity and validity is-sues for CBCF that are relevant to the individual (the specificpatient) and that can be evaluated using person-specific meth-ods. The first two types of validity, content and constructvalidity, pertain to the measurement model of the CBCF – theconstructs and measures used in the formulation, includingthe target/outcome variables (typically distress or dysfunc-tion), triggering variables, and various types of maintainingand mediating variables, including cognitions. The third andfourth types of validity, predictive and treatment-related va-lidity, pertain to the causal or structural model aspect of theCBCF.5 For each type of validity, discussion is organized intofour basic sections: The first section includes a definition ofthat validity issue and a brief description of the origins andapplication of this type of validity issue in traditional psycho-metrics. Each type of validity issue has a meaning or appli-cation to CBCF that may differ somewhat from its conceptu-alization, evaluation, or use with traditional tests or measures.Thus, the second section describes the relevance and applica-tion of this type of validity to CBCF. When relevant, applica-tions of that validity issue to behavioral (and occasionallypsychodynamic) case formulation are mentioned. Specifictopics include distinctions in the nature and conceptualizationof this type of validity on the aggregate versus individuallevels, components or facets of this type of validity relevantto CBCF, and when relevant, the most important conceptual,theoretical, and methodological issues pertinent to applyingthis type of validity to CBCF on the person-specific level. Thethird section discusses methods to evaluate this type of valid-ity issue for CBCFs using a person-specific (intraindividual)approach. Empirical studies that have used this methodologyare reviewed. The final section discusses limitations of andissues in this person-specific validity evaluation approachand includes research as well as clinical training and practiceimplications.

Validity Issues for the CBCF Measure-ment Model Part I: Content Validity

Definition and Application to TraditionalPsychometrics

Within traditional psychometrics, content validity refers tothe extent to which the relevant elements in a domain areadequately represented by items in the test. For example,

for the domain of geometry, an achievement test demon-strates content validity to the extent that items on the testcover the skills relevant for the course or curriculum takenby the students.

Relevance and Application of ContentValidity to CBCF

In CBCF, content validity refers to the extent to which therelevant target and causal variables, their facets and ele-ments, and the relationships among the variables of a par-ticular type are included and represented in the CBCF mea-surement model for a particular person (see Haynes &O’Brien, 2000). For a CBCF, the outcome variables areusually distress (e.g., depression, anxiety, pain) and/or dys-function (e.g., does not get out of bed, misses work, avoidscertain situations). Causal variables may be both environ-mental (e.g., argument with a coworker) and internal (e.g.,automatic thoughts, beliefs). The content validity of the re-lationships between variables refers to relationships be-tween constructs within each type of variable in the mea-surement model. For example, are the expected relation-ships among the different distress constructs adequatelyrepresented in the CF? How about the relationships amongthe various cognitive schema (and related automaticthoughts)? Note that relationships between different typesof variables, such as between situational triggers and dis-tress, are issues relevant to the causal or structural modelcomponent of the CBCF.

The three main components of content validity are rep-resentativeness, relevance, and coverage (Haynes &O’Brien, 2000; O’Brien, Oemig, & Northern, 2010). Thereare several dimensions of representativeness. Does the CFaccurately represent problems, issues, and causal variablesin terms of the following:1. Dimensions: Does the CF adequately capture or repre-

sent the most important dimensions of each behaviorproblem or causal variable for that person, including fre-quency, duration, and intensity? For sad mood, for ex-ample, the most important dimension may be the peakand average intensity during the day. For staying in bedthe most important dimension may be duration.

2. Topography: Does the CF adequately represent impor-tant aspects of the topography of the variables or facets?For example, staying in bed an extra hour is likely to bemuch more of a problem during the weekdays when itmay result in missing classes or being late to work thanon weekends. Content validity also refers to whether themeasures used and the data collected adequately repre-sent these aspects of the CF.



� Anderson and Gerbing (1988) discuss how nomothetic psychological theories can be tested using a structural equation modeling approachand dividing the process into testing the measurement model then the structural model. The latter includes the relationships between theoutcome constructs and the causal constructs. The CBCF, as an idiographic theory, can also be considered to include a measurement modeland a structural model. The latter involves predicted relationships between causal and outcome variables. Within functional analytic clinicalcase models, these relationships are diagramed with single-headed arrows, just as in a structural equation models (Haynes & O’Brien, 2000).

Relevance is important because there may be aspects of theperson’s functioning and experience that are representativebut not particularly clinically important. For example, onceat work a person may interact with coworkers in a friendlymanner and generally complete tasks on time. This mayrepresent a relatively large proportion of the patient’s wak-ing hours and is relevant in that he or she is able to functioncompetently in both the interpersonal and work/task do-mains while at work. However, a CBCF that describesworkplace functioning in detail may be low in relevancebecause it is not focusing on the triggering and maintainingvariables for clinically significant target variables, such assad mood or staying in bed.

Finally, coverage – does the CBCF accurately includethe range of outcome and causal variables and the relation-ships within each type of variable that are relevant for thatperson and representative of the issues that are creating dis-tress or dysfunction?

Content validity issues for CBCF can be defined at boththe individual and aggregate levels. To illustrate these is-sues, consider Clark and Beck’s (2010) recent cognitivemodel of anxiety and the associated “Cognitive Case Con-ceptualization of Anxiety” (pp. 176–178). The latter is acase formulation template that guides the clinician duringthe assessment process and includes specific thoughts, be-haviors, and sensations that form the person-specific infor-mation for the CBCF. For example, to complete part of thetemplate, the clinician and the patient conduct a situationalanalysis, an indepth exploration of a specific recent anxiousevent (such as a panic attack or occurrence of an obsessivethought). Based on that data as well as information fromother sources (e.g., clinical monitoring logs), the clinicianlists the “primary external” and “primary internal/cognitivetriggers” of the patient’s “immediate fear response,” thenlists the “first apprehensive thoughts/images,” then the“physical sensations/symptoms” occurring during “physi-ological hyperarousal,” as well as the patient’s “misinter-pretation” of these cues. This template (only part of whichis mentioned here) is based on Clark and Beck’s cognitivemodel of anxiety. They review empirical studies that sup-port (or occasionally do not support) the validity of each ofthe ten components of this model. This research tests thevalidity of their nomothetic psychological theory (a theoryintended to apply across individuals with anxiety disor-ders).

Consider the following content validity issues and ques-tions relevant to the Clark and Beck (2010) CBCF templateand the resulting CBCF for a specific person. First, on theaggregate level: Are the outcome variables (e.g., severityof anxiety or worry as reported on specific measures suchas the Beck Anxiety Inventory and Penn State Worry Ques-tionnaire) included in the template generally relevant, rep-resentative, and useful to individuals diagnosed with anxi-ety disorders? This could be addressed by using this CBCFtemplate to assess a number of individuals, perhaps includ-ing people with different anxiety disorders, with each pa-tient and clinician rating the relevance and representative-

ness of the anxiety assessed to his or her specific distress.Note that although this is aggregate-level research used toaddress this CF validity issue, it is a different type of re-search than that used to test the components of the Clarkand Beck cognitive model of anxiety. A related aggregate-level CF validity issue is the relevance, comprehensive-ness, and representativeness of the causal variables in theCBCF. This issue focuses on the internal and external trig-gers, apprehensive thoughts and images, misinterpretationof physiological sensations, etc. A related issue is the rel-evance, comprehensiveness, and representativeness of thisinformation in understanding anxiety for individuals withdifferent anxiety disorders. Unfortunately, questions suchas this have been addressed rarely for CBCF (but see Chad-wick, Williams, & Mackenzie, 2003; Pain, Chadwick, &Abba, 2008 for exceptions). A third validity issue about theformulation template is whether it accurately reflects theprocesses that trigger or maintain distress or dysfunctiondescribed in their cognitive model of anxiety. It is possiblethat the model of anxiety may be valid, but the cognitivecase conceptualization may not accurately represent themodel. Alternatively, the template may include informationthat, though not relevant to the model, is nevertheless rel-evant to distress or dysfunction related to anxiety or to itscausal variables

On the individual level, content validity issues for theCBCF focus on the extent to which outcome and causalvariables in the model adequately represent those of great-est relevance to that person in the circumstances of greatestconcern for that person. The CBCF resulting from usingthis template for a person with bipolar disorder would like-ly be low in relevance and representativeness of the distressand dysfunction for that person, making the content valid-ity of this CBCF low for this person. Using a case formu-lation template(s) that is(are) well matched to the problemsand issues of the patient should generally increase the con-tent validity of the CBCF for that person (Haynes, Kaho-lokula, & Nelson, 1999; Persons & Tompkins, 2007).

Strategies to increase the relevance of variables as-sessed, such as the assessment funnel and the use of well-validated standardized measures and multimethod andmultiinformant measurement methods have been discussedextensively elsewhere. However, the important role of idio-graphic assessment in increasing the content validity of aperson’s CBCF merits brief description.

Idiographic assessment is “the measurement of variablesand functional relations that have been individually select-ed, or derived from assessment stimuli or contexts that havebeen individually tailored, to maximize their relevance forthe particular individual” (Haynes, Mumma, & Pinson,2009, p. 179). This tailored measurement may captureunique aspects of a particular person’s functioning and ex-perience not included in standardized instruments or ques-tionnaires. More generally, idiographic assessment shouldboth increase specificity and relevance of the variables tothe life circumstance of the particular person and increasesensitivity to detect relationships between variables within



that person and sensitivity to detect change (Haynes &O’Brien, 2000; Mumma, 2001). Idiographic assessmentshould produce a fine-grained, highly individualizedCBCF that covers the problems/issues and causal variablesof greatest importance and relevance to that individual. In-deed, evaluating the validity of a CF for a particular indi-vidual without including individually relevant (idiograph-ic) problems, goals, mechanisms, and outcomes is probablyinadequate (Haynes & O’Brien, 2000; Nelson-Gray, 1996;Tarrier & Calam, 2002).

Evaluation of Content Validity in CBCF UsingPerson-Specific Methods

Studies evaluating the content validity of CBCFs are rela-tively scant – content validity has generally been evaluatedwhen evaluating convergent and discriminant validity (seebelow). One approach, the method of “mismatched” cases,originally developed for evaluating components of psycho-dynamic formulations (Levine & Luborsky, 1981), com-pares relevance ratings of formulation components for thetarget patient to other cases. Several CBCF investigatorshave used this method; they had participants (and, in somestudies, clinicians and observers) rate the relevance of com-ponents of clinical scenarios, such as specific thoughts, be-liefs, or behaviors (Mumma, 2004; Mumma & Mooney,2007a; Muran, Samstag, & Segal, 1998; Muran, Samstag,Ventur, Segal, & Winston, 2001; Muran & Segal, 1992).Note, however, that this method was used only to evaluaterelevance during the generation of the CF. Also, this meth-od does not insure adequate coverage because the items andvariables evaluated are those included in the CF so somerelevant behaviors may be missing. Nor does it evaluate therelationships between variables.

Several additional strategies are suggested.1. The relevance of constructs or facets (items) within a

construct can be periodically re-evaluated over time bythe patient, clinician, or significant other. Variables oritems that have become low in relevance may bedropped.

2. Similarly, as assessment and treatment progresses, theclinician and patient may become aware of issues, vari-ables, or facets of variables that were not included in theinitial formulation and need to be added. For example,Smith and Mumma (2006) showed how the items on anidiographic questionnaire assessing depression and an-ger were changed over time as the CBCF was modifiedand elaborated by the clinician and patient.

3. Finally, relationships among variables of a certain type(e.g., distress constructs such as anxiety and depression)may change over time and should be periodically reeval-uated to determine whether changes have occurred. (SeeMumma, 2004 for some specific evaluation methods us-ing a person-specific approach with daily ratings.)

Summary: Issues, Limitations andImplications

Content validity includes the components of representa-tiveness, relevance, and coverage. Ideally, the content va-lidity of a CBCF for a particular person should be empiri-cally evaluated repeatedly over time, including relevanceratings of items or formulation components. Items or vari-ables should be added or deleted from the formulation andthe measures used to test and validate the formulation asnecessary. Adding or deleting items or scales creates sta-tistical analysis issues, but these can be handled adequatelyeven in a training or practice context (Smith & Mumma,2006).

Validity Issues for the CBCFMeasurement Model Part II:Construct Validity

Definition

Construct validity issues are central to virtually all mea-surement in clinical psychology and have an importantfunction in CBCF. Construct validity is a complex notionthat includes a number of basic tenants (Burns & Haynes,2006; Mumma, in press; Nunnally & Bernstein, 1994; Sha-dish, Cook, & Campbell, 2002):1. A construct, a hypothesized latent variable, typically

consists of multiple facets or components that may bemanifest in a number of response modes. Thus, a singleitem typically provides insufficient coverage of thesefacets. Also, a single item carries unwanted variance as-sociated with other constructs (known as mono-opera-tional bias). This is a particularly important concern be-cause clinicians may be tempted to use single item mea-sures of complex constructs such as depression, anxiety,or avoidance. Relevant also is that scores across differ-ent response modes do not necessarily converge, as not-ed for the cognitive, behavioral, and physiological re-sponse modes for anxiety (Thorpe & Olson, 1997).

2. To avoid confounding “methods variance” and targetconstruct variance, when possible a construct is bestmeasured using at least two different measurementmethods (e.g., self-report, observer rating; Burns &Haynes, 2006; Campbell & Fiske, 1959; Eid, Lischetz-ke, & Nussbeck, 2006).

3. Each construct needs to be located, both conceptual-ly/theoretically and empirically, in the “nomologicalnet” of other constructs and their measurement instru-ments. An idiographic construct and its scale or itemsshould be justified by explaining how it is not part ofexisting constructs or measurement instruments.



Relevance and Application to CBCF

Despite the critical importance of construct validity in no-mothetic science, including research in psychopathologyand behavior change, construct validity issues have beenall but ignored (with a few exceptions) in CBCF. This maybe because both researchers and practitioners may assumethat construct validity issues have been addressed on thenomothetic level by aggregate-level research. However, asindicated in the section on the person-specific approach,the psychometric structure and validity coefficients fromsuch research may not (and probably do not) apply to theintraindividual relationships hypothesized in CBCFs.

The two most important components of construct valid-ity are convergent and discriminant validity (Campbell &Fiske, 1959). Both of these components are relevant to fourapplications of construct validity issues to CBCF: (1) thefacet and item level, (2) the construct and measure level(Burns & Haynes, 2006), (3) interformulator construct va-lidity, and (4) the construct validity of different approachesto CBCF. Each of these four applications is discussed be-low.

1. Item-level construct validity evaluates the extent towhich different items are tapping into the same versus dif-ferent constructs – convergent and discriminant validity.

Traditional Psychometrics

For between-persons, aggregate-level research, both con-vergent and discriminant validity are readily evaluated us-ing factor loadings. For convergent validity, items withhigher factor loadings are better measures of the construct(i.e., share more variance with the construct). Discriminantvalidity is present if an item intended to measure constructA does not also load on construct B (i.e., is not a double-loading or complex item). Both components of validity canbe evaluated with confirmatory factor analysis (Anderson& Gerbing, 1988; Brown, 2006).


Evaluating the convergent validity of items intended tomeasure a construct in a CBCF is important for a numberof reasons. First, because of the difficulty generalizingfrom aggregate level psychometric structure to the person-specific structure, items validated on the aggregate level donot necessarily validly measure the target construct intra-individually. That is, the construct validity needs to be de-termined within the person-specific framework.

Second, the best set of items to measure a particular con-struct (e.g., depression) will likely vary between persons.Unlike aggregate psychometric studies, the person-specificapproach does not assume that the importance of an itemis the same across different individuals. Item-level evalua-tion of convergent validity selects the “best” item for thatperson – the item with the highest loading in the intraindi-vidual factor analysis (Haynes et al., 2009; Mumma, 2004).

Third, we can ascertain whether certain items may beused to represent the construct, albeit covering a facet thatis less severe or undesirable. For example, when treatingsuicidal behaviors, suicidal thoughts may be less severethan suicidal urges or attempts. Measuring ideation maypermit a more sensitive evaluation of an intervention thanif the full measure was used because urges and attemptsmay be activated primarily under more severe conditions.

Fourth, evaluation of the dynamic (over time) structureof specific items may be useful because a particular con-struct may have some facets that are relatively more stableand others that are more variable over time. For example,some patients may experience a relatively constant or base-line level of depression evident in certain symptoms of de-pression, but may also experience variability in the severityof other depressive symptoms from day to day or in differ-ent contexts. Figure 1 shows a hypothetical result of a con-firmatory dynamic factor analysis of repeated daily ratingsof four depression items. The relatively high lagged load-ing of today’s anhedonia or anergia item scores on yester-day’s depression factor (.70 and .75) indicates that thesetwo facets of depression are relatively more stable overtime. That is, yesterday’s depression substantially influenc-es today’s report of anhedonia and anergia. In contrast, themuch lower lagged loading of today’s sad mood or suicidalthoughts item scores on yesterday’s depression factor (.20and .25) indicates that these two facets of depression varymore from day to day.6 Such variability may lead to under-standing some relevant causal variables – the types of sit-uations or internal states that influence depression severity.Perhaps, for this person, sadness or suicidal thoughts aremore sensitive to changes in specific stressors or hasslesfrom day to day. Also, changes in the item scores for sad-ness and suicidal thought may be more sensitive to initialchanges in depression in response to an intervention thanthe relatively more stable anhedonia or anergia symptoms.The clinician who fails to empirically evaluate these kindsof intraindividual differences in item stability is losing po-tentially important information.

Evaluation of Item-Level Construct Validity

On the individual level, item-level convergent and discrim-inant validity can be evaluated using person-specific meth-



� More precisely, yesterday’s depression factor score explains .702 or 49% of the variability in today’s observed anhedonia score and .752 or56% of the variability in today’s anergia score. In contrast, yesterday’s depression score explains only 4% and 6.25% of the variability intoday’s observed item scores for sad mood and suicidal thoughts, respectively.

ods such as dynamic factor analysis of daily ratings. (SeeMumma, 2004; and Mumma & Mooney, 2007a, for caseexamples using these procedures.) Item-level convergentvalidity is evaluated by inspecting the loadings of the itemson the target construct. Items with poor loadings may bedropped from the scale, while the items with the highestloadings represent the “best” measures of the target con-struct for that person. Dynamic characteristics and stabilityof items can be evaluated using lagged loadings, as de-scribed above. Item-level discriminant validity is evaluatedby allowing these items to also load on other distress factors(e.g., anxiety, anger). Item-level discriminant validity ispresent when an item loads primarily on its target factorwith small loadings on other factors.7

Issues, Limitations, and Implications

All of these procedures assume that the person-specificcontent validity of the items has been established, e.g., thefour items in Figure 1 are relevant to and representative ofdepression symptoms for this person. Once done, person-specific evaluation of item construct validity using dynam-ic factor analysis (as in Figure 1) is important for both idio-graphic items and items based on standardized measures(for reasons described above). However, dynamic factoranalysis assumes equally spaced data and thus may not bevalidly applied to unevenly spaced data, such as when rat-ings are made right after events occurring at irregular in-tervals (e.g., arguments with friends or coworkers). Person-specific methods of analyzing dimensional and dynamicstructure to evaluate the construct validity of unevenlyspaced item scores need to be developed. Also, psychomet-

ric models that bridge person-specific and aggregate levelmodels are needed. For example, a recent model suggestedby Haynes and colleagues (2009) permits the importance(loading) of an item to vary across individuals. For exam-ple, the item “Feel low or blue” may have a high loading(i.e., is a good measure of depression) for one individualbut a relatively low loading for another.

2. Construct and measure-level construct validity is prob-ably familiar to most readers and involves well-knownmethods in traditional psychometrics.

Traditional Psychometrics

Convergent validity involves the extent to which differentmeasures of the same construct converge or agree, partic-ularly if different methods are used (Campbell & Fiske,1959). Discriminant validity involves the extent to whichmeasures of different constructs are separable or distin-guishable, particularly when using the same method (e.g.,self-report questionnaire). Burns and Haynes (2006) ex-panded these notions to apply to multi-informant data col-lected repeatedly over time for aggregate-level research.


Contemporary approaches to convergent validity for inter-individual (aggregate-level) research involve ratings onmultiple traits (constructs and dimensions, such as depres-sion and anxiety) obtained from multiple methods (e.g.,self-report questionnaires, clinician ratings) or multiple

Figure 1. Hypothetical example offour items with concurrent and laggedloadings from today’s factor score andyesterday’s factor score. Anhedoniaand anergia have larger lagged load-ings, indicating higher stability overtime. The numbers next to the lines arefactor loadings.



� Items that double load may be eliminated, or they may be retained if they are conceptualized as an activation link between two constructsor modes such as “concentration problems” might be for depression and anxiety.

sources or informants (e.g., self, parent, teacher) to (hope-fully) converge on a score for each underlying dimension.The data is best analyzed using structural equation model-ing approaches (Brown, 2006; Burns & Haynes, 2006; Eidet al., 2006). This yields correlations between the traits orlatent variables that provide evidence for discriminant va-lidity between the constructs. The stability of the traitscores over time can be evaluated in multiwave longitudi-nal studies.

Analogous construct validity issues for intraindividualdata on measures of case formulation constructs can be de-fined. For example, how strongly are depression and anxi-ety correlated over time within a specific patient? Ideally,this question would be evaluated with both self-report dataand data from other measurement methods. Although mul-timethod or multisource approaches may be readily used indeveloping the CBCF (e.g., interviews, questionnaires,

self-monitoring data), this has, to my knowledge, not beendone for person-specific evaluation of CBCF constructs.

Evaluation of construct validity of variables in theCBCF using person-specific methods focuses not only onconvergent and discriminant validity relationships betweenconstructs of the same type (e.g., anxiety and depression astypes of distress), but also on dynamic relationships. Sev-eral dynamic construct validity issues can be defined. First,evaluation of dynamic structure for a single measure esti-mates the extent that its score at time t predicts the scorethe next time it is measured, time t + 1. If this autocorrela-tion is high, it means scores on the variable are relativelystable over time. Second, dynamic (overtime) relationshipsbetween different variables of the same type (such as mea-sures of distress) are evaluated using crosslagged covari-ances. For example, to what extent does today’s anxietypredict tomorrow’s depression? Such relationships have

Figure 2. Results of confirmatory dy-namic factor analysis of daily ratingsof two idiosyncratic cognitive schema(ICS) hypothesized by the clinicianfor a woman with comorbid mood andanxiety disorders. Dynamic structureis shown by the path from yesterday’sfactor score to today’s factor score,and is stronger for ICS2 (.45) thanICS1 (.32). The concurrent correlationbetween the two ICS factors = .17(from Mumma, 2004).



important implications for the causal model and for treat-ment intervention. For example, Person A may typicallyexperience anxiety before depression, Person B may expe-rience them relatively simultaneously, and Person C maytypically experience them independently of each other. ForPerson A, treating the anxiety initially may decrease thedepression, but this is less likely for Person C. For PersonB, anxiety and depression may share common causal vari-ables, so treatment of those causal variables may decreaseboth.

Evaluation of Measure-Level ConstructValidity Using Person-Specific Methods

Evaluation of convergent and discriminant validity of theconstructs and measures in a CBCF has been done onlyrarely. Figure 2 shows the results of a person-specific con-struct evaluation of two hypothesized idiosyncratic cogni-tive schema (ICS) for a woman with comorbid anxiety anddepression using daily ratings (Mumma, 2004). The latentvariable or factor scores on ICS1 (“I’m stupid and incom-petent”) and ICS2 (“I’m basically lazy and must get every-thing done perfectly”) are mildly correlated at any point intime (r = .17), indicating they are related yet clearly distinctconstructs. ICS2 is more stable than ICS1, with autocorre-lations of .45 and .32, respectively. About 20% (.452) of thevariance in today’s latent (factor) score on ICS2 is predict-ed from yesterday’s score on that latent variable.

Issues and Limitations

As with the evaluation of item-level construct validity,more research is needed to find latent-variable analysismethods suitable for unevenly spaced intraindividual data.However, simpler methods of analysis and briefer mea-sures of constructs can be used in a clinical practice or clin-ical training setting (see Mumma & Fluck, 2010; and Smith& Mumma, 2006 for examples).

Two additional types of construct validity and constructvalidity issues for CBCFs that can be evaluated using per-son-specific methods will be only briefly described due tospace limitations.

3. Interformulator Construct Validity in CBCF


This form of construct validity involves the extent to whichCBCFs independently developed by different formulatorsconverge. Interformulator reliability – when two or moreclinicians formulate the same case – is one aspect of thistype of CBCF construct validity. Both issues are related to

interrater reliability (e.g., of ratings) or interrater agreement(e.g., of diagnoses), but are more complex because the CFis constructed.


Interformulator convergent validity is a somewhat broadernotion than interformulator reliability because it includesconvergence across different types of raters who may varyin perspective, extent of training, etc. For example, Muranand Segal (1992) and Muran and colleagues (1998, 2001)had the formulator, patient, interviewer, and observer ratecomponents of self- and interpersonal scenarios for rele-vance during the formulation development stage.

Studies comparing the CBCFs of experts to novices ortrainees may also address the issue of interformulator con-vergent validity. The few studies of interformulator agree-ment for CBCFs have focused on agreement between cli-nicians on components of the CF, such as problems, issues,and beliefs (Kuyken et al., 2005; Persons & Bertagnolli,1999; Persons, Mooney, & Padesky, 1995). In these studies,all clinicians view the same videotaped materials and areasked to either write the problems, issues, and beliefs – orto rate the presence of certain beliefs from a list. The listor ratings are then compared to a criterion or benchmarkCF developed by an expert in CBCF. Unfortunately, thesestudies simply assume the expert’s formulation is more val-id than the trainees’ formulations. They do not empiricallyevaluate the validity of the expert’s formulation. Otherstudies use raters to evaluate the convergent and discrimi-nant validity of cases formulated by different clinicians(e.g., Mumma & Smith, 2001) or evaluate expert-novicedifferences on measures of formulation quality such as de-gree of elaboration and precision (Eells et al., 2005; Kuy-ken et al., 2005).

Evaluation of Interformulator ConstructValidity for CBCFs Using Person-SpecificMethods

In contrast, although rarely done, interformulator conver-gent validity can be empirically evaluated using personspecific methods. Mumma and Mooney (2007a) comparedthe CBCFs of the same case by an expert and trainee clini-cian. The expert’s formulation was more differentiated interms of the number of idiosyncratic cognitive schema hy-pothesized as well as number of relevant but distinct dis-tress and dysfunction constructs. The convergent and dis-criminant validity of the constructs hypothesized by eachclinician were evaluated using confirmatory dynamic fac-tor analyses of daily ratings. The four idiosyncratic cogni-tive schema hypothesized by the expert clinician demon-strated good convergent and discriminant validity, whereasthe two schema hypothesized by the trainee showed inad-



equate discriminant validity (and were combined into a sin-gle schema for subsequent analyses).

Issues, Limitations, and Implications

The empirical evaluation of convergent/discriminant valid-ity for the constructs and their measures in a CBCF, andhow these differ for the constructs hypothesized by differ-ent formulators, could be a highly useful format to teachCBCF skills in doctoral training programs. Of course, thisis time consuming, requires viewing the same videotapesto develop the formulation, and places additional con-straints on the data collected from the patient during theperson-specific validation (see Mumma & Mooney, 2007a,for more details).

4. Convergence Across Different Methods of CBCF. Asmentioned above, several methods have been proposed fordeveloping CBCFs, including those of Persons (1989,2008), Kuyken and colleagues (2009), and Beck (1995).The extent to which these different methods converge intheir formulation of a particular case has not been studied.However, person-specific methods (such as dynamic factoranalysis) could be used to empirically evaluate the conver-gent and discriminant validity of the constructs included inthe CF developed within each approach.

Validity Issues in the CBCF CausalModel I: Predictive Validity


In traditional psychometrics, criterion validity is typicallybroken down temporally depending upon whether the cri-terion to be predicted is in the present (concurrent validity)or the future (predictive validity). For example, scores ona standardized intelligence test for a 14-year-old may con-currently predict performance on achievement tests andpresent grades in classes, or they may predict future aca-demic performance, such as grades as a senior in highschool. Incremental predictive validity refers to whetheradditional predictors significantly improve prediction ofthe criterion.


First, in behavioral CFs, predictive validity is a central is-sue in functional-analytic approaches to behavioral CF(e.g., Haynes & O’Brien, 1990, 2000; Iwata et al., 1994;Sturmey, 2008). The clinician assesses the role of variousstimuli and operant contingencies in eliciting or maintain-

ing certain behaviors. This functional analysis has directrelevance for treatment planning – the individual’s re-sponse to various interventions is predicted based on thefunctional CF, and interventions are then selected to opti-mize the chances of achieving targeted goals (Nelson-Gray,2003).

Although less frequently discussed (compared to func-tional analytic approaches to behavioral CF), predictive va-lidity in CBCF focuses on hypothesized relationships in thecausal model – predicted relationships between scores onoutcome measures of distress or dysfunction and one ormore causal variables such as environmental triggers andcognitions. The hypothesized predicted relationships mayinvolve a single predictor (a simple or bivariate relation-ship), several predictors (multiple), incremental prediction(e.g., do cognitions predict distress over and above envi-ronmental triggers?), moderated prediction (e.g., do certainthoughts trigger distress only when the person is in a certainsituation or state?), or meditational relationships (e.g., doesa comment by a coworker trigger a depressive reaction be-cause it activates beliefs of worthlessness?).

Predictive validity can be divided into intra- versus ex-trasession predictions: predicting the person’s response toevents occurring within a session versus behavior externalto the session. Predictions of response to interventions, atreatment validity issue, are discussed below.

Many of the hypotheses in the causal model part of theCBCF involve predictions of the effects of triggering,maintaining, and/or mediating variables on outcome vari-ables, such as different kinds of thoughts or beliefs predict-ing depression versus anxiety. Predictive validity refers tothe accuracy of these predictive hypotheses. It involves em-pirically testing the CBCF predictions. The target behavioris considered the criterion. However, it is important to di-rectly and empirically measure both criterion and predic-tors (Mumma, in press). Although helpful and useful fordeveloping predictive CBCF hypotheses, simply using aclient’s report of the behavior from an interview is insuffi-cient for evaluating predictive validity.

Predictive validity on the aggregate level: Prediction ofbehavior is an important function of nomothetic models ofpsychopathology as well as certain standardized psycho-logical tests (e.g., tests of intelligence, the MMPI). Actuar-ial or statistical prediction models can be useful for predict-ing certain outcomes (e.g., differences between individualsin severity of suicidal ideation or urges) from a set of pre-dictors. The predictors are trait-like or relatively stablecharacteristics of individuals, such as gender, age, frequen-cy and intensity of alcohol use, level of hopelessness, etc.This results in a predicted severity of suicidal ideation/urg-es for each person (i) depending on their level of each pre-dictor (see equation 1):

Yi’ = A + B1X1i + B2X2i + B3X3i + B4X4i (1)

where Yi’ = predicted severity of suicidal ideation/urges forperson i



B1 to B4 are regression coefficientsX1i to X4i are the scores of the ith person on subject char-acteristics such as age, gender, alcohol use, hopelessness,andA is a constant

It is widely recognized that such actuarial prediction mod-els are generally more consistent and accurate than predic-tions based on clinical judgments (Garb, 1998; Grove et al.,2000).

To summarize, a between-person formulation or modelof psychopathology can be used to form an interindividualprediction model (equation 1). This yields a score for aperson, say Susan, that represents her general or overallpredicted level of suicidal ideation/urges. This kind of pre-diction is useful for making general treatment decisions,such as whether Susan needs to be hospitalized or can betreated with outpatient therapy.

Predictive validity on the individual level: The predic-tive validity of a CBCF can be evaluated by the extent towhich it makes an incremental contribution to the predic-tion of the behavior or functioning of a particular individ-ual, over and above that available from such nomotheticprediction models. That is, the CBCF should predict a per-son’s behavior with greater specificity and precision thanwould be obtained from aggregate-level, between-personsmodels (such as equation 1). For example, a CBCF canincrease the precision and specificity of prediction of sui-cidal ideation/urges by focusing on specific and even high-ly idiosyncratic circumstances that activate such thoughtsor urges for a particular individual. For example, Susanmight find interactions with a verbally abusive mother-in-law particularly distressing, increasing the severity of hersuicidal ideation/urges, whereas a very different set of cir-cumstances may predict increased suicidal ideation or urg-es for another individual.

Consider a set of external or internal circumstances orconditions that may trigger suicidal ideation or urges forSusan. These are included in her CBCF by the clinician. Aperson-specific, intraindividual statistical prediction mod-el can be derived from this formulation. Here, we are pre-dicting the severity of suicidal ideation/urges for Susan(who is person i = 1) on particular occasions (where j = 1is occasion 1, j = 2 is occasion 2, etc.). This prediction ismade using her scores on several predictor variables usingequation 2:

yij’ = Yi’ + b1ix1ij + b2ix2ij + b3ix3ij (2)

where yij’ is the predicted severity of suicidal ideation/urg-es for person i on occasion j,Yi’ is the predicted general (average) level of suicidal idea-tion for person i from equation (1),b1i to b3i are the regression weights for the intraindividualprediction for person i on her three predictorsx1ij to x3ij are her scores on the three predictors x1 to x3(these predictors may be unique to person i); these scores

vary from occasion to occasion, andj is the occasion.

So, for Susan, who is person 1 (i = 1), her general level ofsuicidal ideation/urges is Y1’ (as estimated from the inter-individual equation 1). Let’s assume Y1’ = 2.15 for Susan.Susan’s three predictors are x11j = how depressed she hasfelt during the past hour, x21j = whether she’s talked withher mother-in-law during the past hour (scored 1) or not(scored 0), and x31j is whether she’s at work (scored 0) ornot (scored 1). Again, x11 to x31 are predictors in Susan’sCBCF. The clinician predicted that her suicidal idea-tion/urges will be more severe when she’s feeling more de-pressed (x11j), has had a recent interaction with her moth-er-in-law(x21j = 1), and is not at work (x31j = 1).

Note that x12j, x22j, and x32j are predictors for a secondperson James as specified in James’ CBCF. (The first sub-script for each predictor variable refers to person 2, who isJames.) The predictors for James may be quite differentthan those for Susan. The CBCF tells us what the predictorsare for each person.

Returning to Susan: Susan completes ratings on herthree predictor variables and on the strength of her suicidalthoughts/urges (rated 0 = absent to 10 = extreme) duringmultiple occasions over a 2-week period. From this data,the intraindividual regression coefficients b11, b21, and b31

(where the subscript for each coefficient refers to person 1who is Susan) can be calculated (using a dynamic regres-sion model appropriate for this kind of data). Let’s say Su-san’s intraindividual statistical prediction model is:

y1j’ = 2.15 + .5x11j + 3x21j + 1.25x31j (3)

Once the data have been collected and analyzed, this modelis a person-specific validity evaluation for the clinician’sthree CBCF hypotheses concerning triggers for Susan’ssuicidal ideation or urges. Using (3), Susan and her clini-cian can predict how strong her suicidal thoughts/urges willbe on a new occasion. They just need to know her scoreson her three predictors on that occasion. For example,Tuesday at 3:00 PM Susan is feeling mildly depressed (x11j

= 3, her rating on a 0 to 10 scale), she has just talked withher mother-in-law on the phone (x21j = 1), and she is atwork (x31j = 0). Her predicted level of suicidal idea-tion/urges on that occasion is:

y1j’ = 2.15 + (.5)(3) + 3(1) + 1.25(0) = 6.65

on a scale from 0 = absent to 10 = extreme.The important point is that the variables included in this

intraindividual statistical prediction model for Susan areobtained from Susan’s CBCF. The CF defines the outcomevariables relevant for Susan (such as suicidal ideation/urg-es) and includes hypotheses about what causal variables arerelevant for this outcome variable as well as what variablestrigger and maintain her suicidal ideation/urges. The clini-cian develops this prediction model using the relevant vari-ables from Susan’s CBCF and then tests that part of the CF



causal model for Susan. The model has different outcomevariables and different predictors for Susan than for anotherperson. The CF fine-tunes the prediction of Susan’s suicidalideation/urges, by increasing the specificity, relevance, andrepresentativeness of triggering variables for her. That is,it specifies the causal variables most relevant for her. Thisis important because in aggregate-level, nomothetic re-search, highly specific variables such as whether Susan hastalked with her mother-in-law during the past hour (x21j inequations 2 and 3) or whether she’s at work or not (x31j)may not predict enough of the variance in the outcomemeasure (suicide ideation/urges) to be clinically or statisti-cally significant. However, highly specific and idiosyncrat-ic external and internal conditions can be assessed idio-graphically and, when included in the CBCF, may havestrong predictive validity for a particular client (Mumma,2001, 2004).

To summarize: Collecting the data for, and calculatingSusan’s intraindividual statistical prediction model (equa-tion 3), evaluates the predictive validity of the part of Su-san’s CBCF that focuses on the causal variables for hersuicidal ideation/urges. It is a person-specific validity eval-uation of that part of her CBCF.

The predictive validity of a CBCF also refers to the typeand form of the predicted relationships. Does the magni-tude and type of relationship hypothesized in the CBCFaccurately represent the type of statistical relationship be-tween the variables as it occurs for this person in his/herdaily functioning? For example, perhaps Susan’s contactwith her abusive mother-in-law triggers suicidal idea-tion/urges that are particularly strongly when she has beenfeeling more depressed than usual. Susan’s CBCF shouldrepresent this as a moderated relationship, an interactionwith a hypothesized effect above and beyond the main ef-fects of both contact with her mother-in-law and howstrong her feelings of depression were during the past hour.The validity of this CBCF hypothesis can be evaluated us-ing the following intraindividual statistical prediction mod-el for Susan:

y1j’ = Y1’ + b11 × 11j + + b21 × 21j + + b31 × 31j +b41(x11j * x21j) (4)

wherex11j is her depression rating for last hour,x21j is whether or not she’s talked with her mother-in-lawduring the past hour,x31j is whether she is at work or not, andx11j * x21j is the interaction of depression and talking withmother-in-law.

Assuming b11, b21, and b31 remain the same and b41 isestimated to be .333, we would now predict the strength ofSusan’s suicidal ideation/urges to be:

y1j’ = 2.15 + (.5)(3) + 3(1) + 1.25(0) + .333(3*1)= 7.65 (5)

Finally, the CBCF should accurately represent the forms ofthe relationships between causal and target variables. Per-haps the patient’s suicidal ideation is triggered only if thecontact with her mother-in-law intensifies to the point thatshouting occurs. Does the CBCF represent this as a nonlin-ear relationship such as a step function where suicidal idea-tion remains low but becomes abruptly strong when a cer-tain level of intensity in the argument is reached? (SeeHaynes, 1992; Haynes & O’Brien, 2000, for further discus-sion of types and forms of causal relationships.)

Note that the above approach examines the CBCF andthe intraindividual or person-specific statistical predictionmodel based on the CF in terms of incremental predictionof Susan’s suicidal ideation/urges. That is, the prediction isincremental to what is provided by aggregate models ofpsychopathology or the interindividual statistical predic-tion in Equation 1. Of course, predictions for Susan couldbe made based on the CBCF and the intraindividual dataonly.8 These predictions are not necessarily incremental tothe aggregate level prediction.

Evaluation of CBCF Predictive Validity UsingPerson-Specific Methods

Unfortunately, relatively little empirical work has beendone to evaluate hypothesized relationships between cog-nitions and distress or symptoms in CBCFs, although anumber of investigators have highlighted the need for thisresearch (Bieling & Kuyken, 2003; Kuyken, 2006; Kuykenet al., 2009).

Recently, Mumma and colleagues (Mumma, 2004;Mumma & Mooney, 2007a, 2007b) demonstrated how thepredictive validity of hypothesized CBCF relationships forcognitions and stressful events predicting severity and typeof distress (depression, anxiety) can be evaluated using per-son-specific methods and intraindividual statistical predic-tion models such as those discussed in the previous subsec-tion. Daily ratings were obtained on measures of anxietyand depression as well as on the particular thoughts andbeliefs unique to that person that were hypothesized tomaintain these two types of distress (called idiosyncraticcognitive schema; Clark & Beck, 1999). The daily ratingswere analyzed with dynamic time series regression andconfirmatory dynamic factor analysis. Figure 3 shows howdaily variability in depression and anxiety scores was pre-dicted by specific hypothesized idiosyncratic cognitiveschema for a woman with comorbid mood and anxiety dis-orders. The magnitude of the relationship obtained from thedynamic regression analysis is summarized by R2 – the pro-portion of daily variance in the outcome variable (e.g., de-



Specifically, in Equation 2 or Equation 4, the Y1’ is replaced by a constant estimated solely from the intraindividual data.

pression) explained by the predictor. Since there is a singlecognition predictor for anxiety and depression, the R2 cor-responds to r = .61 and .66, respectively. Using Cohen’s(1988) criteria, both effect sizes are large (r > .50).

Alternatively, Figure 4 shows how predicted relation-ships in the CBCF between several cognitive schema andseveral types of distress were evaluated in a single analysis

using confirmatory dynamic factor analysis for a man withmajor depressive disorder and generalized anxiety disorder.A glance at this figure shows the similarity of this person-specific approach using daily ratings to the familiar struc-tural equation modeling diagram. For example, hypothe-sized relationships are tested between latent variables,which correct for attenuation and bias due to measurement

Figure 3. Predictive validity test oftwo hypothesized relationships be-tween (detrended) daily ratings of dis-tress by two idiosyncratic cognitiveschema. Top: anxiety predicted byICS1 “I’m stupid and incompetent”with R2 = .37. Bottom: depression pre-dicted by ICS3 “It’s futile and hope-less. I’ll never be good enough” withR2 = .436. Solid lines are the (detrend-ed) observed MASQ subscale score.Dotted lines are the predicted anxietyor depression score from the cognitionscores (from Mumma, 2004.).

Figure 4. Evaluation of predictive va-lidity of the expert clinician’s CBCFfor concurrent prediction of depres-sion, mixed anxiety symptoms, andanger from four idiosyncratic cogni-tive schema (ICS) using confirmatorydynamic factor analysis. Standardizedsolution. χ2(df = 531) = 815.19,RMSEA = .083, NNFI = .85, CFI =.89. Concurrent regressions from thelatent ICSs to the latent distress vari-ables are shown by solid lines; dynam-ic loadings of ICS item parcels areshown by dotted lines. GD: generaldistress. Note: Lagged regressions be-tween the ICSs, disturbances of theICS and distress factors (and their co-variances within each set), and errorvariances for item parcels are notshown (from Mumma & Mooney,2007a).



error. Tests of CBCF predictions using the person-specificapproach may also include crosslagged relationships be-tween causal and outcome variables. For example, thoughtsof worthlessness today may predict depression tomorrow.

Issues, Limitations and Implications

One limitation of the above approaches to person-specificvalidation of CBCF predictions is that data collection ide-ally should be done for about 60 or more occasions to ob-tain statistically valid results. However, in training or prac-tice contexts, the length of data collection can be reducedand simpler analyses used (Mumma & Fluck, 2010; Smith& Mumma, 2006). An alternative approach is to use eco-logical momentary assessment, involving data collectionmultiple times a day, in real time, typically using personaldata assistants. This approach is particularly interesting be-cause it can provide more fine-grained, sensitive tests ofrelationships predicted in the CBCF because the ratings aremade in real time and are less subject to the biasing influ-ences of retrospective recall (Stone, Shiffman, Atienza, &Nebeling, 2007). Furthermore, a major practical advantageis that CBCF predictions can be tested using data collectedover just 3 or so weeks. A limitation of this approach, how-ever, is that the data may be unevenly spaced,9 therebycomplicating the analyses. However, Riggins and Mumma(2008; Riggins, 2009) showed how such data can be ana-lyzed using readily available multilevel modeling software.Further advances in statistical analysis approaches for un-evenly spaced data are needed to further increase the abilityto evaluate predictive validity in CBCFs.

Predictive validity is perhaps more difficult to evaluatefor CBCFs than behavioral CFs for several reasons. Thebehavioral clinician may be able to manipulate the anteced-ent conditions (e.g., potential discriminative stimuli), thecontingencies, or both so as to provide relatively immediateand unambiguous information about their function for thetarget behavior (see, e.g., Iwata et al., 1994). A behavioralCF with high predictive validity will specify the antecedentcircumstances and operant contingencies that maximallypredict the target behavior. In contrast, it is more difficultto manipulate a person’s thoughts and beliefs. Changingappraisals, interpretations, and/or attributions may takemany sessions or even months of cognitive therapy.

Because stimulus and reinforcing conditions can be ma-nipulated, behavioral CFs based on a functional analysiscan often be evaluated using single-subject experimentaldesigns such as ABAB reversal and multiple baseline de-signs (see, e.g., Kazdin, 2003; Petermann & Müller, 2001).These designs can be relatively strong on internal validity– the causal role of a particular antecedent or contingencycan be evaluated with little ambiguity. Because cognitions

are less readily manipulated, passive-observational designsusing daily ratings or ecological momentary assessmentneed to be used to test the predictive validity of CBCFs (seeFigures 3 and 4). This requires data collection over severalweeks or even months and more complicated statisticalanalyses. Finally, the results are weaker in making causalinferences (internal validity).

Research implications: Predicting specific behaviors isan important part of a functional analysis and is includedin many types of behavioral formulations. Treatment plan-ning can then be based on these predictions. However,CBCFs have generally been less focused on prediction. Ag-gregate-level studies comparing nomothetic predictions toformulation-based predictions are needed to directly ad-dress the core incremental validity issue in evaluating thepredictive validity of CBCFs.

Expert-novice differences in the predictive validity ofCBCFs have received little attention, although person-specif-ic methods can be used to empirically investigate such differ-ences by comparing the CBCF of the same case developedby an expert and novice clinician(s). For example, Mummaand Mooney (2007a) compared the predictive validity ofCBCFs of the same case that were independently developedby an expert clinician and a trainee, using the same data (in-terview videotapes and self-report measures). Compared tothe novice clinician, the expert’s formulation explained, onaverage, about twice the proportion of variability in the cli-ent’s daily ratings of depression and anxiety.

To summarize: The utility of the person-specific approachfor testing and evaluating the validity of predictions andrelationships hypothesized in the CBCF is potentially enor-mous. The empirical work so far has only scratched thesurface. Predictions involving moderator and mediatingvariables are highly relevant for CBCFs, yet their validityremains to be empirically evaluated using person-specificmethods. For example, a CBCF mediational hypothesismay state that a certain situation (e.g., argument with a su-pervisor or boss) may trigger distress (e.g., predict depres-sion), but that this relationship is mediated throughthoughts of failure. The validity of this hypothesis can betested with data collected over time on that particular indi-vidual.

The CBCF specifies outcome and causal variables thatare relevant and specific to a particular individual. In-deed, both outcome and causal variables may be quiteidiosyncratic, context-embedded, or unique to that indi-vidual. The predictive validity of CBCF hypotheses canbe tested using the person-specific methods described inthis section. One potential result of using these person-specific methods is to create a person-specific or an in-traindividual statistical prediction model, as describedabove (see Mumma, 2001). This is potentially very im-



For example, the person may make a scheduled rating upon awakening, another scheduled rating 3 hours later at lunch, a rating 1 hour laterwhen a stressful event has occurred, a scheduled rating 5 hours later after dinner, and another stressful event-related rating 2 hours later.

portant because such a prediction system has the sameadvantages over clinical judgment that Meehl originallydescribed in 1954, a conclusion repeatedly bolstered bydozens of empirical studies (see Arkes, 1981; Dawes &Corrigan, 1974; Garb, 1998; Grove et al., 2000; Meehl,1986; Wedding & Faust, 1989 for reviews).

Validity Issues for the CBCF CausalModel II: Treatment Related Validity

Some issues relevant to treatment validity or treatment util-ity of CBCFs evaluated using between-groups experimentswere mentioned above. Treatment-related validity issuesthat can be addressed using intraindividual or person-spe-cific approaches are discussed briefly below.

Predicting Response to an Intervention

A CBCF can be used to predict the client’s response to anintervention or set of interventions (as in a treatment pack-age). Although similar to predictive validity (discussedabove), here the prediction focuses on the client’s re-sponse to an (intentional) intervention. Predictions can beeither of the client’s response to the clinician’s interven-tions within the session or outside of the session. Table 1shows examples of the various combinations of predictivevalidity (top rows) and compares this to this type of treat-ment validity (bottom rows) both within and external tothe session. Predicting the client’s response to interven-tions has important utility for treatment planning and out-come. For example, Persons (2008; Persons et al., 2001)suggests that the CBCF can be used to predict potentialobstacles to treatment based on the client’s earlier re-sponse to interventions. Despite the potential utility ofsuch CBCF-based predictions, evaluating the validity ofthese types of predictions has to date not been formallystudied in cognitive behavior therapy.

Predicting the client’s response to an intervention iscentral to behavioral CF. The clinician conducts a func-tional assessment to predict the client’s response to dif-ferent types of situations and contingencies, selects theintervention most likely to work, and predicts the client’sresponse. These interventions may be intrasession or mayinvolve changing the antecedents and contingencies inthe natural environment. For example, based on the func-tional analysis and behavioral CF, the clinician may pre-dict a child’s response to an intervention by parents orteachers (Hoff, Ervin, & Friman, 2005; Iwata et al., 1994;Kearney & Silverman, 1999; McMahon, 1999). Behav-ioral intervention studies evaluate predicted response tointerventions using single-subject experimental designs,as mentioned above. Multiple-baseline designs are par-

ticularly useful because they can have strong internal va-lidity with fewer ethical problems than reversal designs.

Evaluation of this type of treatment-related validity inCBCF using person-specific methods is relativelystraightforward. The patient’s response to the interven-tion is assessed and compared to the predicted response.However, this cannot be adequately evaluated without anintraindividual baseline of the target behavior (e.g., anx-iety) prior to the intervention. Formal studies of this typeof treatment validity issue for CBCF and treatment areneeded.

Are Therapy Processes or InterventionsCongruent with the CBCF More Beneficial?

The importance of evaluating treatment adherence in CBTtreatment protocols has been recognized for some time(Waltz, Addis, Koerner, & Jacobson, 1993). But what arethe effects of therapist interventions that are congruentversus incongruent with the CBCF? Interestingly, a fewstudies have evaluated this type of treatment validity usingboth aggregate and intraindividual designs for psychody-namic formulations (e.g., Crits-Christoph, Cooper, & Lu-borsky, 1988; Messer, Tishby, & Spillman, 1992; Silber-schatz, Fretter, & Curtis, 1986). Several studies also eval-uated the match of the therapy package withcharacteristics of the patients, such as depression relatedto social skill deficits versus negative cognitions (Mc-Knight, Nelson, Hayes, & Jarrett, 1984; Nelson-Gray,2003; Nelson-Gray, Herbert, Herbert, Sigmon, & Bran-non, 1989). However, the effects on treatment outcome ofcongruence of the interventions with the CBCF has notbeen empirically evaluated. This could be done so usingperson-specific methods.

Table 1. Examples of predictive and treatment validity: byevent and patient response

Inter-vention

Event Patient behavior/response

Intrasession Extrasession

No Intrasession Patient cries (event)then withdraws (re-sponse)

Patient cries (event)then goes home anddrinks (response)

Extrasession Patient cries (re-sponse) as reports in-cident with boss(event)

Patient drinks (re-sponse) after inci-dent with boss(event)

Yes Intrasession Patient rejects cogni-tive reframe

Patient angry aboutreframe after session

Extrasession Patient angry nextsession after tryinghomework

Patient does not doassigned homework

Event: does the event occur within the session (intra) or outside of thesession (extra)?



Accuracy of the CBCF and Progress, Change,or Outcome

Some developers of CBCF manuals suggest that monitor-ing response to treatment is the most important method forevaluating the validity of the CBCF (e.g., Persons et al.,2001; Persons & Tompkins, 2007). Clearly, the ultimategoal of the formulation is to improve clinical judgmentsand decision making so as to optimize treatment outcome(Haynes & O’Brien, 2000). However, there are a numberof additional reasons why the client may or may not im-prove (on outcome measures) that are distinct from thequality of the CBCF (Aston, 2009; Mumma, 1998). Theseinclude nonspecific effects such as expectancies of both thetherapist and the client and the quality of the therapeuticalliance, the extent to which the interventions selected weretheoretically and empirically relevant and optimal giventhe formulation, and the extent to which the interventionsas actually implemented adhered to the formulation-basedtreatment plan and were implemented in a competent man-ner. Most clinicians are trained to appreciate internal valid-ity limitations of single-group research. Application of thisskepticism to explaining change (or lack thereof) duringtreatment would be beneficial.

Comparison to Formal or StructuralCharacterizations of CFs

The validity concerns that can be evaluated using person-specific or intraindividual methods differ from, but are notincompatible with, CF validation approaches that evaluateformulation quality using methods such as those developedby Eells and colleagues (2005) and Kuyken and colleagues(2006). For example, a CF can be reliably rated for its de-gree of complexity, elaboration, degree of inference, or pre-cision of language. Such evaluations require judges to ratethe CF on these dimensions, whereas validity evaluationusing the person-specific approach does not require judges,but rather the cooperation of a motivated patient to collectdata over time.

General Issues in Evaluating theValidity of CBCFs

Sampling Plan

In terms of the utility of a CBCF for predicting the func-tioning of a person, a method for sampling the client’s be-havior which is optimally sensitive for testing the CF hy-potheses for that person needs to be agreed upon by theclinician and the patient. The sampling plan refers to thefrequency and circumstances (occasions/contexts) under

which the client makes ratings on the individualized ques-tionnaire. Some general guidelines:1. Hypotheses about maintaining variables operating over

days may be best evaluated using daily ratings. Exam-ple: The person is depressed or anxious for days orweeks at a time, there is variability in the severity, but itis probably not highly context dependent.

2. Hypotheses about maintaining variables operating overhours (e.g., the person’s anxiety or depression variesduring the day and may be context dependent) may bebest evaluated using ecological momentary assessmentat predetermined convenient times or intervals duringthe day.

3. Hypotheses about triggering variables (hassles, stressor,or negative thoughts) that elicit distress may be bestevaluated when ratings are done soon after the trigger.Example: The person becomes depressed after certaininteractions with family members or coworkers. Ideally,all interactions would be sampled, not just those fol-lowed by depression (see Steiger, Gauvin, Jabalpurwala,Séquin, & Stotland, 1999).

Frequency of Conducting CBCF ValidityEvaluation Using the Person-SpecificApproach

Applying these principles of CBCF validation using theseperson-specific methods is time consuming and requiresthe time and cooperation of an interested and motivatedpatient. A collaborative approach to developing the formu-lation, such as that described by Kuyken and colleagues(2008), is important when evaluating the validity of theformulation. Application of these methods with selectedcases in clinical training should be advantageous not onlyfor those cases selected, but may help the trainee under-stand how to conduct a person-specific validity evaluationof a CBCF for other patients. It is also useful to establishhabits of evaluation using person-specific methods that canthen be applied to clinical practice. Practitioners may ben-efit from conducting a person-specific validity evaluationon certain cases, such as a certain number or percentage ofcases in which treatment was formulation based. Cognitivebehavioral therapists often involve their patients in someform of ongoing data collection, and these procedurescould be modified or supplemented to address CF valida-tion issues. Modifications and simplifications of the aboveapproach for evaluating CBCF validity are available for usein clinical training and practice settings (see, e.g., Mumma& Fluck, 2010; Smith & Mumma, 2006).

Reactivity

Self-report measures collected daily or several times a dayare potentially reactive. This reactivity may at times be



beneficial; however, the clinician should be sensitive topossible negative reactions and should keep in mind thatthe data could be influenced by such reactive effects.

General Summary and Conclusions

The CBCF is an idiographic theory of the person or per-son/situation which explains the person’s distress and dys-function and has relevance to treatment planning. It is thebasis for individualized or formulation-based treatment.There is an increasing interest in and now manuals for de-veloping CBCFs. Yet, there is a paucity of research evalu-ating the validity of these theories of the person-situation,despite multiple calls to do so. Perhaps it is now time togive more attention to testing and evaluating the validity ofthese CBCFs.

Empirically validating CBCFs is important for clinicalresearch. More studies comparing formulation-based tomanualized cognitive behavior therapy are needed. How-ever, as discussed above, some of the significant designproblems with studies to date could be addressed by includ-ing procedures to validate the CBCFs in the formulation-based treatment condition using the person-specific meth-ods described in this article.

Evaluation of CBCFs during clinical training providesan important type of feedback on the accuracy and validityof the CF for trainees. Clinical judgment improves withfeedback, particularly if the feedback is quick, relevant,and accurate (Garb, 1998). Empirical feedback on the va-lidity of the clinician’s CBCF has typically not been avail-able to trainees, but could be if the validation proceduresdescribed in this article were to be used. CBT practitionersmay also benefit from these procedures being applied to afew of their complex and comorbid cases where treatmentis formulation based.

Wilson (1996) argued that a major concern for formula-tion-based approaches to treatment is that the CF is a com-plex clinical judgment vulnerable to judgmental and infer-ential biases. Standardized treatments are better, he argues,because they are based on statistical prediction models:Given the disorder, the best prediction of favorable treat-ment response is to use the empirically validated manual-ized treatment. However, the advantages of actuarial pre-diction do not accrue solely to aggregate level predictionmodels. The CBCF presents an opportunity for the clini-cian to develop a statistical prediction model of high rele-vance and specificity for his or her particular client. Buttesting these predictions, that is, evaluating the predictivevalidity of the formulation, is a critical step that has all butbeen ignored. However, with the advent of ecological mo-mentary assessment and portable data collection devices,the clinician, working together with the client, now has theopportunity to test these predictions and thus, potentially,to develop a person-specific or intraindividual statisticalprediction model tailored to the issues and life circumstanc-

es of the particular individual. Of course, research compar-ing the efficacy of manualized treatment to treatment basedon CBCFs that have been validated using person-specificmethods is an important next step.

References

Allport, G. W. (1962). The general and the unique in psycholog-ical science. Journal of Personality, 30, 405–422.

Anderson, J. C., & Gerbing, D. W. (1988). Structural equationmodeling in practice: A review and recommended two-stepapproach. Psychological Bulletin, 3, 411–423.

Arkes, H. R. (1981). Impediments to accurate clinical judgmentand possible ways to minimize their impact. Journal of Con-sulting and Clinical Psychology, 49, 323–330.

Aston, R. (2009). A literature review exploring the efficacy ofcase formulations in clinical practice. What are the themes andpertinent issues? Cognitive Behavior Therapist, 2, 63–74.

Beck, J. S. (1995). Cognitive therapy: Basics and beyond. NewYork: Guilford.

Bieling, P. J., & Kuyken, W. (2003). Is cognitive case formulationscience or science fiction? Clinical Psychology: Science andPractice, 10, 52–69.

Brown, T. A. (2006). Confirmatory factor analysis for appliedresearch. New York: Guilford.

Burns, G. L., & Haynes, S. N. (2006). Clinical psychology: Con-struct validation with multiple sources of information and mul-tiple settings. In M. Eid & E. Diener (Eds.), Handbook of mul-timethod measurement in psychology. Washington, DC:American Psychological Association.

Campbell, D., & Fiske, D. (1959). Convergent and discriminantvalidation by the multitrait-multimethod matrix. Psychologi-cal Bulletin, 56, 81–105.

Chadwick, P., Williams, C., & Mackenzie, J. (2003). Impact ofcase formulation in cognitive behavior therapy for psychosis.Behavior, Research and Therapy, 41, 671–680.

Chambless, D. L. (1996). In defense of dissemination of empiri-cally supported psychological interventions. Clinical Psychol-ogy: Science and Practice, 3, 230–235.

Clark, D. A., & Beck, A. T. (1999). Scientific foundations of cog-nitive theory and therapy of depression. New York: Wiley.

Clark, D. A., & Beck, A. T. (2010). Cognitive therapy of anxietydisorders: Science and practice. New York: Guilford.

Cohen, J. (1988). Statistical power analysis for the behavioralsciences (2nd ed.). Hillsdale, NJ: Erlbaum.

Cole, D. A., & Maxwell, S. E. (2003). Testing mediational modelswith longitudinal data: Questions and tips in the use of struc-tural equation modeling. Journal of Abnormal Psychology,112, 558.

Cook, T. D., & Campbell, D. T. (1979). Quasiexperimentation:Design & analysis issues for field settings. Chicago: Rand Mc-Nally.

Crits-Christoph, P., Cooper, A., & Luborsky, L. (1988). The ac-curacy of therapists’ interpretations and the outcome of dy-namic psychotherapy. Journal of Consulting and Clinical Psy-chology, 56, 490–495.

Dawes, R., & Corrigan, B. (1974). Linear models in decisionmaking. Psychological Bulletin, 81, 95–106.



Eells, T. D., Lombart, K. G., & Kendjelic, E. M. (2005). The qual-ity of psychotherapy case formulations: A comparison of ex-pert, experienced, and novice cognitive-behavioral and psy-chodynamic therapists. Journal of Consulting and ClinicalPsychology, 73, 579–589.

Eid, M., Lischetzke, T., & Nussbeck, F. W. (2006). Structuralequation models for multitrait-multimethod data. In M. Eid &E. Diener (Eds.), Multimethod measurement in psychology(pp. 283–300). Washington, DC: American Psychological As-sociation.

Emmelkamp, P. M., Bouman, T. K., & Blaauw, E. (1994). Indi-vidualized versus standardized therapy: A comparative evalu-ation with obsessive-compulsive patients. Clinical Psychologyand Psychotherapy, 1, 95–100.

Ferrer, E., & Nesselroade, J. R. (2003). Modeling affective pro-cesses in dyadic relations via dynamic factor analysis. Emo-tion, 3, 344–360.

Garb, H. N. (1998). Studying the clinician: Judgment researchand psychological assessment. Washington, DC: AmericanPsychological Association.

Ghaderi, A. (2006). Does individualization matter? A randomizedtrial of standardized (focused) versus individualized (broad)cognitive behavior therapy for bulimia nervosa. Behavior Re-search and Therapy, 44, 273–288.

Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson,C. (2000). Clinical versus mechanical prediction: A meta-anal-ysis. Psychological Assessment, 12, 19–30.

Haynes, S. N. (1992). Models of causality in psychopathology:Toward dynamic, synthetic, and nonlinear models of behaviordisorders. New York: Macmillan.

Haynes, S. N., Kaholokula, J. K., & Nelson, K. (1999). The idio-graphic application of nomothetic, empirically based treat-ments. Clinical Psychology: Science and Practice, 6, 456–461.

Haynes, S. N., & Lench, H. C. (2003). Incremental validity of newclinical assessment measures. Psychological Assessment, 15,456–466.

Haynes, S. N., Mumma, G. H., & Pinson, C. (2009). Idiographicassessment: Conceptual and psychometric foundations of in-dividualized behavioral assessment. Clinical Psychology Re-view, 29, 179–191.

Haynes, S. N., & O’Brien, W. H. (1990). Functional analysis inbehavior therapy. Clinical Psychology Review, 10, 649–668.

Haynes, S. N., & O’Brien, W. H. (2000). Principles and practiceof behavioral assessment. New York: Plenum.

Hoff, K., Ervin, R., & Friman, P. (2005) Refining functional be-havioral assessment: Analyzing the separate and combined ef-fects of hypothesized controlling variables during ongoingclassroom routines. School Psychology Review, 34(1), 45–57.

Hokanson, J. E., Tate, R. L., Niu, X., Stader, S., & Flynn, H.(1994). Illustration of concomitant time series analyses in acase of somatoform disorder. Cognitive Therapy and Re-search, 18, 413–437.

Horowitz, M. J., & Eells, T. D. (2007). Configurational analysis:States of mind, person schemas, and the control of ideas andaffect. In T. D. Eells (Ed.), Handbook of psychotherapy caseformulation (2nd ed., pp. 136–163). New York: Guilford.

Iwata, B. A., Pace, G. M., Dorsey, M. F., Zarcone, J. R., Vollmer,B., & Smith, J. (1994). The function of self-injurious behavior:An experimental-epidemiological analysis. Journal of AppliedBehavior Analysis, 27, 215–240.

Jacobson, N. S., Schmaling, K. B., Holtzworth-Munroe, A., Katt,

J. L., Wood, L. F., & Follette, V. M. (1989). Research-struc-tured vs. clinically flexible versions of social learning-basedmarital therapy. Behavior Research and Therapy, 27, 173–180.

Jose, A., & Goldfried, M. (2008). A transtheoretical approach tocase formulation. Cognitive and Behavioral Practice, 15,212–222.

Kazdin, A. E. (2003). Research design in clinical psychology (4thed.). Boston: Allyn and Bacon.

Kearney, C. A., & Silverman, W. K. (1999). Functionally basedprescriptive and nonprescriptive treatment for children and ad-olescents with school refusal behavior. Behavior Therapy, 30,673–695.

Korchin, S. J. (1976). Modern clinical psychology. New York: Ba-sic.

Kuyken, W. (2006). Evidence-based case formulation. In N. Tar-rier (Ed.), Case formulation in cognitive behavior therapy.New York: Rutledge.

Kuyken, W., Fothergill, C. D., Musa, M., & Chadwick, P. (2005).The reliability and quality of cognitive case formulation. Be-havior Research and Therapy, 43, 1187–1201.

Kuyken, W., Padesky, C. A., & Dudley, R. (2009). Collaborativecase conceptualization: Working effectively with clients in cog-nitive-behavioral therapy. New York: Guilford.

Levine, F. J., & Luborsky, L. (1981). The Core Conflictual Rela-tionship Theme: A demonstration of reliable clinical inferenceby the method of mismatched cases. In S. Tutman, C. Kaye,& M. Zimmerman (Eds.), Object and self: A developmentalapproach (pp. 501–520). Madison, CT: International Univer-sities Press.

Luborsky, L. (1995). The first trial of the P technique in psycho-therapy research – A still lively legacy. Journal of Consultingand Clinical Psychology, 63, 6–14.

Luborsky, L., Crits-Christoph, P., & Mellon, J. (1986). Advent ofobjective measures of the transference concept. Special issue:Psychotherapy research. Journal of Consulting and ClinicalPsychology, 54, 39–47.

McKnight, D. L., Nelson, R. O., Hayes, S. C., & Jarrett, R. B.(1984). Importance of treating individually assessed responseclasses in the amelioration of depression. Behavior Therapy,15, 315–335.

McMahon, R. J. (1999). Parent training. In Russ & Ollendick(Eds.), Handbook of psychotherapies with children and fami-lies (pp. 153–180). New York: Kluwer.

Meehl, P. (1954). Clinical versus statistical prediction. Minneap-olis: University of Minnesota Press.

Meehl, P. E. (1986). Causes and effects of my disturbing littlebook. Journal of Personality Assessment, 50, 370–375.

Messer, S., Tishby, O., & Spillman, A. (1992). Taking contextseriously in psychotherapy research: Relating therapist inter-ventions to patient progress in brief psychodynamic therapy.Journal of Consulting and Clinical Psychology, 60, 678–688.

Molenaar, P. C. M. (1985). A dynamic factor model for the anal-ysis of multivariate time series. Psychometrica, 50, 181–202.

Molenaar, P. C. M. (2004). A manifesto on psychology as idio-graphic science: Bringing the person back into scientific psy-chology, this time forever. Measurement, 2, 201–218.

Molenaar, P. C. M., & Campbell, C. G. (2008). Discussion of thespecial issue on growth models for longitudinal data in educa-tion. Educational Research and Evaluation, 14, 377–390.

Molenaar, P. C. M., & Valsiner, J. (2005). How generalizationworks through the single case: A simple idiographic process



analysis of an individual psychotherapy. International Journalof Idiographic Science, 1. Retrieved from http://www.val-siner.com/articles/molenvals.htm

Mumma, G. H. (1998). Improving cognitive case formulationsand treatment planning in clinical practice and research. Jour-nal of Cognitive Psychotherapy, 12, 251–274.

Mumma, G. H. (2001). Increasing accuracy in clinical decisionmaking: Toward an integration of nomothetic-aggregate andintraindividual-idiographic approaches. The Behavior Thera-pist, 24, 77–94.

Mumma, G. H. (2004). Validation of idiosyncratic cognitive sche-ma in cognitive case formulations: An intraindividual idio-graphic approach. Psychological Assessment, 16, 211–230.

Mumma, G. (in press). Current issues in case formulation. In P.Sturmey & M. McMurran (Eds.), Forensic case formulation.UK: Wiley.

Mumma, G. H., & Fluck, J. (2010). Validity evaluation in cogni-tive-behavioral case formulation: A practical, clinician-friendly approach. Unpublished manuscript under review.

Mumma, G. H., & Mooney, S. R. (2007a). Comparing the validityof alternative cognitive case formulations: A latent variable,multivariate time series approach. Cognitive Therapy and Re-search, 31, 451–481.

Mumma, G. H., & Mooney, S. R. (2007b). Incremental validity ofcognitions in a clinical case formulation: An Intraindividualtest in a case example. Journal of Psychopathology and Be-havioral Assessment, 29, 17–28.

Mumma, G. H., & Smith, J. L. (2001). Cognitive-behavioral in-terpersonal scenarios: Interformulator reliability and conver-gent validity. Journal of Psychopathology and Behavioral As-sessment, 23, 203–221.

Mumma, G. H., & Wilson, S. B. (1995). Procedural debiasing ofanchoring effects in clinical-like judgments. Journal of Clini-cal Psychology, 51, 841–853.

Muran, J. C., Samstag, L. W., & Segal, Z. V. (1998). Interpersonalscenarios: An idiographic measure of self-schemas. Psycho-therapy Research, 8, 321–333.

Muran, J. C., Samstag, L. W., Ventur, E. D., Segal, Z. V., & Win-ston, A. (2001). A cognitive-interpersonal case study of theself. Journal of Clinical Psychology, 57, 307–330.

Muran, J. C., & Segal, Z. V. (1992). The development of an idio-graphic measure of self-schemas: An illustration of the con-struction and use of self-scenarios. Psychotherapy, 29,524–535.

Needleman, L. D. (1999). Cognitive case conceptualization. Mah-wah, NJ: Erlbaum.

Nelson-Gray, R. O. (1996). Treatment outcome measures: Nomo-thetic or idiographic? Clinical Psychology: Science and Prac-tice, 3, 164–167.

Nelson-Gray, R. O. (2003). Treatment utility of psychological as-sessment. Psychological Assessment, 15, 521–531.

Nelson-Gray, R. O., Herbert, J. D., Herbert, D. L., Sigmon, S. T.,& Brannon, S. E. (1989). Effectiveness of matched, mis-matched, and package treatments of depression. Journal of Be-havior Therapy and Experimental Psychiatry, 20, 281–294.

Nesselroade, J. R., McArdle, J. J., Aggen, S. H., & Meyers, J. M.(2002). Dynamic factor analysis models for representing pro-cess in multivariate time-series. In D. S. Moskowitz & S. L.Hershberger (Eds.), Modeling intraindividual variability withrepeated measures data (pp. 233–265). Mahwah, NJ: Erl-baum.

Nezu, A. M., Nezu, C. M., & Lombardo, E. (2004). Cognitive-be-havioral case formulation and treatment design: A problemsolving approach. New York: Springer.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory(3rd ed.). New York: McGraw-Hill.

O’Brien, W. H., Oemig, C. K., & Northern, J. J. (2010). Behav-ioral assessment with adults. In J. Thomas & M. Hersen (Eds.),Handbook of clinical psychology competencies (pp. 343–365).New York: Springer.

Pain, C. M., Chadwick, P., & Abba, N. (2008). Clients’ experienceof case formulation in cognitive behavior therapy for psycho-sis. British Journal of Clinical Psychology, 47, 127–138.

Pankratz, A. (1991). Forecasting with dynamic regression mod-els. New York: Wiley.

Persons, J. B. (1989). Cognitive therapy in practice: A case for-mulation approach. New York: Norton.

Persons, J. B. (2008). The case formulation approach to cogni-tive-behavior therapy. New York: Guilford.

Persons, J. B., & Bertagnolli, A. (1999). Interrater reliability ofcognitive-behavioral case formulations of depression: A rep-lication. Cognitive Therapy and Research, 23, 271–283.

Persons, J. B., Davidson, J., & Tompkins, M. A. (2001). Essentialcomponents of cognitive-behavior therapy for depression.Washington, DC: American Psychological Association.

Persons, J. B., Mooney, K. A., & Padesky, C. A. (1995). Interraterreliability of cognitive-behavioral case formulations. Cogni-tive Therapy and Research, 19, 21–34.

Persons, J. B., Roberts, N. A., Zalecki, C. A., & Brechwald,W. A. G. (2006). Naturalistic outcome of case formulation-driven cognitive-behavior therapy for anxious depressed out-patients. Behavior Research and Therapy, 44, 1041–1051.

Persons, J. B., & Tompkins, M. A. (2007). Cognitive-behavioralcase formulation. In T. D. Eells (Ed.), Handbook of psycho-therapy case formulation (2nd ed., pp. 290–316). New York:Guilford.

Petermann, F., & Müller, J. (2001). Clinical psychology and sin-gle-case evidence. New York: Wiley.

Riggins, J., (2009). Evaluating case formulation and change pro-cess/change mechanism hypotheses in cognitive behavioraltherapy of adults with comorbid depression and anxiety. Un-published doctoral dissertation, Texas Tech University.

Riggins, J., & Mumma, G. H. (2008, November). Using EMA totest cognitive-behavioral case formulation hypotheses. Pre-sented at the 42nd Annual Convention of the Association ofBehavioral and Cognitive Therapies, Orlando, FL.

Schulte, D., Kunzel, R., Pepping, G., & Schulte-Bahrenberg, T.(1992). Tailor-made versus standardized therapy of phobic pa-tients. Advances in Behavioral Research and Therapy, 14,67–92.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experi-mental and quasiexperimental designs for generalized causalinference. New York: Houghton Mifflin.

Silberschatz, G., Fretter, P., & Curtis, J. (1986). How do interpre-tations influence the process of psychotherapy? Journal ofConsulting and Clinical Psychology, 54, 646–652.

Smith, P. N., & Mumma, G. H. (2006, August). Testing a cogni-tive case formulation in a clinical training setting. Evaluatingcognitive behavioral case formulation hypotheses using re-peated measurement. Symposium presented at the 114th An-nual Convention of the American Psychological Association,New Orleans, LA.



Steiger, H., Gauvin, L., Jabalpurwala, S., Séquin, J. R., & Stot-land, S. (1999). Hypersensitivity to social interactions in bu-limic syndromes: Relationship to binge eating. Journal of Con-sulting and Clinical Psychology, 67, 765–775.

Stone, A. A., Shiffman, S., Atienza, A. A., & Nebeling, L. (2007).The science of real-time data capture: Self-reports in healthresearch. New York: Oxford.

Sturmey, P. (2008). Behavioral case formulation and intervention.New York: Wiley.

Sundberg, N., & Tyler, L. (1962). Clinical psychology: An intro-duction to research and practice. New York: Appleton-Centu-ry-Crofts.

Tabachnick, B., & Fidell, L. (2007). Using multivariate statistics(5th ed.). New York: Pearson.

Tarrier, N. (Ed.). (2006). Case formulation in cognitive behaviortherapy: The treatment of challenging and complex cases. NewYork: Routledge.

Tarrier, N., & Calam, R. (2002). New developments in cognitive-behavioral case formulation. Epidemiological, systemic, andsocial context: An integrative approach. Behavioral and Cog-nitive Psychotherapy, 30, 311–328.

Thorpe, G. L., & Olson, S. L. (1997). Behavior therapy: Concepts,procedures, and applications (2nd ed). Boston: Allyn and Ba-con.

Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993).Testing the integrity of a psychotherapy protocol: Assessment

of adherence and competence. Journal of Consulting and Clin-ical Psychology, 61, 620–630.

Wedding, D., & Faust, D. (1989). Clinical judgment and decisionmaking in neuropsychology. Archives of Clinical Neuropsy-chology, 4, 233–266.

Wells, A. (2006). Cognitive therapy case formulation in anxietydisorders. In N. Tarrier (Ed.), Case formulation in cognitivebehavior therapy: The treatment of challenging and complexcases (pp. 52–80). London: Routledge.

Wilson, G. T. (1996). Manual-based treatments: The clinical ap-plication of research findings. Behavior Research and Thera-py, 34, 295–314.

Wood, P., & Brown, D. (1994). The study of intraindividual dif-ferences by means of dynamic factor models: Rationale, im-plementation, and interpretation. Psychological Bulletin, 116,166–186.

Zayfert, C., & Becker, C. B. (2007). Cognitive-behavioral therapyfor PTSD: A case formulation approach. New York: Guilford.

Gregory H. Mumma

Department of PsychologyTexas Tech UniversityLubbock, TX 79407USAE-mail [email protected]



Validity Issues in CB Formulation

Documents

Transcript of Validity Issues in CB Formulation