The Efficacy and Cost Effectiveness ofPsychotherapy

October 1980

NTIS order #PB81-144610

Library of Congress Catalog Card Number 80-600161

For sale by the Superintendent of Documents, U.S. Government Printing OfficeWashington, D.C. 20402 Stock No. 052-003 -00783-5

Foreword

This volume is a background paper for OTA’s assessment, The lmplications ofCost-Effectiveness Analysis of Medical Technology. That assessment analyzes thefeasibility, implications, and usefulness of applying cost-effectiveness and cost-benefitanalysis (CEA/CBA) in health care decisionmaking. The major, policy-oriented reportof the assessment was published in August 1980. In addition to the main report andthis background report, there will be four other background papers: 1) a documentwhich addresses methodological issues and reviews the CEA/CBA literature, 2) a diag-nostic X-ray case study, 3) 17 other case studies of individual medical technologies,and 4) a review of international experience in managing medical technology. Anotherrelated report was published in September of 1979: A Review of Selected Federal Vac-cine and Immunization Policies.

Background Paper #.3: The Efficacy and Cost Effectiveness of Psychotherapy w a sprepared in response to a request of the Senate Finance Committee. OTA established aspecial advisory panel to assist in the development of this paper; in addition, OTAconsulted with the members of the advisory panel to the overall assessment and with agroup of ad hoc reviewers from various mental health fields. We are grateful for theirassistance.

JOHN H. GIBBONSDirector

I l l

Advisory Panel on The Implications ofCost-Effectiveness Analysis of Medical Technology

John R. Hogness, Panel ChairmanPresident, Association of Academic Health Centers

Stuart H. AltmanDeanFlorence Heller SchoolBrandeis University

James L. BenningtonChairmanDepartment of Anatomic Pathology and

Clinical LaboratoriesChildren Hospital of San Francisco

John D. ChaseAssociate Dean for Clinical AffairsUniversity of Washington School of Medicine

Joseph FletcherVisiting ScholarMedical EthicsSchool of MedicineUniversity of Virginia

Clark C. HavighurstProfessor of LawSchool of LawDuke University

Advisory Panel on Psychotherapy

Jerome D. FrankProfessor Emeritus of PsychiatryThe Johns Hopkins University School

of Medicine

Donald F. KleinDirector of ResearchNew York State Psychiatric Institute

Beverly Benson LongPresidentNational Mental Health Association

Sheldon LeonardManagerRegulatory AffairsGeneral Electric Co.

Barbara J. McNeilDepartment of RadiologyPeter Bent Brigham Hospital

Robert H. MoserExecutive Vice PresidentAmerican College of Physicians

Frederick MostellerChairmanDepartment of BiostatisticsHarvard University

Robert M. SigmondAdvisor on Hospital AffairsBlue Cross and Blue Shield Associations

Jane Sisk WillemsVA ScholarVeterans Administration

Morris ParloffChief, Psychotherapy and Behavioral

Intervention SectionNational Institute of Mental Health

Hans H. StruppDistinguished Professor of PsychologyVanderbilt University

Gary R. VandenBosDirector, National Policy StudiesAmerican Psychological Association

Thomas McGuireAssistant Professor of EconomicsBoston University

OTA Staff for Background Paper #3

Joyce C. Lashof, Assistant Director, OTAHealth and Life Sciences Division

H. David Banta, Health Program Manager

Clyde J. Behney, Project Director, Implications of Cost-EffectivenessAnalysis of Medical Technology

Leonard Saxe, Study Director-, Background Paper #3

Kerry Britten Kemp, Writer/EditorVirginia Cwalina, Administrative Assistant

Shirley Ann Gayheart, SecretaryNancy L. Kenney, Secretary

Contractors

Brian T. Yates, The American University

Frederick Newman, Eastern Pennsylvania Psychiatric Institute

OTA Publishing Staff

John C. Holmes, Publishing Officer

Kathie S. Boss Debra M. Datcher Joanne Mattingly

Contents

1. SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2. DEFINITIONS AND SCOPE OF REVIEW . . . . . . . . . . . . . . . . . . . . . . .

Theoretical Assumptions . . . . . . . . . . . . . . . . . . . . , . . . . . . . . . . . . . . . . .Mental Disorders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Practitioners. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Delivery Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Scope of This Report. ., . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3. METHODS FOR ASSESSING THE EFFECTIVENESSOF PSYCHOTHERAPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Measures of Psychotherapeutic Outcomes . . . . . . . . . . . . . . . . . . . . . . . . .Measurement Criteria ., . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Measures From Patients. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Measures From Family and Friends . . . . . . . . . . . . . . . . . . . . . . . . . . .Measures From Therapists. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Measures From Community Members. . . . . . . . . . . . . . . . . . . . . . . . .Summary , . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Research Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Therapy Versus No-Therapy Designs . . . . . . . . . . . . . . . . . . . . . . . . .Therapy Versus Therapy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Therapy Versus “Placebo’’ Therapy. . . . . . . . . . . . . . . . . . . . . . . . . . .Quasi-Experimental Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Program Evaluations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Difficulties in Conducting Research. . . . . . . . . . . . . . . . . . . . . ..., . .

Integrating Findings From Psychotherapy Research . . . . . . . . . . . . . . . . . .Final Comments , . . . . . . . . . . . . . . . . . . . . . . . . . . . ..., . . . . . . . . . . . . .

4. LITERATURE ON THE EFFECTIVENESS OF PSYCHOTHERAPY. . . .

Scope of Review. . . . . . . . . . . . . . . . . . . . . . . . ..., . . . . . . . . . . . . . . . . ,Efficacy Reviews. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Eysenck’s Reviews. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Meltzoff and Kornreich’s Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . .Bergin’s Reviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Rachman’s Critique. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Luborsky, Singer, and Luborsky’s Review. . . . . . . . . . . . . . . . . . . . . .NIMH Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Smith, Glass, and Miller’s Meta-Analysis . . . . . . . . . . . . . . . . . . . . . .

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5. METHODS FOR ASSESSING THE COST EFFECTIVENESS ANDCOST BENEFIT OF PSYCHOTHERAPY. . . . . . . . . . . . . . . . . . . . . . . . .

Cost Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .General Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Methods for Cost Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Discussion of Cost Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Benefit Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

9

1013141617

21

22222223232424252626272729293133

37

37383840414243444547

51

5252535454

Methods for CEA/CBA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56Cost-Effectiveness Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56Cost-Benefit Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Summary . . . . . .. .. .. .. ... . .$....... .. . $ . . . . . . . . . . . . . . . . . . . . 59

6. ILLUSTRATIVE COST-EFFECTIVENESS AND COST-BENEFITANALYSES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Treatment-Related Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64Inpatient Therapy.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64Community-Based Versus Institutional Therapy.. . . . . . . . . . . . . . . . 6 5Residential Versus Institutional Therapy for Problem Children . . . . . . 67

Therapist Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Professional Therapists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Nonprofessionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Drug Addiction and Medical Utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . 70Psychotherapy for Drug Addictions . . . . . . . . . . . . . . . . . . . . . . . . . . 70Psychotherapy and Overutilization of Medical Services . . . . . . . . . . . 72

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

APPENDIXES

A. Description of Other Volumes of the Assessment . . . . . . . . . . . . . . . . . . . . 7 7B. Health Program Advisory Committee, Acknowledgments . . . . . . . . . . . . . 79

REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Summary

Summary

The treatment of mental, emotional, and be-havioral dysfunctions has become one of themost controversial areas of health policy. Eventhough the prevalence and the pernicious effectsof mental disorders are well known and havebeen documented recently in sources such as thereport of the President’s Commission on MentalHealth (219), opinion about what should bedone to treat these problems is not unanimous.A number of proposals to expand the mentalhealth services system and to make treatmentmore widely available (e. g., through expandedinsurance coverage) have been made. Yet, thereis not agreement about how to expand mentalhealth services, nor about what would be gainedby their expansion.

In light of these disagreements, it is perhapsunderstandable that policy makers have been re-luctant to commit additional public resources tomental health treatments. To make psychother-apeutic treatments more widely available is po-tentially expensive. It has been suggested, how-ever, that the provision of psychotherapy re-duces other costs currently borne by society. Acentral question is the extent to which psycho-therapy can be scientifically assessed and itsvalue demonstrated in a way useful for policy-making. The present report analyzes the currentscientific literature on the evaluation of psycho-therapy. It examines the efficacy and cost effec-tiveness of psychotherapeutic treatments andconsiders both the methodological problems ofassessing psychotherapy and the state of currentknowledge about its effects. It is hoped that thisreport will inform the developing congressionaldebate on Federal research and funding forpsychotherapy.

As a background paper prepared in conjunc-tion with OTA’s assessment The Irnplications ofCost-Effectiveness Analysis of Medical Technol-ogy (see 203), the present report emphasizesmethodological issues related to the assessmentof psychotherapy and the use of cost-effective-ness/cost-benefit analysis (CEA/CBA) to eval-uate psychotherapy’s worth. It documents the

use of various evaluation strategies and consid-ers the available evidence concerning psycho-therapy’s efficacy and cost effectiveness. Theconclusions of the report relate both to the con-duct of research on psychotherapy and thestatus of present scientific knowledge aboutpsychotherapy.

The report discusses four issues centrally re-lated to the evaluation of psychotherapy: 1) thedefinition and complexity of psychotherapy; 2)the degree to which psychotherapy is amenableto scientific analysis and the availability of ap-propriate methods for studying psychotherapy;3) the evidence as to psychotherapy’s efficacy,including the results of analyses that synthesizefindings across studies; and 4) the appropriate-ness of CEA/CBAs of psychotherapy and the re-sults of their application. Below, each of thesesections of the report is briefly summarized.

Chapter 2 reviews a number of definitionsand views of psychotherapy. It also attempts todelimit the scope of this background paper.Psychotherapy is not a simple treatment, andpart of the confusion about its effectiveness hasto do with the use of different views of whatcomprises psychotherapy. In order to representthe variety of contemporary therapy practices,the present report adopts a relatively compre-hensive definition of psychotherapy. Includedare treatments based on Freudian ideas aboutpsychodynamics, as well as newer therapiesbased on behavioral theories of learning andcognition. One finding of the report is thatpsychotherapies are not distinguishable solelyby their theoretical bases. In addition to theview of psychopathology adopted by the thera-pist, therapist variables (e.g., training and per-sonality characteristics), patient variables (e. g.,seriousness of condition), and the treatment set-ting (e. g., hospital, private office) affect the na-ture of psychotherapy treatments. Although theinclusion of such complex factors makes theanalysis of psychotherapy more difficult, theirinclusion is necessary to adequately assess theeffects of psychotherapy.

Chapter 3 describes the scientific basis of ef-forts to assess the effectiveness of psychothera-py. Although the definition of psychotherapyemployed in this report is complex, and perhapsopen to dispute, the methods for assessing psy-chotherapy are better established. The presentreport describes and analyzes various methodo-logical strategies for measuring the outcomes ofpsychotherapeutic treatment and the ways inwhich the reliability and validity of these meas-ures are established. It also describes the varietyof research design strategies that have been usedto establish cause-effect relationships betweenpsychotherapy and particular outcomes. Thediscussion indicates the conditions under whichrandomized control group procedures can beused to assess psychotherapy. Also analyzed arethe use of quasi-experimental and nonexperi-mental procedures which, depending on whattypes of information are required, may provideuseful data. The problems of carrying outpsychotherapy research are also discussed, in-cluding the difficulties of withholding treatmentfrom members of a control group and theproblems of assessing multifactor treatmentprograms.

Also considered in chapter 3 are two recentmethodological developments. One is the con-duct of program evaluation studies, in whichsets of psychotherapy variables are investigatedsimultaneously. Such evaluation studies may beuseful where several psychotherapy-related var-iables are considered in conjunction with oneanother and where it is difficult to separatetreatment components. Another recent method-ological development is the use of systematicprocedures for synthesizing the findings ofmultiple investigations. These data integrationmethods are new and somewhat controversialprocedures for assessing the implications of thepsychotherapy research literature. Both of thesemethods, along with scientifically rigorous stud-ies of psychotherapy, may prove useful to de-veloping mental health policy.

Chapter 4 selectively describes the substan-tive literature on psychotherapy’s effects. Anumber of prominent reviews of the psychother-apy literature, and the commentary generatedby these reviews, are analyzed (along with sev-

eral individual evaluative studies), Despite somefundamental differences, both in the criteria forassessing psychotherapy and in the studies in-cluded, the reviews all report—under specifiedconditions—evidence for psychotherapy’s effec-tiveness. This finding is stronger the more recentthe literature that is reviewed. In fact, thereseems to be little negative evidence as to the effi-cacy of psychotherapeutic treatments. Althoughit is difficult to make global statements, the evi-dence seems more supportive of psychotherapythan of alternative explanations (e.g., spontane-ous remission, placebo effects). The availableresearch, some of which meets rigorous meth-odological standards, seems to indicate thatpsychotherapy treatment is clearly better thanno treatment. However, while the literature sup-ports a generally positive conclusion with re-spect to the effectiveness of psychotherapy,there is a lack of specific information about theconditions under which psychotherapy is effec-tive. It is not clear which aspects of therapy(e.g., treatment protocol v. the nature of thetherapist’s relationship with the patient) are re-sponsible for particular outcomes.

Methods for assessing the costs and benefitsof psychotherapy and for developing CBAs aredescribed in chapter 5. The application of CEA/CBA to psychotherapy is much more recent,and less developed, than efficacy research. Al-though the methods for CEA/CBAs of psycho-therapy are based on applications of such analy-ses to other types of health and nonhealth prob-lems, in some instances (e.g., application of the“willingness-to-pay” concept), the translation topsychotherapy is difficult. An additional prob-lem with psychotherapy assessments has to dowith the comprehensiveness of cost and benefitassessment—in particular, our ability to valuein pecuniary terms the effects of psychotherapy.Because of this difficulty, much of the recentcost analysis research has involved cost-effec-tiveness comparisons rather than cost-benefitcomparisons. Potentially, however, both CEAand CBA techniques may be useful to improveour understanding of the effects of psychother-apy and the resources necessary for its efficientuse. Such CEA/CBA research may be an impor-tant adjunct to effectiveness studies that willenhance their policy use.

C/I. 1—Summary ● 5

Chapter 6 reviews the available literature onthe use of CEA/CBA for assessing psychothera-py. The literature which reports actual CEA/CBA studies varies, both in terms of its focus ondifferent problems and its methodological ade-quacy. Because of the problem of valuing psy-chotherapy outcomes, a great number of theavailable cost studies focus on low-cost treat-ments. This may create an incorrect impressionabout psychotherapy’s effects. In terms of theirlimitation of problem focus, many of these anal-yses have been conducted on treatment settingscharacteristics (e. g., institutional v. noninstitu-tional care), rather than on different therapiesor therapists. The findings of these studies seemto indicate that more efficient ways of deliveringpsychotherapy can be developed. Unfortunate-ly, it is difficult to interpret the results of manyof these studies, because they ignore importantcosts or benefits or because they use inadequate

research designs. In some cases (e. g., the effectsof psychotherapy on medical utilization rates),at least the potential for psychotherapy to pro-vide society with large net benefits has beendemonstrated. It will be necessary, however, tovalidate these findings using better proceduresand a wide range of mental health problems.

In summary, OTA finds that psychotherapyis a complex—yet scientifically assessable—setof technologies. It also finds good evidence ofpsychotherapy’s positive effects. Although thisevidence may not be generalizable to the widerange of problems for which therapy is em-ployed, it suggests that additional research mayprovide data useful for the development of men-tal health policy. Given the potential net ben-efits of psychotherapy, this effort would seem tobe justified.

2mDefinitions and Scope of Review

2Definitions and Scope of Review

The term psychotherapy has been used to re-fer to a wide variety of treatments employed toameliorate mental distress, mental illness, andproblems of coping with daily life, Psychother-apy is a global term, and one commentator hasfound at least 40 different definitions in thescientific literature (300). The definitions rangefrom broad and inclusive descriptions of vari-ous ways of helping other individuals to morelimiting definitions which are very specific as tothe nature of the problem and treatment (see,e.g., 118). Differences in the definitions used bythose who discuss psychotherapy are a likelyreason that discussions of psychotherapy havebeen so difficult to understand and evaluate.The present review attempts to specify the na-ture of psychotherapy, so as to avoid some ofthese problems.

To illustrate some of the difficulties of differ-ent views of psychotherapy, consider a broaddefinition such as “the treatment of emotionaland personality problems and disorders by psy-chological means” (144). This definition refersto treatments that may have either a good orpoor scientific basis, as well as to treatmentsthat are delivered by either professionallytrained or untrained personnel. In contrast, amore specific definition (which has been favoredby some researchers) describes psychotherapyin terms of its techniques, the qualifications ofthe therapist, and nature of the patient’s prob-lem (181):

Psychotherapy is taken to mean the planfulapplication of techniques derived from estab-lished psychological principles, by persons qual-ified through training and experience to under-stand these principles and to apply these tech-niques with the intention of assisting individualsto modify such personal characteristics as feel-ings, values, attitudes, and behaviors which arejudged by the therapist to be maladaptive ormaladjustive.

Apart from their level of specificity, defini-tions vary with regard to what is included as thedisorders to be treated by psychotherapy. Al-

though some definitions, like the above, are notspecific as to the nature of the disorder (see,e.g., 84), diagnosis is becoming more system-atic; viz, the newly developed Diagnostic andStatistical Manual of Mental Disorders (63).The new diagnostic manual includes disordersthat range from organic brain syndromes, schiz-ophrenic and paranoid disorders, to affectivedisorders such as depression, anxiety, and disor-ders such as phobias or posttraumatic stress;also included are behavior aberrations such askleptomania or pathological gambling, person-ality disorders, alcohol- and drug-related prob-lems, and interpersonal difficulties such as mari-tal or family disturbances or education andwork blockages.

One additional feature of the various ap-proaches to defining psychotherapy is that def-initions vary in terms of their underlying theo-retical assumptions. These assumptions lead todifferences in defining the goal of psychothera-py, the basic techniques, and the role of the pa-tient and therapist. As Hogan (118) has noted,psychotherapy can refer to a wide variety ofvaguely defined processes. Attempts to specifythe nature of psychotherapy result, in part, indefinitions that vary in terms of the practition-ers and settings which are included as accept-able. Different definitions require different lev-els of professional training and experience forpsychotherapy practitioners. For example, med-ical definitions of psychotherapy (see 163) maylimit primary responsibility for psychotherapytreatment to physicians, while nonmedical mod-els (e. g., 250) may suggest central roles for psy-chologists, social workers, and nurses, as wellas educational and pastoral counselors. Thevarious forms of psychotherapy are further af-fected by the variety of settings in which ther-apy takes place (e. g., psychiatric hospitals, out-patient clinics, private offices) and the variety ofmodalities that are offered (e. g., individual,family, or group counseling).

In order to adequately summarize psycho-therapy research and practice, it is necessary to

Y

10 ● Background Paper #.3: The Efficacy and Cost Effectiveness of Psychpytherapy

reflect components of a number of definitions ofpsychotherapy. In this report, the definition ofpsychotherapy is limited only in the sense thatan effort is made to view psychotherapy as afield that is both scientific and professional.Thus, our definition will emphasize therapiesthat have an established scientific base and thatare delivered by trained professionals. In addi-tion, although it seems unnecessary (as well asdifficult) to differentiate between levels ofpsychopathology, this report is most concernedwith those disorders that cause severe dysfunc-tion. Although psychotherapy may be useful in

THEORETICAL ASSUMPTIONS

When discussing psychotherapy, one typical-ly refers to differences between treatments basedon different underlying theoretical assumptions.There is a wide variety of beliefs about peopleand pathology, and this has resulted in the de-velopment of a number of different psychother-apeutic treatments. The variety of psychothera-peutic treatments, at least in terms of the num-ber of labels used to describe psychotherapies,is somewhat bewildering. Nevertheless, manytreatments have similar origins, and it is possi-ble to examine the range of therapies by consid-ering only a few major theoretical perspectives.For present purposes, therapies based on psy-chodynamic, humanistic/ phenomenological,and behavioral theories are discussed as the per-spectives that underlie most current practice.These theoretical perspectives do not exist inpure form, but they illustrate the most impor-tant differences between treatment approaches(see 259).

Psychodynamic. —The psychodynamic per-spective, which grew out of the classical psy-choanalytic theory of Freud (88), is the oldestand the most complex of the three orientations.Although psychoanalytic theory cannot be de-scribed here in detail, the role of unconsciousmotivation, one aspect of Freud’s theory, canprovide a basis for understanding the nature ofpsychodynamic (also called analytic) treat-ments. The importance of unconscious motiva-tion is central to all of Freud’s psychological in-

aiding individuals’ adjustment and improvingthe quality of life, the primary concern here iswith problems that require some form of outsideintervention.

The following section briefly describes thescope of psychotherapy practice, in terms of thedefinitional variables described above. It ishoped that this review will orient readers to thecentral features of psychotherapy and will iden-tify the central factors that must be accountedfor in an assessment of psychotherapy.

terpretations. The driving forces of an individ-ual’s life are seen as originating in some innersystem of which he or she might, at the most, beonly vaguely aware. Most of the unconsciouscannot be called into conscious awareness; anymove in this direction meets with resistance, orcensorship.

According to Freud’s theory, some uncon-scious impulses come into conflict with envi-ronmental constraints and moral prohibitions.To keep such threatening impulses from cominginto consciousness, part of the normal energy ofmental life is used to provide a constant defense,or resistance, against the acceptance of such im-pulses. These impulses, however, are not elimi-nated by such a defensive reaction; instead, theyexpress themselves in indirect ways, leading tobehavior that the individual is sometimes un-able to explain. Neurosis occurs when such be-havior is especially prominent in the life of anindividual.

Thus, the goal of analytic psychotherapy isgradually to promote the patient’s insight intohis or her underlying motivations and conflictsby repeatedly working through the patient’s in-tellectual and emotional defensive resistances.Acquisition of insight, along with affective un-derstanding, is presumed to be a sufficient andnecessary condition for restructuring significantaspects of the patient’s personality and for pro-moting the development of more adaptive be-haviors in real life.

Ch. 2—Definitions and Scope of Review ● 11

The methods that use therapies based on thepsychodynamic approach are intended to aidthe patient in bringing unconscious material toawareness. The core of therapy lies in trans-ference. Freud suggested that patients inevitablyproject into the therapeutic situation feelingsand attitudes from their past, and that theysometimes reenact with the therapist importantsituations and traumatic experiences repressedsince childhood. By becoming aware of theseunresolved conflicts, patients can master themand liberate their energy to develop a maturepersonality. Therapeutic techniques are de-signed so as to encourage and to analyze thisprocess of transference.

In Freudian psychotherapy, the therapist alsomakes timely interpretations of the patient’s freeassociations and dreams and deals with the pa-tient’s resistances to the content of this un-conscious material. Psychoanalytic therapy hasprovided the model for most “verbal” therapies(i.e., therapies based on conversations betweentherapist and patient). Although infrequentlypracticed in pure form, in part because many ofFreud’s students (e.g., Adler, Jung) developedtheir own forms of therapy, psychoanalytic the-ory has had a major influence on other types ofpsychotherapeutic treatments.

Humanistic/Phenomenological. —The secondmajor orientation that has influenced presentday treatment can be called the humanistic/phenomenological perspective. The approach ishumanistic because it posits that the basic aimof the individual is the achievement of personalgrowth. Personal growth is variously conceivedof as self-actualization (e.g., 234,235,236),meaning in life (e.g., 87), and cognitive com-plexity (e.g., 137). The perspective is also calledphenomenological because it holds that thecentral focus of treatment should be an individ-ual’s “phenomenal field. ” Thus, a person’sperceptions about himself or herself and theworld (rather than unconscious thoughts) arebelieved to be central to understanding. Thetheory further posits that each person has a fun-damental urge to preserve his or her self-concept(i.e., perceptions about self).

This theoretical approach, like psychoanalyt-ic theory, has influenced how psychotherapy is

practiced by a variety of different therapists. Amajor form of therapy based on this perspectiveis “client-centered” or “nondirective” psycho-therapy. This therapy is grounded in the earlytheoretical work of Rogers (234; see also 235,236). According to Rogers, when individualshave needs, impulses, or experiences that arenot consistent with their self-concept, tensionand anxiety result. More severe problems resultwhen individuals react with defensive mecha-nisms to avoid awareness of their contradictoryresponses by ignoring important responses o rby distorting their thinking about them. Therole of the therapist, according to Rogers, is tobean empathetic responder.

A central tenet of client-centered therapy isthe belief that each individual has the resourcesfor growth. These resources merely need to bereleased for the person to resolve a problem andachieve maturity. Thus, therapy facilitates thepatient’s growth by promoting the free expres-sion of feelings and by refraining from imposingpatterns and values. Out of the therapeutic rela-tionship, patients (almost always referred to asclients) evolve a new conceptualization of theself. This view of themselves is more tolerant,especially of their failings, and it resolves thedifferences between their ideal image and theiractual perception of themselves. As a result,anxiety and dysfunctional tendencies are les-sened, and there is greater objectivity in thehandling of reality. The new self-image shouldalso be accompanied by a more harmonious ex-pression of attitudes and feelings.

Therapists’ efforts to interpret, evaluate, orguide the individual (which are central to ana-lytic therapy) are felt by humanistic therapiststo hamper patients’ emerging sense of self-growth and self-direction. Techniques common-ly employed in client-centered therapy are in-tended to facilitate individuals who seek help inachieving their own insight. These techniquesinclude attentive listening to clients’ commu-nications for content and feeling, reflecting orverbally focusing clients’ feelings, and encour-aging clients in their efforts to manage their ownproblems.

Behavioral. —The third major orientation isthe behavioral perspective. It developed, at least

initially, from an extensive body of experimen-tal research and theory associated with animaland human learning research. Theorists of be-havior such as Skinner (263,264,265), Pavlov,and Hull—who were not psychotherapists or di-rectly interested in the treatment of mental dys-function—are most responsible for the ideasthat underlie this approach. A wide variety oftherapeutic approaches derived from these basicresearch efforts have been developed by latertheorists such as Wolpe (302) and Bandura (11).

The basic assumption of behavioral theoriesis that most persistent pathological behavior isacquired through a process of learning. In es-sence, experiences in the individual’s life in-stigate patterns of behavior which remain unlessother conditioning experiences modify them.Therapies based on behavior theory usually be-gin with systematic analysis of the problematicsituation and an identification of the specificbehavioral or situational factors to be modified.Once it is known how and under what condi-tions these behaviors are maintained in the indi-vidual’s behavioral repertoire, the therapist in-terferes directly with this preestablished order.Internal (cognitive) or external (environmental)patterns of behavior contrary to the well-beingof the patient are modified by systematic re-training procedures.

Current clinical practice in behavioral ther-apy is derived from two laboratory-developedtheories of learning. The first, operant condi-tioning, is derived directly from research bySkinner (e.g., 263). Because behavior is pre-sumed to be lawful and predictable, behavior-ists believe it is controllable once the relevantreinforcing variables (those that maintain thebehavior) have been determined. Operant con-ditioning includes aversive counterconditioningto eliminate undesirable behaviors and/or posi-tive reinforcement to establish and maintainnew adaptive behaviors. Aversive countercon-ditioning entails pairing the patient’s behavioraldeviation with an unpleasant stimulus. For ex-ample, alcoholism has been treated by adminis-tering a substance to patients which would in-duce violent and uncomfortable physical reac-tions in the presence of alcohol. Positive rein-forcement entails rewarding the patient for pro-

ducing desired behaviors. For instance, positivereinforcement has been used to treat autisticchildren who are disassociated from reality andunable to engage in gratifying interpersonal re-lationships. These children are rewarded withverbal approval and candy when they show anyevidence of social responsiveness or interaction.

Another form of clinical practice, developedby Wolpe (302), is reciprocal inhibition. Thistherapy is an outgrowth of Pavlov’s classicalconditioning studies and of Hull’s learningtheory. The theory underlying reciprocal inhibi-tion is that neuroses (in particular, phobias) arecharacterized by persistent anxiety responses tosituations in which there is no objective danger.The neuroses are produced when an individualassociates high intensities of anxiety and neutralstimulus events. Therapy based on principles ofreciprocal inhibition systematically desensitizesthe patient to particular stimulus events. Thus,a patient with neurotic anxiety is taught to re-lax. Then, in a state of relaxation, he or she isasked to think about a mildly anxiety-provok-ing situation. This technique is then used withincreasingly more disturbing situations. Relaxa-tion inhibits anxiety, and the patient is progres-sively able to master the situations previously

associated with anxiety reactions.

Although many of the behavioral therapiesrely directly on learning paradigms tested withlaboratory animals, a more recent trend (inpart, exemplified by Bandura’s work (11,12)) isthe development of cognitive behavior thera-pies. Within such therapies, laws of learning areused as the basis for modifying cognitive proc-esses (see 164, 180). The patient in cognitive be-havior therapy takes an active role in identify-ing problems and working with the therapist todevelop mechanisms to control desirable or un-desirable thoughts. Bandura’s (12) social learn-ing theory is a related approach that emphasizesthe role of observing others (models) who arereinforced or punished. There are also a varietyof eclectic behavior therapies (e.g., 154) whichemploy a varying set of learning principles.

C o m m o n F a c t o r s . — Psychotherapy , asshould be clear from the above, is derived froma range of assumptions, Although the threetheoretical perspectives described above under-

lie most forms of therapy, there is considerableeclecticism in practice. A practitioner is likely tohold some views or use techniques that are con-sistent with one dominant theoretical perspec-tive and to hold other views and use other tech-niques that are quite inconsistent with it.

In part, the reason for this eclecticism is that anumber of important similarities exist acrossdifferent theoretical persuasions. Some theoristssuch as Frank (84,86), in fact, argue that psy-chotherapeutic change is predominantly a func-tion of factors common to all therapeutic ap-proaches (e. g., 84). The primary ingredients ofsuch common, nonspecific, factors are the ther-

MENTAL DISORDERS

Although it is true that one source of confu-sion about psychotherapies is the divergenttheoretical assumptions of therapists, it shouldalso be recognized that the nature of the prob-lems dealt with by specific therapies varieswidely. Not only are a broad range of problemsconsidered psychopathological, but for any oneindividual, the identification of a disorder isoften dependent on his or her own reports ofsymptoms. In contrast to physical diseases,which are usually accompanied by measurablephysiological changes, mental disorders mustusually be identified by a patient’s or anotherperson’s report of a problem. Notwithstandingthe many exceptions to this distinction betweenphysical and mental illness, the difficulties of di-agnosis of mental illness are widely recognized.An additional problem in diagnosis is that theseverity of a mental/psychological problem isbased on the context in which it occurs. A prob-lem that may be diagnosed as a severe disorderin one situation (e. g., at work) may be consid-ered a minor disturbance in another context(e.g., at home).

Widely varying phenomena have been con-sidered mental disorders. Most therapists andresearchers seem to contend that any problemwhich causes a patient discomfort and dysfunc-tion is a mental health problem requiring treat-ment. For present purposes, the most importantaspect of the problem would seem to be the level

apist’s understanding, respect, interest, encour-agement, and acceptance. Thus, while the con-tents and procedures of psychotherapy may dif-fer across theoretical orientations, all forms ofpsychotherapy share common “healing” func-tions. All therapists combat the patient’s demor-alization and sense of hopelessness by the rela-tionship they establish with the patient and byproviding an explanation for previously inex-plicable feelings and behavior. According tothose who maintain that such nonspecific fac-tors are responsible for psychotherapy’s effects,one reason for the success of therapy is becauseit removes the mystery from the patient’s suffer-ing and supplants it with hope.

of dysfunction. Any person who is unable tocarry out his or her normal responsibilities is apotential candidate for therapy. Level of dys-function, however, is a difficult distinction tomake. An individual may appear to be function-ing well, but may be extremely troubled. Onewould want to alleviate such dysfunction andprevent more serious dysfunction.

Below, the disorders treated by psychothera-py have been grouped within three general cate-gories: neuroses, psychoses, and conduct disor-ders. While the most current diagnostic manual(63) lists mental disorders somewhat differently,for the purpose of describing the range of men-tal disorders, this organization seems moreappropriate.

Psychoses are severe disorders characterizedby grossly illogical thought patterns and dis-torted perceptions of reality. The psychotic in-dividual may have prolonged periods of melan-choly or elation and may exhibit extreme isola-tion or withdrawn behavior. The symptoms of aschizophrenic disorder, the most prevalent typeof psychosis, include bizarre delusions, auditoryhallucinations, and incoherence. Schizophreniais accompanied by a deterioration in level offunctioning in areas such as work, social rela-tions, and self-care.

Neuroses, the most common disorders, areproblems marked by feelings of anxiety, fear,

14 ● Background Paper #3: The Efficacy arid Cost Effectiveness of Psychotherapy

depression, guilt, and other unpleasant emo-tions. Anxiety, for example, is related to phys-ical symptoms such as trembling, sweating, andheart pounding. It may result in the individual’sfeeling continually worried and being overly at-tentive. The individual may feel impatient andirritable, and may have insomnia. Whileassociated with discomfort, such anxiety maynot impair social or occupational functioningmore than mildly. In contrast, agoraphobia (anirrational fear of crossing or being in openspaces) causes individuals to avoid certain situa-tions, and thus may interfere seriously withsocial and occupational functioning. Neuroticconflicts may also be the basis of other prob-lems, such as divorce and child abuse.

Conduct disorders are characterized by a lackof normal feelings and conscience that results inbehaviors which violate basic social rules. Con-duct disorders are reflected by problems such asuncontrolled impulses (e.g., outbursts of aggres-siveness, failure to resist impulses to steal or toset fires) and drug and alcohol abuse. Conductdisorders among children and adolescents in-clude such symptoms as physical violenceagainst persons or property (e.g., vandalism,rape, mugging), chronic violations of rules(e.g., truancy from school, alcohol or drugabuse), and persistent lying and stealing.

In addition to the above disorders, there are avariety of conditions which reflect interactionsbetween different types of problems. Mostprominent here are psychosomatic disorders,where physical symptoms are thought to have apsychological base (e.g., 188). Some have sug-gested (e.g., 152) that over 50 percent of the pa-tients seen in general medical practice have psy -

PRACTITIONERS

As indicated above, the delivery of psycho-therapy differs not only in terms of theoreticalassumptions and types of disorders treated, butalso in terms of the personnel who offer ther-apy. Therapists have an important influence onthe nature of psychotherapeutic treatment. Dif-ferences in therapists’ theoretical orientation, orpreferred technique, as well as differences in

chologically based problems. Conditions suchas asthma, colitis, anorexia nervosa, as well asphysical problems such as obesity and head-aches due to stress, have been considered suit-able problems for psychotherapeutic treatment.

The severity of any psychopathological disor-der depends, in part, on the individual’s reac-tion. For example, whereas depression in someindividuals may be associated with poor appe-tite, insomnia, and loss of energy, in others itmay be associated with an attempted suicide.Individuals also differ in the degree of subjectivedistress they experience and the urgency withwhich they desire and need help. Whereas forsome individuals, mild depression is so bother-some that they cannot function, others whoappear very depressed carry out normal lives.Whereas some individuals who seem only mild-ly troubled feel in great need of therapeutic help,others who have suffered major trauma are re-sistant to seeking help.

The diagnosis of a disorder, as well as itsseriousness, is thus inevitably part of a subjec-tive decisionmaking process on the part of thetherapist. The therapist is usually responsiblefor the diagnostic decision and is influencedboth by the patient’s report of the problem andby objective evidence (e.g., data on the patient’sinability to communicate with others). As de-scribed in the next section, particular therapistsmay view these problems differently, some fo-cusing on behavioral disorders, others on dys-functional thoughts and emotions. These differ-ent emphases sometimes make it difficult tocompare effectiveness across psychotherapiesand psychotherapists.

their training, contribute to the difficulty of de-scribing and assessing psychotherapy.

Professional therapists differ in a number ofways, both related and unrelated to their train-ing and disciplinary orientation. Thus, thera-pists may differ with regard to their underlyingassumptions concerning psychotherapy. They

Ch. 2—Definition and Scope of Review ● 15

may adopt a psychodynamic, humanistic/phe-nomenological, or behavioral perspective, or aneclectic set of beliefs based on several majororientations. Therapists also differ in the mo-dalities and techniques they employ to providetreatment. For example, practitioners may per-form therapy mainly with individuals, groups,or families; as well, they may focus more onearly events in the patient’s life or on recentevents, or lead a patient to insights concerningbehavior or feelings. These differences in ap-proach are not clearly associated with the thera-pist’s professional training and identification.

The nature of the relationship that a therapistestablishes with a patient may be a particularlyimportant factor in the nature of changes causedby psychotherapy. The patient reacts to thetherapist in a general way, not necessarily tospecific characteristics. Thus, therapists and re-searchers have been very concerned with thetherapist as a human being (see 274). Profes-sional therapists obviously differ in terms oftheir personal characteristics. Some of thesecharacteristics, such as sensitivity, empathy, ac-tiveness, and involvement, are seen by sometheorists (e.g., 86) as essential to successfultherapeutic outcomes. Therapists also differwith regard to age, sex, cultural background,ethnic factors, level of professional experience,psychological sophistication, social and culturalvalues, and cognitive styles. Any of these char-acteristics may have a significant effect on theway in which psychotherapy is delivered. Thesecharacteristics may interact with the practi-tioner’s theoretical perspective to produce aunique therapeutic treatment.

For the purposes of this report, it seems im-portant to distinguish professional from non-professional helpers. While it is sometimes dif-ficult to make distinctions between what profes-sionals and nonprofessionals actually do intreatment settings, the professional usually hasa very different orientation and different re-sponsibilities from those of the nonprofessional.Obviously, persons with mental distress can behelped by a variety of other individuals, includ-ing family and friends. These other individualsmay assist the patient in understanding his orher problems and in making behavioral adjust-

ments. The unique contribution of profession-ally trained therapists is important to specify.

The primary professionals who deliver psy-chotherapy are psychiatrists, clinical psycholo-gists, and psychiatric social workers (see 118).In 1977, there were approximately 28,000 psy-chiatrists professionally active in the UnitedStates. Of this number, approximately 24,500spend at least half of their time in patient care;the others are primarily administrators, teach-ers, or researchers (5). Psychiatrists are medicalschool graduates who have had a l-year intern-ship and a 2- to 3-year residency in psychiatryat an approved hospital or clinic. In addition,they must have practiced their specialty for 2years and be tested in order to qualify for cer-tification by the American Board of Psychiatryand Neurology.

One problematic aspect (103) to understand-ing the delivery of psychotherapy is that manynonpsychiatric physicians also provide psycho-therapy services. Thus, according to the Na-tional Institute of Mental Health (see 225), overhalf of the patients seen for psychiatric prob-lems are treated by primary care physicians. It isimportant to emphasize, however, that whilemany nonpsychiatric physicians are involved inmental illness diagnosis and treatment, their in-volvement may be very limited (see 60).

In terms of psychologists, according to a 1976membership survey by the American Psycho-logical Association, there were approximately23,000 psychologists engaged in mental healthactivities. These psychologists worked in indi-vidual or group practice settings, mental healthcenters, general or psychiatric hospitals, andoutpatient clinics. A 1977 survey (183) foundsimilar results, although it uncovered a slightlyhigher number (approximately 26,000) of clin-ical psychologists (because nonmembers of theAssociation were located).

Generally, clinical psychologists hold a doc-toral degree (either a Ph. D. or a Psy. D.),although some (less than 10 percent) hold only amaster’s degree. An integral part of the trainingof clinical psychologists is a l-year internship ina clinic or hospital setting approved by the

16 ● Background Paper #.3: The Efficacy and Cost Effectiveness of psychotherapy

American Psychological Association; in addi-tion, clinical psychologists typically haveseveral years supervised experience, and Ph.D.sreceive training in research (which culminates ina dissertation). A psychologist who has had atleast 5 years of postdoctoral experience may ap-ply for advanced certification from the Amer-ican Board of Professional Psychology.

A large number of psychotherapists are clini-cal social workers. In 1978, 10,922 social work-ers were included in the Register of ClinicalSocial Workers (see 2); in addition, many othersocial workers who are members of the NationalAssociation of Social Workers practice psycho-therapy. Psychiatric social workers have had 2to 3 years of postgraduate training in a school ofsocial work which typically includes an intern-ship. Psychiatric social workers with a mastersdegree in social work and 2 years of experiencemay take a test for certification by the Academyfor Certification of Social Workers.

Although the largest number of identifiablepsychotherapists are either psychiatrists, psy-chologists, or psychiatric social workers, vari-ous other professionals are trained in psycho-therapy and administer it to patients. Theseother professionals include psychiatric nurses,

DELlVERY SETTINGS

Variations in psychotherapy are also seen inthe diversity of settings in which therapy takesplace. These settings may have an important im-pact on the nature of psychotherapy. In somecases, the setting influences the type of therapyavailable to the patient. Settings may includetraditional inpatient mental hospitals, commu-nity mental health centers (CMHCs), private of-fices, residential treatment centers, and otherlocations such as schools, offices, and militaryfacilities.

Prior to the establishment of CMHCs in 1963,most psychotherapy was provided in mentalhospitals; certainly, these settings providedmuch of the societally supported mental healthtreatment. Although there is some controversy

pastoral counselors, educational psychologists,and occupational therapists, as well as a numberof individuals from other disciplines. Nurses, inparticular, play an important role in the provi-sion of psychotherapy. According to a 1972 sur-vey (see 2), almost 38,000 registered nursesreported working in a “psychiatric/mentalhealth area;” about 4,000 of them had master’sdegrees. These nurses and other professionalshave had unique disciplinary training, but theyhave also completed internships with (or underthe supervision of) psychiatrists and psycholo-gists, and they often provide treatment in com-bination with other professional therapists. Toassess the effects of psychotherapy, the role ofthese professionals needs to be recognized.

In addition, the work of paraprofessionals(i.e., psychiatric aides and trained volunteers)should be noted. The Alcohol, Drug Abuse, andMental Health Administration (2) has estimatedthat over 150,000 paraprofessionals work indirect contact with mental patients. Many psy-chotherapy settings rely on the services of per-sonnel who, though not formally trained astherapists, have a great deal of contact with pa-tients. Such paraprofessionals may play a rolein identifying patient problems and maintainingtherapy begun by the professional therapist(4,65).

as to the relationship between the developmentof CMHCs and deinstitutionalization of mentalhospital patients, there has been a sharp declinein the number of hospital beds for mental pa-tients. In 1962, psychiatric hospitals had a totalof 717,000 beds with 91.0-percent occupancy(194); in 1978, there were only 219,517 bedswith 80.7-percent occupancy (121). Althoughlarge numbers of the severely disturbed are stillhospitalized, their stays are generally muchshorter than was previously the case. The na-ture of treatment has also changed. Patients to-day have more frequent access to specific treat-ments (i. e., treatment other than being “housed”in the hospital), including group therapy, indi-vidual therapy, and psychoactive drugs (i. e.,substances that alter mental states). In addition,

patients are usually exposed to a number of dif-ferent therapists (e.g., a psychiatrist who super-vises their use of drugs, a psychologist who doesindividual counseling, and a psychiatric nurseor social worker who leads group therapy).

CMHCs and other outpatient services repre-sent the fastest growing segment of the mentalhealth delivery system. Since their inception in1963, CMHCs have been made available to al-most 50 percent of the U.S. population. In 1975,according to the National Institute of MentalHealth (197), patient visits to CMHCs totaled1,961,000. As do outpatient mental hospitals,CHMCs use a variety of mental health profes-sionals, and a patient is likely to have exposureto several modes of treatment. CMHCs tend toemphasize short-term treatment, even thoughthey often deal with chronic patients.

A large segment of mental health treatmenttakes place in private offices. Once availableonly to the well-to-do, private practice hasflourished recently as public and private in-surance coverage for mental health disorders

SCOPE OF THIS REPORT

As should be clear at this point, psychothera-py refers to a broad range of treatments used toameliorate a number of different kinds of condi-tions, by different therapists, and in a variety ofsettings. The present report, whose goal is to de-scribe the scientific basis for assessing psycho-therapy, attempts to delimit its discussion intwo ways.

First, the report attempts to describe thoseaspects of psychotherapy that have the most im-portant implications for financing and reim-

has become more widespread. It is estimatedthat there are now 15,562 physicians, 4 , 7 0 0psychologists, and 2,189 social workers engagedmore than half-time in the private practice ofpsychotherapy (2).

Several other settings for psychotherapy areworth noting. Outpatient mental health servicesare growing in acceptance as part of medicalprimary care. Health maintenance organiza-tions, hospital-based outpatient clinics, andeven some small physician groups are includingmental health professionals on their staffs (e. g.,26). In general, such services are used to supple-ment medical treatments and/or to reduce reli-ance on physical health care. Much psychother-apy also goes on in nonpsychiatric residentialtreatment centers. For example, under some cir-cumstances, homes for children with behavioraland learning problems and prisons can be con-sidered treatment centers. In addition, many or-ganizations such as schools, industries, and themilitary provide in-house resources to deliverpsychotherapy for members with mental healthproblems.

bursement policy. This is reflected in an empha-sis on severe dysfunctions (which are, potential-ly, the most costly) and an effort to distinguishthe applicability of psychotherapy to particularproblems. Second, the report attempts to de-scribe the current range of practice and researchon psychotherapy by referring to the fourcategories of variables described above. Thus,the theoretical assumptions of therapy, and thecharacteristics of patients, therapists, anddelivery settings serve as categories that need tobe considered in psychotherapy assessments.

3.Methods for Assessing the

Effectiveness of Psychotherapy

3 ■

Methods for Assessing theEffectiveness of Psychotherapy

Given the diversity of theoretical approachesto psychotherapy, as well as the range of mentalhealth problems, therapists, and delivery set-tings, it is difficult to give simple answers toquestions about psychotherapeutic efficacy. Al-though the position adopted here will be that itis possible to answer efficacy questions throughscientific research, such questions are un-doubtedly difficult to answer. Both because ofthe inherent nature of psychotherapy and be-cause of the nature of the scientific researchprocess, answers to questions about psychother-apy will require a complex sequence of steps andthe adaptation of several research technologiesto the problem of psychotherapy.

Questions about psychotherapy are complex,because, for policy purposes, they have beenstated very globally. It must be recognized thatgeneral statements as to whether psychotherapyis effective are necessarily equivocal and mustbe tempered by information as to the specificconditions under which particular treatmentsare efficacious. The way in which the level ofspecificity of one’s question affects research onpsychotherapy and the methodological consid-erations that affect the conduct and interpreta-tion of efficacy research are described below.

Most research assessing the effectiveness ofpsychotherapy has examined very specific is-sues. Which technique is more effective and howeffectiveness is moderated by differences amongpatients, therapists, and settings are the typicalfoci of psychotherapy outcome research (see207,287). Much of this research shows thatsome techniques are more effective than others,although, unfortunately, no-treatment or place-bo treatment control conditions are not alwaysincluded as part of these studies. Without suchcomparisonthe ultimateto assess.

conditions, their implications foreffectiveness questions are difficult

Questions about the effectiveness of specifictypes of psychotherapy usually deal with theconditions under which therapy is provided.Thus, the type of mental dysfunction, the char-acteristics of the patient, and the characteristicsof the delivery system are the central variablesbeing tested. Although this research can yieldgeneralizations to policy about psychotherapeu-tic treatment, the inherent limitations should berecognized. Unless generalizability has been em-pirically established by tests conducted with arange of patients in an actual treatment site, theconclusions must be regarded as tenuous.

A number of methodological issues that arisein assessing the efficacy of psychotherapy aredescribed in the following sections. The firstissue has to do with how one measures the out-comes of therapy. There is a substantial litera-ture describing procedures for determining thepresence and strength of particular results oftherapy (see 293). This literature is examinedbelow in terms of the development of useful pol-icy data about psychotherapy. A second set ofmethodological issues concerns the design ofpsychotherapy research and the confidence onecan have that obtained changes are a result of aparticular psychotherapeutic intervention. Theresearch design problems have to do with deter-mining the reason for outcomes and organizingresearch so that extraneous factors can be ruledout as the cause of treatment effects (see 41,105). Also considered below are the problems ofactually conducting psychotherapy research, in-cluding the ethical and pragmatic issues of em-ploying random assignment procedures, A sepa-rate set of methodological problems having todo with the synthesis and interpretation of mul-tiple efficacy studies is also described. A numberof techniques have been developed for review-ing and integrating findings, and, potentially,these methods may allow more definitive assess-ments of the psychotherapy literature.

21

22 . Background Paper #3: The Efficacy and Cost Effectiveness of psychotherapy

MEASURES OF PSYCHOTHERAPEUTIC OUTCOMES

Measuring the outcomes of psychotherapeu-tic treatment has been a major focus of psycho-therapy theory and research for at least the lasttwo decades. Developing a technology for meas-uring outcomes is the first step in determiningpsychotherapy’s efficacy. It involves decisionsabout what variables are important to assess, aswell as the development of measurement tech-niques that can be used in actual treatment set-tings. Because psychotherapy takes a number offorms (e.g., treatment based on different theo-retical assumptions), much of the literaturedeals with the selection of outcome measuresthat are appropriate to the goals of the treat-ment under study. One recent focus has been thedevelopment of measurement procedures thatcan be used across different types of psycho-therapy. Some of this literature has proposed“batteries” of instruments (e.g., 293), whose in-tent is to capture (through the use of severaltypes of measurement procedures) the corechanges that result from any application ofpsychotherapy.

Underlying the effort to develop such com-mon measures is a belief that no one instrumentor set of procedures can measure all the out-comes of psychotherapy (21). Different psycho-therapeutic techniques applied by a range oftherapists to various patient populations mayrequire different measures of outcome. Thus,functional measures of behavioral effects, al-though perhaps appropriate to assess behavioraltherapies, might be inappropriate to assess psy-chodynamically based therapies. Similarly, cog-nitive measures might be seen as inappropriateto evaluate behavioral therapies.

Notwithstanding the unique goals of partic-ular therapies, there seems to be support for theconcept that many of the changes produced bypsychotherapy can be assessed along some com-mon matrix. Probably, this implies the use of amatrix that includes both behavioral and cogni-tive variables. Any single study of psychother-apy, thus, would incorporate a number of meas-ures, not necessarily tied to the goals of thetherapy. The use of such multiple common out-comes also makes monitoring potential detri-

mental effects more possible.not be detectable if uniquemeasures are used.

Such effects maytherapy-relevant

Measurement Criteria

To be useful in an effectiveness analysis,measures of psychotherapy outcome must beboth reliable and valid (see 201). Reliabilitymeans that the measure gives the same findingover multiple uses (assuming no change in whatis being measured) and provides the same find-ings when used by different researchers. Valid-ity means that the measure assesses the out-comes that it is supposed to measure and pro-vides data that are generalizable (see, e.g., 48).Neither reliability nor validity is intended as atheoretical concept; each is established by pre-testing the measuring instrument.

The use of reliability and validity criteriaresults in particular measurement processes’ be-ing validated to measure specific therapeutic ef-fects. Thus, for example, a single instrumentmight not be reliable and valid for assessing im-provement in both depression and agoraphobia;reliability and validity, at least, would have tobe separately established for each condition.Moreover, outcomes have different meanings topatients, the therapist, and interested third par-ties. Validity may depend on who’s perspectiveone adopts in assessing the measurement instru-ment. Below, some of the differences in meas-ures designed to collect data from various indi-viduals affected by therapy are described.

Measures From Patients

Reports and ratings by patients of their be-havior, thoughts, and feelings represent onetypical, and usually important outcome meas-ure. Often, such data are collected on question-naires or through interviews. These measuresstructure verbal reports of the patient’s abilityto cope with various problems, and includepaper-and-pencil measures of personality andadjustment (31). Although patient assessmentmeasures yield important data, they have ob-vious limitations, These limitations include

Ch. 3—Methods for Assessing the Effectiveness of Psychotherapy ● 23

social desirability effects (patients’ respondingto create a certain impression) and response bias(e.g., positive responding to express apprecia-tion to the therapist).types of measures cansuch response patterns.

One type of patientself-report instrumentreport on aspects of

Validation using- otherdetermine the effects of

measure is a functionalwhich asks patients totheir daily functioning

(123). Although narrative reports of functioningmay be collected as a part of any therapeutictreatment, instruments that incorporate stand-ardized questions have been used to systematizethis data collection. Such an instrument may in-clude a series of structured questions about timelost from work, feelings of guilt, and satisfac-tion with therapy. Careful pretesting of thesequestions (e.g., by comparing responses of indi-viduals known to be psychologically impairedwith those not impaired) yields a subset of thesequestions. Responses can be quantified andsummed to form composite scores (total andsubset) of social adjustment and functioning. Anumber of these instruments have been devel-oped which show high reliability and validity(see 123).

Another type of patient measure is repre-sented by “psychometric” questionnaires andpersonality tests, such as the Minnesota Multi-phasic Personality Inventory (MMPI). Such in-struments, which are perhaps the most commontype of measuring tool for both diagnostic andassessment purposes, ask a variety of questionsabout the respondent’s thoughts. Patients, forexample, those suffering from depression andanxiety (210), have been shown to respond tothese questions according to particular patterns.For MMPI, a subset of questions has been devel-oped which tends to be answered differently bythose who are trying to fake responses and thosewho are not; thus, social desirability and otherbiased response patterns can be detected (see55).

Measures From Family and Friends

Friends, work associates, and relatives of pa-tients are often asked to supply informationabout a patient’s functioning, and such dataconcerning patient behavior and inferred mental

states have been used to assess therapy out-comes (e.g., 116; see, in particular, a report bythe National Institute of Mental Health, 293).The value of these measures may be due to thegreat amount of information that family,friends, and work associates have about thefunctioning of the patient. These individualshave a chance to observe the patient in a varietyof situations and may thus have a great deal of“data” on which to base their responses to ques-tionnaires. Although these individuals have self-interests which can bias the data, they may haveless motivation than the patient to reflect social-ly desirable responses and less need to showgratitude to the therapist.

The outcome variables included on thesequestionnaires also assess outcomes that may bemore important to “society” than the variablesincluded on patient self-report questionnaires.For example, questions about how much dis-turbance the patient causes in family life orwork routines, or how often the patient hassecured employment or performed satisfactorilyin school are typically included in these instru-ments. Such questions may be more importantfor societal evaluations of the usefulness oftherapy than questions about coping with dailyactivities that are typical of self-report meas-ures. These techniques are validated both bycomparing the observations of those who knowthe patient against one another (e. g., familymembers compared to work associates) and bycomparing these data with other available in-formation about the patient.

Measures From Therapists

Measures taken from the therapist and othersinvolved in the therapeutic process are anotherfrequently used source of outcome data. Whilethe therapist can provide a first-hand perspec-tive on the therapeutic process, such data maybe biased because the therapist has a vested in-terest in producing positive outcomes. Never-theless, a substantial research literature on such“clinical judgments” exists, and there is someevidence that therapists can provide useful andrelatively unbiased reports. Particularly interms of assessments of patient functioning,there is evidence that therapists can providevalid data (e.g., 67,108,161,184,198).

24 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

Generally, the more specific and concrete theobservations required of therapists, the greaterthe resulting interrater reliability and validity(186,187). These observations can range from“counts” of behavior exhibited by patients dur-ing therapy to therapist ratings of the patient’sfunctioning with his/her family (based, per-haps, on how the patient has reported familyrelationships). To assess some therapies, wherethe concern is more with mental phenomenathan behaviors, instruments have been designedto capture nonobservable behavior. While suchinstruments can provide one type of informa-tion about the effect of therapy, their useful-ness may depend on how the goals of therapyhave been described and the availability ofvalidation data.

A focus on the specific behaviors of the pa-tient may allow therapists to provide easily vali-dated data about patient functioning; however,the therapist’s desire to show improvements as aresult of therapy may bias the use of such meas-ures. In an attempt to remove this bias fromfunctioning judgments, some researchers havetrained members of a therapy staff who are notindividuals actually involved in therapy to con-duct such assessments (e.g., 209). A variety ofsuch “blind” data collection techniques havebeen employed.

Measures From Community Members

Information on the outcome of psychother-apy can also be collected, more broadly, fromcommunity members or agencies. These meas-ures may include patient data on criminal arrestrates, measures related to the patient’s work, orrates of medical utilization (from nonpsychiatricillness). The range of potential communitymeasures is very large and, though often un-wieldy, seems to reflect important informationthat is needed to assess adequately the effects ofpsychotherapy. As is described later in the dis-cussion of cost-benefit measurement (ch. 5),

such variables are important to assess in orderto conduct comprehensive cost-effectivenessand cost-benefit analyses (CEA/CBAs).

Many of the data collected on communityvariables can be assessed in relatively directways. The information often can be gleanedfrom existing court records, hospital charts, in-surance claims, and similar records. To be uti-lized as part of an effectiveness assessment,however, such measures must be tested for relia-bility and validity. Oftentimes, the validity ofrecord data is easy to establish because of its ob-vious relationship to desirable psychotherapyoutcomes (referred to as content validity). Itshould be noted that much of this informationcan also be collected directly from patients (e.g.,data on medical utilization, if it can be shownthat patients report these data accurately).

Community data have also been taken tomean a patient’s economic contribution to his orher community (see, e.g., 305). Thus, the pa-tient’s net monetary contribution in terms ofearnings and taxes may be used as one measureof psychotherapeutic outcome (see ch. 5). Theseoutcomes have been assessed for psychotherapyin a number of investigationsare described later in termspsychotherapy.

Summary

(e.g., 51,114) andof CEA/CBAs of

The outcomes of psychotherapy can be meas-ured in a variety of ways. It is probably im-possible to develop any single measure of out-come which would reflect the diverse changesthat might be brought about by psychotherapy.At present, there exists a diverse set of pro-cedures for eliciting information from patients,family, therapists, and others. Data from eachof these sources, if properly assessed for reliabil-ity and validity, provide information needed toassess psychotherapeutic effectiveness.

RESEARCH DESIGNS

The development of reliable and valid out-come measures represents only one part of theassessment problem. One must be able to deter-mine which of the many processes utilized intherapy are responsible for improvements andmust also be able to test the possibility that non-therapeutic components are responsible for ob-tained effects. The particular research designsand techniques used to test psychotherapeuticefficacy are, in part, determined by what ques-tions are being asked. These questions dependon the goals of both the patients and the therapyprogram. The present discussion focuses on thepossible ways of carrying out psychotherapyoutcome research and how questions about psy-chotherapy can actually be tested.

A basic element of good research design is toframe the questions to be tested in a very spe-cific way. Whereas the global question of psy-chotherapy’s efficacy may be obvious (i. e., “istherapy effective?”), the questions asked ofresearch need to be more circumscribed(21,207). Such circumscribed questions usually

develop as research progresses through a seriesof stages (105). At a formulative stage, researchis based on extensive observation and summarydescriptions of these observations. Then, thefocus shifts to a description of patterns amongdata elements. At a later stage, explanations andtheories of the observed patterns are formed. Itis usually these later explanations that are testedin formal experiments and that represent thebulk of the outcome research literature. Al-though effectiveness studies can be conductedon a post hoc basis (i. e., a design constructedafter the data have been collected), such re-search usually can be interpreted only whentheory and formal experimental evidence areavailable (e. g., 47,1.51 ).

The purpose of the present report is to under-stand psychotherapeutic effectiveness; thus, thefocus is primarily on data collected in actualtreatment settings (in vivo). A great deal of psy -

chotherapy research has been carried out underlaboratory or analog conditions, however, andat least some researchers (e.g., 10) view the datafrom this research as very important. To the ex-tent that such data provide theoretical supportfor in vivo findings, they are probably neces-sary to consider. OTA, in previous discussionsof the evaluation of medical technology (202),has regarded this as efficacy rather than effec-tiveness research (ideal v. actual conditions).When applied to psychotherapy, however, theefficacy/effectiveness distinction seems ambigu-ous because of the variety of factors (therapist,patient, setting) which affect the outcome of aparticular treatment and the absence of a cleardemarcation between laboratory and nonlabo-ratory conditions. It seems desirable in the caseof psychotherapy, instead of trying to differenti-ate between assessments of efficacy and effec-tiveness research, to note clearly the differentconditions under which research is conducted.

In terms of developing research tests of thera-peutic effects, some limitations of prevailing sci-entific logic should also be noted (see 41), Test-ing particular hypotheses does not permit thepsychotherapy researcher to prove that a partic-ular therapeutic effort causes a particular out-come; instead, one tests whether alternative ex-planations can be disregarded (47,105). Result-ing inferences are probabilistic and indicate,within identifiable error rates, the likelihoodthat generalization can be made from the study’ssample to other populations.

The inherent variability of human behavior,thoughts, and feelings often makes the findingsof psychotherapy research equivocal. A greatdeal of variability will exist under any condi-tions, and these variables must be separatedfrom the effects of the treatment. In addition,because patients can be affected by many vari-ables (i.e., factors other than psychotherapy)that cannot be controlled, there may be numer-

ous alternative explanations for any apparentimprovements produced by psychotherapy. Thenumber of alternative explanations can be re-duced somewhat by careful design of psycho-therapy research and by the inclusion of controlconditions that hold all elements constant, ex-cept some aspect of a treatment. A variety of ex-perimental designs have been developed to re-duce the plausibility of a number of standardalternative explanations. These research designsare not unique to psychotherapy (see 27,41,94,165), but their use in assessing the efficacy ofpsychotherapy raises a significant set of uniqueproblems. Several types of research designs thatcan be used to assess psychotherapy are de-scribed below.

Therapy Versus No-Therapy Designs

The classical research design, as applied topsychotherapy, assigns patients either to receivepsychotherapy or not to receive psychotherapyaccording to a random selection procedure. Itcan be expected that randomization will distrib-ute differences in patient characteristics (e.g.,level of mental dysfunction, amenability totreatment) equally between the psychotherapyand no-psychotherapy conditions. Typically,the psychotherapy condition is referred to as the“experimental” condition, while the no-psycho-therapy condition is referred to as the “control”condition. The functioning of each subject in theexperiment is measured following psychother-apy (or following an equal interval of time forcontrol subjects). Although measures can betaken at other times (e.g., pretherapy), addi-tional design problems (having to do with theeffect of taking a test or responding to a ques-tionnaire more than once) result with suchprocedures.

The measurements of functioning (includingcognitive and behavioral outcomes) obtainedfrom the treatment condition contain three pos-sible “effects:” 1) “effects” of treatment, 2) “ef-fects” due to whatever nontherapy factors areaffecting the patient, and 3) “effects” of thehaphazard fluctuations in functioning measurescaused by imperfections in measurement instru-ments and the variability of behavior. Themeasurements of patient functioning obtained

from the control condition are used to “sub-tract” the “effects” due to treatment from the“effects” due to other factors. A control condi-tion is necessary to perform the above “subtrac-tion” because it provides the only empirical wayof knowing how nontherapy factors and meas-urement problems affected the outcome.

When this type of therapy versus no-therapyresearch design is used, explanations of appar-ent improvements in patient functioning thatare actually due to factors other than therapyitself can be rejected with reasonable confi-dence. As long as patients have been assignedrandomly to therapy and no-therapy condi-tions, it can be assumed (within identifiableprobability limits) that the obtained “effects”are due to the psychotherapeutic treatment. Ifpatients are not randomly assigned, but are“matched” on various characteristics, such anunequivocal statement is not possible. It can beargued that differences existed between experi-mental and control group subjects that were notcontrolled by matching and that these charac-teristics are responsible for differences obtainedbetween therapy and no-therapy groups. Theabsence of a control group makes such in-ferences about causal factors extremely difficultto develop.

Therapy Versus Therapy

The basic rationale used to distill the effects ofpsychotherapy from effects of measurement andfactors unrelated to therapy can be extended totest for the superiority of one psychotherapyover another. In such a therapy comparisonstudy, patients are assigned randomly to ther-apy A, therapy B, . . ., and, perhaps, to a no-therapy or delayed-therapy group. In effect,such a design results in the use of multiple treat-ment groups. The use of no-treatment controlgroups is not ruled out, but such groups areoften not employed in therapy comparison stud-ies because the purpose of such studies is theassessment of the best therapy. It should benoted that comparisons across therapies areoften made without use of an experimental re-search design. When that is done, patients whohave received different therapies are comparedwithout regard to the selection factors that in-

fluenced which patients received which therapy.In such nonexperimental research, differencesmay be due to a variety of factors (e.g., pre-existing differences between patients in eachgroup), and these variables need to be con-trolled before such data are useful.

In another common design used to comparetherapies, two potentiality effective therapytechniques are presented separately to twogroups of subjects and in combination to a thirdgroup (yielding individual and combinationtherapy conditions). Statistical analyses areused to separate the effects of the therapies andto compare them with no-therapy conditions. Aparticular concern of the statistical analysis is totest for interaction effects (e. g., where therapiescombine to produce an effect that is differentfrom the sum of the two effects alone). A vari-ant of this type of design has been used to assessthe joint, as well as separate, effects of chemo-therapy and psychotherapy (e.g., 142).

Therapy Versus “Placebo” Therapy

One problem that has plagued much psycho-therapy research (as well as other research onmedical interventions) is that some of the effectsobtained by psychotherapy researchers may bedue to placebo effects. The “aura” of being intherapy and the expectancy that one is finallyabout to be “cured” may be a form of treatment(136,256). The problem of separating these ef-fects from those of formal therapies is analogousto the use of sugar pills in controlled medicalresearch and involves the use of placebo-controlconditions. In such conditions, the patient mayreceive attention from a therapist, but therapeu-tically meaningful discussion is avoided and nospecific techniques are used (e.g., 176). It iseasier to employ such control procedures whentesting the efficacy of behavioral, or even psy-chodynamic, therapies, in which the therapistplays an active and directive role.

Probably, because of the nature of psycho-therapy and the difficulty of specifying theprecise ingredients of therapy, it is impossible tocontrol all placebo effects (207,280). The rela-tionship that a therapist establishes with a pa-tient is acknowledged (e. g., 84) to be an impor-tant component of therapy, and it is not clear

how such effects should be distinguished fromtreatments per se. It should also be recognizedthat while placebo effects may inflate the true ef-fects of therapy, they may often distort the dataobtained in control conditions. Few control con-ditions can be “pure” in the sense that patientsdo not interact with a therapist. If for no otherreason than to monitor the patient, those in con-trol conditions must usually be supervised by atherapist. The effects of this supervision maymake therapy appear to be less effective.

Control conditions may be introduced toassess the effects of such factors as therapist “de-mand characteristics. ” An ardent researcher ortherapist may communicate to patients what theresults of an experiment are “expected” to be(e.g., 240). Experiments using se] f-report meas-ures obtained from patients are especially proneto this problem. To examine the strength ofthese demand characteristics, psychotherapyresearchers can use designs in which patients aretold that therapy will produce a temporary

worsening of functioning, when in actuality theresearcher expects no worsening. Alternatively,the researcher may tell patients in some condi-tions that therapy should not be effective forseveral months, when gradual improvement ac-tually is expected. If the “demanded” effects arenot found in these conditions, the researcher canhave some confidence that demand character-istics are not causing therapy effects. Althoughthese controls may not be employed in everyoutcome study, they are often used in develop-ing research designs.

Quasi-Experimental DesignsFor a variety of reasons, including ethical

problems of withholding treatment (see discus-sion below), the experimental designs describedabove are often not employed. The real or per-ceived difficulties of assigning patients on a ran-dom basis to therapy or no-therapy conditionsmake other types of comparisons necessary.There are a number of quasi-experimental de-signs that can be used in such circumstances (see41), Such designs are sometimes considered tobe poor substitutes for “pure” experimentaldesigns, but this may not be the best way toview them. Quasi-experimental designs, if care-

28 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

fully constituted, eliminate the most importantplausible alternative explanations for the resultsof an experiment and can provide useful infor-mation. Quasi-experiments based on thesedesigns, however, may involve complex statis-tics and require the collection of many moredata (perhaps from fewer subjects) than a “true”experiment would.

One common quasi-experimental design inpsychotherapy is referred to as an intensivedesign (97). Typically, intensive designs usefewer subjects than experimental designs, butcompensate by obtaining more measures on afrequent basis and by obtaining clear “baseline”measures of patient functioning. The intent isthat sudden improvements will be dramatic andclosely correlated with the onset of therapy. Tothe extent that these changes are sudden, manyof the alternative explanations can be ruled out(despite the lack of a control group). The use ofintensive designs, however, depends on the typeof therapy and the outcomes that are expected.Only therapies that posit observable and rapidchanges (typically, the more behavioral thera-pies) can be tested with such designs.

Return-to-Baseline Designs.—In this design,patient functioning is measured several timesbefore therapy (the baseline period) and severaltimes during therapy (the manipulation period).Therapy is then withdrawn abruptly (reversal tosecond baseline), and it is expected that therewill be a return to lower levels of functioningsimilar to those found during the first baselineperiod. Repeated measurement continues duringthis period and during a return to therapy(second manipulation). Therapy is then phasedout with the hope that improvements will bemaintained.

An example of this type of design is Allen,Hart, Buell, Harris, and Wolf’s (3) study of asingle child. The child had isolated herself fromother children, causing severe psychologicalproblems. The amount of time that the childspent with adults and other children was meas-ured during the baseline period. A simple ther-apy that provided reinforcement for her inter-actions with other children was then provided.Measurement continued during this period, as

well as after therapy. Later, there was an abruptwithdrawal of the reinforcement (reversal tosecond baseline), followed by its reinstitution.The results showed very noticeable changes inthe girl’s interactions. In the absence of otherfactors (besides therapy), the effects were seenas demonstrating therapeutic efficacy.

Intrasubject Multiple Baseline Designs.— Toobtain information analogous to that obtainedfrom a separate no-therapy control group, it isalso possible to separately treat and assess func-tioning in different areas of the patient’s life.Thus, treatment is designed to improve oneaspect of functioning, and the patient’s otherbehavior is used as a control. Such a design,while having advantages because changes canbe compared with only one subject, suffers fromthe obvious problem that effects may generalizeacross function areas. In fact, for some therapiesgeneralization of effects could be expected, andan intrasubject multiple baseline design wouldlead to “missing” the effects of therapy.

An example of this type of design is pro-vided by Morganstern (189). The effectivenessof aversive conditioning on obesity (pairingnauseous smoke with eating certain foods) wasexamined by conditioning a patient to one typeof food, then after several days, to another. Be-cause eating of other foods was found not tochange until aversive conditioning was appliedto the particular foods, it was possible to inferthat the therapy produced changes.

There are a number of variations on thisintrasubject multiple baseline design. Multiplesubjects can be used, and different combinationsof treatment can be tested. This design does noteliminate alternative explanations due to place-bo or demand characteristics, but can eliminatesome problems inherent in nonrandomized ex-periments. Thus, additional subjects can serveas controls to assess the importance of factorssuch as spontaneous remission (improvementwithout treatment). However, the use of thisdesign may be severely limited by the types ofproblems (very specific) and the therapies(mostly behavioral) which one tests.

Program Evaluations

The design considerations described aboveimplicitly refer to psychotherapy as a unitarytreatment that can be applied or not appliedto form experimental and control conditions.However, as chapter 2 points out, psychother-apy treatments are comprised of a number offactors. The theoretical orientation of the ther-apy, usually the basis of a label for the treat-ment, may not adequately describe the treat-ment as it is actually delivered. Depending onthe nature of patients’ problems, therapists’ ori-entation and skill, as well as aspects of the deliv-ery setting, therapy may have different out-comes. One possibility is to treat each of thesefactors as an independent variable and to con-struct multifactor designs. In such designs, sub-jects are randomly assigned to different thera-pies, therapists, and settings. A more recenttrend, though, has been to conduct such out-come research through program evaluations(see 212). In such evaluative research (e.g., 9,159,252), one evaluates a complex of treatmentvariables that have been organized as a program(e .g . , a community mental health center(CMHC)).

Thus, for example, a Program evaluationstudy of a CMHC tries to assess how and towhat extent patients who receive treatment atthe center are aided. The CMHC, in addition tobeing community based (which is hypothesizedto be an adjunct to treatment), may offer pa-tients a number of therapies, and patients maybe treated by multiple professionals and para-professionals. Under such circumstances, wherejoint effects of these treatments are expected andwhere it is extremely difficult to separate—forresearch design purposes—the components oftreatment, program evaluation yields a designthat may be more compatible with the actualcircumstances. Evaluative research does notpreclude the conduct of experiments where in-dividual aspects of the treatment are assessed.

The designs for program evaluation studiesof psychotherapy can include aspects of thetrue and quasi-experimental designs describedabove. In general, the same methodologicalconsiderations for research designs apply to

program evaluations (229,298,303). There is, infact, a substantial literature on the use of the ex-perimental designs in program evaluation (e.g.,230,249). The literature describes both howcomplex variables such as psychotherapy can beconceptualized and the conditions for imple-menting randomized designs. Similarly, theliterature on quasi-experimental designs hasbeen related to program evaluation (e.g., 41,230). The principal difference between these de-signs and traditional research design is in howone conceptualizes the treatment. In a programevaluation study, the treatment includes a num-ber of elements. These elements, at least in theinitial stages of such an evaluation, are not sepa-rately tested.

Even though program evaluations are de-signed specifically to aid in policy decisionmak-ing, there are a number of endemic problems.Just as it is difficult to organize a traditional psy-chotherapy outcome study (i. e., randomly orotherwise assign patients to treatments), it is dif-ficult to organize program evaluation studies. Infact, because the randomization units are morecomplex, such evaluation studies are often verydifficult to carry out well, and there is a sub-stantial literature about implementation fail-ures. In addition, program evaluation studiesmay not resolve the underlying conceptualproblems involved in assessing psychotherapy.It may be difficult to determine through a pro-gram evaluation what factors were responsiblefor the success or failure of the program.

Difficulties in Conducting Research

There are a variety of problems which make itdifficult to conduct psychotherapy outcome re-search. Some of these problems, which relate tothe inappropriateness of some methods to testparticular therapies (e.g., intensive designs) andthe problem of multiple factors affecting out-come (e. g., the role of therapist variable), havealready been described. There are also prob-lems, however, which are perhaps more impor-tant, having to do with the pragmatic and eth-ical difficulties of conducting experimentalresearch.

30 Background Paper #3: The Efficacy and Cost Effectiveness of psychotherapy

While the advantages of experimental meth-ods to develop unequivocal data are well knownand widely accepted, there are obvious ethicalproblems connected with decisions to withholdtreatment (e.g., 45,229) or to make treatmentavailable on a random basis. Especially if onebelieves that therapy is efficacious, it is dif-ficult not to allow certain patients to receivetreatment for research purposes. Several consid-erations are important in thinking about thisdilemma.

One important consideration is the necessityof conducting research, especially that whichemploys randomized control groups. Such stud-ies (which, in medicine, are referred to as ran-domized clinical trials) make it possible to assesscausality in the most unambiguous way (see,e.g., 36,41,165). Although, from a theoreticalpoint of view, the value of randomized controlgroup studies has long been recognized, there isnow some evidence that such studies better en-able researchers to detect inefficacious treat-ments (see, e.g., 94). If less adequate designs areused (i.e., designs not employing randomizedcontrol conditions), decision errors may result.Thus, it may appear, perhaps because of the ef-fects of other variables, that the treatment is ef-fective when it is not.

One resolution to the ethical problems pre-sented by needing to withhold treatment is sug-gested by the nature of the dilemma. If a treat-ment has not been demonstrated to be effective,then it may not be unethical to deny treatmentto some individuals. The treatment, in theabsence of empirical data, may not be accom-plishing anything. Actual treatment providers(i.e., therapists) may not share this view and,therefore, may be reluctant to participate in thisform of experimental research. At the veryleast, practitioners might want to be able tosupervise control group subjects and, in effect,provide partial treatment.

One pragmatic resolution to the ethical dilem-ma of this research is to provide control groupsubjects with access to treatment once the ex-periment has been concluded. This is often re-ferred to as the “waiting list” approach. It ismade a more attractive option in some studies(which compare several therapeutic approaches)

by offering the delayed control group subjectsthe form of treatment which the experimentdemonstrated as best. Obviously, this optioncan be provided only when therapy is of a rela-tively short duration. It is, however, easy to im-plement when resources are very limited. If onlya small group of patients can be given treatmentat any one time, a waiting list (established byrandom assignment) may be practicable (al-though it might be viable only when patients donot have other treatment options).

Whatever the justification for random assign-ment to treatment and control conditions, ex-plicit guidelines (established by the Departmentof Health and Human Services and professionalsocieties) must be followed to protect patients’rights. One central principle of these guidelinesis that participation be voluntary, based on pa-tients’ (or legal surrogates’) “informed consent”(see, e.g., 6). Informed consent procedures re-quire that subjects be informed as to the pur-poses of the experiment, known risks, data to becollected, and how the data will be used. Sub-jects also have the right to leave the experimentat any time and, usually, must be told of othertreatment resources. The procedures also re-quire that review panels be established to ap-prove research protocols and monitor the con-duct of this human research.

Although such procedures may create differ-ential attrition across conditions of an experi-ment, and perhaps result in only certain types ofpatients’ or treatments’ being involved in re-search, it does not rule out the use of experimen-tal designs. Under a variety of circumstances,individuals will agree to participate in experi-mental research, and the problems may havemore to do with the researchers or therapists in-volved than with patients. As noted above, untilunequivocal evidence is available about psycho-therapy and more resources are available forsuch treatments, it seems necessary to conductsuch experimental studies. These studies arecritical to providing unequivocal tests of thetheoretical hypotheses suggested by basic re-search. For many therapies, without experi-mental evidence, there will be no way of resolv-ing questions about alternative explanations fortheir effects (see 207). In order to carry out such

studies, it may be necessary for the Governmentto develop special rules for their conduct. Thus,for example, special reimbursement proceduresmay need to be applied so that treatment and re-search costs can be separated.

Given the difficulties of conducting in vivoexperimental research, it is perhaps not surpris-ing that much of the best controlled research hasbeen laboratory analogs to clinical settings. Inthese analog settings, psychotherapy is offeredto patients who often have less severe dysfunc-tions than are seen in psychotherapy clinics.However, these patients can be assigned to no-treatment and placebo conditions because thepsychotherapy researcher controls the programand because the loss of the therapeutic benefithas less impact. This type of research may yield

more rigorous experimental findings, but haslow external validity; that is, the patients (oftenundergraduate students), therapists (often doc-toral students), and procedures (more theoreti-cally oriented and less eclectic than is typical)are not representative of those used in “real”psychotherapy (e.g., 162,169). It should benoted that some researchers suggest that this isthe best way to conduct rigorous investigationsof the effectiveness of psychotherapy and thatthe innovative techniques developed and testedin such settings can be transferred to more“messy, ” real world settings (10, 135, 136). I nessence, they argue that the differences betweenactual and analog settings are not as great asmight be anticipated and that generalizabilitycan be demonstrated for these analog studies (atleast for some therapies and conditions oftreatment ).

INTEGRATING FINDINGS FROM PSYCHOTHERAPY RESEARCH

Although the key to evaluating the efficacy ofpsychotherapy lies in the conduct of well-designed research which uses multiple measuresthat are both reliable and valid, this may not besufficient for policymaking purposes. No oneresearch design (including true experiments andprogram evaluations) will enable the develop-ment of definitive information about the effectsof psychotherapy, nor will the measures used inany one study be adequate for all purposes. Inpart, this is because of the current state oftheorizing about psychopathology, which en-compasses a number of approaches each withdifferent ideas about research. The problems,however, are not only theoretical, but also re-flect the limitations of the scientific process withrespect to developing unambiguous conclusionson the basis of individual research studies.Given the diversity of criteria against whichsuch psychotherapy research must be evaluated,as well as the divergent theoretical views,methods are required to synthesize, aggregate,or integrate the findings of multiple studies.

Despite the fact that it might be desirable tohave a few studies which “settle” the effec-tiveness question, for the most part, it is neces-

sary to treat cautiously the results of individualstudies. Their results must be judged againstother research designed to test the same or simi-lar hypotheses. Traditionally, such judgmentshave been made through literature reviews,where scholars analyze a body of research.These scholarly analyses are reviewed by peersand pub] i shed so that o t her researchers cancomment. Because of the scientific standardsunderlying peer review (see, e.g., 39), most suchreviews reflect honest efforts to weigh the ap-propriate evidence.

However, even though the evidence may bereported accurately, it is fairly easy to be selec-tive about the research data one includes. It israrely possible, especially when considering thehost of problems for which psychotherapy hasbeen applied, to include all potentially relevantresearch. Any researcher who conducts a reviewmakes a number of implicit and explicit choicesabout what will be included. As well, choicesare made about what elements of the studies willbe discussed and those that will be ignored; ineffect, the author “slants” the review to supporta particular hypothesis or viewpoint. Clear cri-teria are not always provided for judging the

methodological adequacy of the studies in-cluded in the review.

To make this process more scientific, or atleast more systematic, a number of procedureshave recently been developed and applied toassessments of the psychotherapy outcome liter-ature. These procedures, although they havetheir own limitations and are by no meansunanimously accepted, represent attempts tomake sense of the burgeoning and often contra-dictory research about psychotherapy. Twosuch procedures are discussed below: “box-score analyses” and “meta-analyses.”

Box-Score Analyses.—The procedure of“box-score analysis” begins with identificationof a population of research studies (see, e.g.,162). Usually, the reviewer establishes certainstandards and excludes studies that are not suffi-ciently rigorous by methodological criteria, orare otherwise not appropriate. The latter cate-gory might include studies that are well de-signed but lack sufficiently reliable or valid out-come measures. It might also include studiesthat do not have patients assigned randomly toconditions. The difference between this proce-dure and that used in literature reviews is notsharply defined and is only the greater degree towhich criteria are explicitly described in box-score analyses and used to select research for thereview.

All studies meeting the reviewer’s criteria areculled from some defined set of sources (e. g.,journals) and are then sorted into categories.Typical categories might be dysfunction treated,therapy technique, and/or training of therapist.Finally, the reviewer evaluates each study’s out-come (e. g., “yes, ” “no, “ “equivocal”) and talliesscores using the predefine categories (e. g., dys-function, technique, therapists).

This method of categorization and then tally-ing of effects is designed to systematize the lit-erature review process. However, it still leaves agreat deal of room for individual judgment andtypically uses a rather simplistic measure oftreatment outcome. Importantly, it does nottake account of the strength of findings withinparticular studies. A box-score analysis does nottake account of the methodological rigor of astudy, except in global fashion (e. g., by exclud-

ing studies without a particular kind of controlcondition). It is, nevertheless, a useful proce-dure, and a number of important box-score-typereviews of the psychotherapy literature are re-ported in chapter 4.

Meta-Analyses.—As described by Smith andGlass (267) and Smith, Glass, and Miller (268),“meta-analysis” is a procedure for integratingresearch findings (see also 239). It is a more re-cent, and undoubtedly controversial, set of pro-cedures (see ch. 4). Meta-analysis employs sta-tistical techniques for aggregating data and fordetermining relationships between causal varia-bles and outcomes. Usually, the first step in ameta-analysis is the precise description of a pop-ulation of studies on which the analyses will bebased. In this respect, a meta-analysis is con-ducted similarly to a box-score analysis.

In a meta-analysis, however, studies are thencoded on a set of variables that are thought to berelated to outcomes—the number of these vari-ables is at the discretion of the analyst. The ex-perience of the therapist, the patients’ symp-tomatology, the quality of the research design,and the setting of the treatment are examples ofthe categories of variables that would be codedin an analysis of psychotherapy. These meas-ures are later correlated with the outcomes(usually quantified) and used as the basis fororganizing outcome results in terms of aspectsof the studies. In Smith, et al. ’s (268) work, anoutcome measure for psychotherapy based onthe size of the effect (standardized) was com-puted for almost 500 studies. The studies camefrom a population of controlled (i.e., compari-son group) investigations. Although a global ef-fect size measure, as calculated by Smith, et al.,does not differentially weigh studies accordingto the quality of the measures employed or thedesign, such factors are controlled by codingeach study in terms of instrument and designvalidity. If, for example, there is a difference be-tween the size of the effects in studies that usedpoor measures and those that used good meas-ures (i. e., low valid v. high valid), then oneknows that a bias of some sort is operating.

Meta-analysis uses systematic methods to un-cover trends in the available research literature.

Assuming that there are some good studies andcoding criteria can be agreed on, it should proveto be a useful tool in understanding the researchon the effects of psychotherapy. A potentialproblem, however, is that by focusing on avail-able research, one ignores the fact that only re-search that reports positive and/or statisticallysignificant findings may be published. Althoughsome believe (241 ) that this could be a problem

FINAL COMMENTS

This chapter has described the methodologi-cal strategies that can be used to assess psycho-therapy. Implicit in this discussion was an as-sumption that psychotherapy represents a re-searchable intervention that can be evaluatedusing scientific criteria of measurement anddesign. Valid measures of psychotherapy out-come can be developed, and designs that allowrelatively unequivocal assessment of psycho-therapy can be constructed. Although there aresome who maintain that it is not possible toassess psychotherapy because of its inherentcomplexity, it is not clear what types of in-formation would be excluded by the scientificanalyses described here.

Whether the methods described here havebeen applied appropriately and to what endsthey have been used, however, is a differentquestion. Despite the possibility of conductingresearch on the outcomes of psychotherapy, asis noted in the next chapter, well-conducted re-

only when there are a few studies reporting sig-nificant findings and when the magnitude of thedifferences between groups is very small, obvi-ously, the quality of available research is a criti-cal factor in the usefulness of meta-analysis.Other problems, such as the compatibility ofstudies, are described in detail in the next chap-ter as part of a substantive review of Smith, etal.’s work.

search is inadequate to answer at least some ofthe important questions about psychotherapy(207,277). The reasons for the lack of researchare not clear, but probably have to do with bothsubstantive features of psychotherapy and acomplex set of pragmatic factors.

These factors include the difficulty of concep-tualizing the multiple factors that are part ofpsychotherapy, as well as attitudinal differenceson the part of researchers and therapists. Inpart, the development of program evaluationtechniques may facilitate the conduct of policy-relevant research. In addition, more attention,by researchers, therapists, and the Governmentto the issues of conducting ethically acceptablecontrolled experiments, may help to developbetter research. Finally, attention to the issue ofconclusion-making, based on an analysis ofmultiple investigations, may lead to better un-derstanding of exactly what is achieved by psy-chotherapy under what conditions.

I

“Literature on the Effectivenessof Psychotherapy

4.Literature on the Effectiveness

of Psychotherapy

Undoubtedly, there is confusion about the ef-fectiveness of psychotherapy; both in the pro-fessional and popular literature, a variety ofclaims have been made about the worth of psy-chotherapeutic treatments. Although it is notpossible here to resolve fully the differences be-tween conflicting viewpoints, the present chap-

SCOPE OF REVIEW

Although no one has classified all the researchconducted on psychotherapy, it is clear fromeven a cursory review of the literature that thereis a plethora of research (see, e.g., 208,276). Theresearch ranges from very specific analysesof psychotherapeutic procedures to large-scaleprogram evaluations. Clearly, there are differ-ences in the usefulness of particular types of re-search for the general question of psychothera-py’s efficacy, but it is difficult to differentiateresearch according to a relevance criterion.

While some commentators have made a dis-tinction between basic and evaluative research(e.g., 66, 268), the utility of any research toanswer efficacy questions is usually a matter ofdegree. Basic research on psychotherapy doesnot indicate whether or not treatments are effec-tive as practiced, but does provide some under-standings about the nature of the intervention.For those concerned with understanding theprocesses underlying psychotherapy, such dataare very important. Conversely, evaluativestudies, although they provide direct informa-tion about the actual effectiveness of a treat-ment, often yield equivocal analyses of causali-ty and must be interpreted in conjunction withbasic research data. In the present chapter, cate-gorizations are avoided; the analysis includes avariety of research studies which have been con-sidered, by at least some reviewers, to be rele-

ter briefly reviews the major areas of disputeand attempts to apply the methodological prin-ciples described in the previous chapter to assessthe various claims about psychotherapy. The le-gitimate disagreements among reviewers are ac-knowledged, and an effort is made to assess thedifferences objectively.

vant to understanding the efficacy problem.Several additional caveats should also be noted.

One important issue relating to how one as-sesses the efficacy literature is the nature of theproblems for which psychotherapy treatmentsare offered. Thus, a great deal of research ex-amines the treatment of very specific nondis-abling mental/behavioral problems, and it isdifficult to assess the generalizability of thisliterature to understanding the efficacy oftreatments for very severe and disabling condi-tions. Unfortunately, the problem is even morecomplex, for much of the psychotherapy litera-ture deals with conditions that are difficult toclassify. In part, this problem has to do with thefact that definitions of severity are relative anddependent on the context in which the problemexists. The present chapter attempts to clarify,wherever possible, the degree to which findingsabout particular problems are generalizable toother mental health problems.

In terms of the scope of the present chapter,the main focus is on a description and analysisof previous reviews of the literature. As dis-cussed in chapter 3, no matter how well con-ceived and executed, single studies have limiteduse within a scientific framework. A number ofvery extensive reviews of the literature (i e.,discussions of multiple studies) have alreadybeen conducted, and an effort is made in the

37

38 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

present chapter to describe these reviews as fullyas possible. Where appropriate, as part of thisanalysis, the data and methods of particularstudies discussed by these reviews are presented.OTA’s decision to focus on reviews has to doboth with a goal to best represent the availableliterature and with the fact that these reviewscomprise a focal point for much of the currentdebate about psychotherapy’s effectiveness.

One omission from the literature reviews inthe present chapter are popularized summariesof the psychotherapy literature. For example,Gross’ recent book The Psychological Societyhas not been included. Gross’ treatise is an inter-esting (although critical) report on psychothera-py, but is primarily a secondary source reviewof the literature (see also 279,281). Instead ofdiscussing such work directly, the chapter de-scribes the scholarly reviews on which Grossbased his book. Not included here either are

EFFICACY REVIEWS

In the reviews described below, the data re-ported are reviewed first; then, some of thecommentary that has been stimulated by the re-view is discussed. The methods used by the re-viewer to select studies and analyze them arecompared against the methodological criteriadescribed in chapter 3. As noted above, the goalis to describe the research that is most relevantto the efficacy question and to assess what ap-pear to be the most reasonable implications ofthis research literature.

Eysenck’s Reviews

The earliest and probably the most controver-sial review of the psychotherapy outcome litera-ture was conducted by Eysenck almost 30 yearsago (70; see also 71,72), Eysenck, in his earlyreview, considered 24 research articles which in-cluded 8,053 cases of psychotherapy conductedwith neurotic patients. He divided the studies,on the basis of his assessment of their thera-peutic approach, into two groups: 1) psycho-analytic therapy and 2) eclectic therapy. Theprincipal criterion for assessing effectiveness

some of the scholarly literature reviews whichhave as their primary focus the contrast of onetherapy or approach with others. We were cen-trally interested in reviews that dealt, mostgenerally, with the problem of whether psycho-therapy is efficacious under what conditions.The central focus of a review on this questionguided OTA’s selections.

The reviews discussed below are presented inapproximate chronological order according totheir appearance in the literature. It is hopedthat this ordering will give readers a sense ofhow research and thinking about effectivenessof psychotherapy have evolved during the pastseveral decades. The chapter also describes thefindings of a number of the most importantstudies on outcome. These descriptions are pre-sented, where necessary, to facilitate under-standing of the issues raised by reviewers.

was a rating Eysenck developed of improvementfollowing therapy.

The most controversial aspect of Eysenck’s re-port was his comparison of improvement ratesin therapy against an improvement rate that hecalculated for two no-treatment groups. In theeclectic therapy studies, Eysenck calculated animprovement rate for therapy of 64 percentwithin 2 years. He compared this rate with whatwas achieved in comparison groups of patientswho did not have therapy. Eysenck found thatthe no-treatment improvement rate was approx-imately the same as that achieved in the eclectictherapy studies. The improvement rate calcu-lated for psychoanalysis was lower, approx-imately 44 percent, and was below that of theno-treatment groups, One source of Eysenck’sdata on no-therapy improvement was a studyby Denker (61) of patients with emotional prob-lems who were treated by general medical prac-titioners (another source was Landis (153)).Denker’s data consisted of insurance company

records on 500 individuals who had submittedmental disability claims. He had found that

Ch. 4—Literature on the Effectiveness of Psychotherapy ● 39

within 1 year, without receiving any specificpsychotherapy, 44 percent had returned towork, and that by the end of 2 years, an addi-tional 27 percent had returned to work.

Eysenck’s work unleashed an extraordinaryreaction, some of which still affects thinkingabout psychotherapy research (17, 18,20,160,181 ,242,268). Much of the reaction has beennegative, and, in response, Eysenck and hiscolleagues have tried to refute claims of hiscritics and to provide additional data (e. g.,70,71 ). The debate between Eysenck and otherresearchers is important to unravel, both be-cause of its centrality in discussions about psy-chotherapy, and because of its implications forthe most important questions about psychother-apy. In terms of Eysenck’s original report, threetypes of problems seem to be important in inter-preting his analysis: 1) the nature of the datathat Eysenck reviewed, 2) the utility of Denker’scomparison data, and 3) Eysenck’s interpreta-tion of the data.

The first problem, that of specificity of thedata, has been raised by a number of Eysenck’scritics (e. g., 160). In essence, these reviewershave claimed that it was inappropriate to gen-eralize very widely about psychotherapy on thebasis of Eysenck’s data. The most important dif-ficulty is that it is not clear what was done ineach of the studies. One cannot determine howmuch therapy was actually received by patientsand what the quality of this treatment was (19).It also seems important to note that the researchreported by Eysenck was conducted prior to1950, at a time when nonpsychodynamicallybased therapies were only beginning to be used.The generalizability of this work to the types ofnonpsychodynamic therapies carried out today,as well as to currently practiced psychoanalytictreatments, is not at all clear (see, e.g., 269).

The second problem has to do with the “con-trol” group data used by Eysenck in his anal-ysis. Eysenck’s control data for the effects oftreatment were drawn from a nonrandomly se-lected control group. It is difficult to determinehow this nontreated group might have differedfrom individuals who received therapy. Theymay have been more or less troubled and dys-functional than those who sought treatments.

Not only are the differences between the controland treatment group subjects not clear, but asnoted by Meltzoff and Kornreich (181), neitherDenker’s (61) nor Landis’ (153) data were rep-resentative (i. e., the data were not derived froma survey).

Meltzoff and Kornreich (181) also note thatthe control group data did not really representno-treatment data. Especially in Denker’s study,where patients were diagnosed by general medi-cal practitioners, a form of treatment was pro-vided. Patients were provided “sedatives, ton-ics, as well as reassurance, and a placebo type oftreatment. ” Furthermore, since none of these pa-tients was severely disturbed to begin with, thereturn-to-work measure may not indicate an ac-tual remission of the symptoms that firstbrought the individual to the practitioner.

Finally, a number of commentators havepointed to errors in Eysenck’s categorizing ofstudies and errors in the way he handled data.Bergin (19) reanalyzed the data used in Ey -senck’s original report and finds a different rateof recovery for psychoanalytic treatment. WhatBergin did, among other procedures, was toeliminate cases where individuals left therapy;Eysenck had, instead, counted these cases asfailures. Bergin also claims to have found cod-ing errors in Eysenck’s original report (e. g., out-come data incorrectly represented). Bergin’s re-analysis had yielded an improvement rate of 65percent for the eclectic therapies (similar toEysenck’s). He notes that improvement rates forall types of treatment may be deceptive, becausethere was a great deal of systematic variationacross personality types, therapists, and clinics.Bergin concludes that global questions of psy-chotherapeutic efficacy are “silly” and that onemust analyze specific therapies for specificproblems.

Eysenck acknowledges many of the methodo-logical problems with his data and has arguedthat the burden of proof to provide evidenceshould be on those who seek to promote psy-chotherapy (see 223). He has also updated hisoriginal research, and, in later reviews (e. g.,69), cites an additional 11 studies. Although hisinitial conclusion about the lack of effectivenessof psychotherapy remains, he finds supportive

evidence for at least one type of psychotherapy,a behavioral approach based on the work ofWolpe. It should also be noted that a number ofcommentators (e.g., 181) have criticized Ey -senck for selectively reviewing the literature. Bythe time of his 1965 paper (which was publishedalong with 17 critiques by prominent research-ers), there were at least 70 control group studiesof psychotherapy. Most of these were not in-cluded in Eysenck’s review, although his reasonsfor not selecting these studies are not clear.

Although Eysenck’s work is often interpretedas an indictment of all psychotherapy, as notedabove, Eysenck (69) cites evidence as to theefficacy of therapies based on behavior theory.Thus, he describes a study conducted by Laza-rus (155) of phobic patients who were treatedeither by group desensitization (a behavior ther-apy), group interpretation (a psychodynamictherapy), and a combination of nonbehavioralmethods. The results showed a 72-percent re-covery rate for the behavior therapy versus a12-percent recovery rate for the verbal therapy.Eysenck also relies on an unpublished report byWolpe (the developer of desensitization thera-py) in which the results of Wolpe’s investiga-tions (122 cases) were compared with reportedresults from two large psychoanalytic institu-tions (approximately 400 cases). Wolpe had re-ported that 90 percent of his cases were cured ormuch improved versus 60 percent of the psycho-analytically treated patients.

In trying to make sense of the controversyover Eysenck’s research, a number of aspects ofhis work should be highlighted. First, whetherone accepts his findings or not, the primary pur-pose of the original review was to assess thedata for treatment of neurotics. Severe mentaldisability was considered only secondarily inlater reports. Second, while Eysenck’s critiquehas been interpreted as a critique of all psycho-therapy (and psychotherapy research), his com-ments are most critical of psychoanalyticallyderived therapies. He is supportive, as describedabove, of therapies based on behavior theory.Finally, it should be noted that Eysenck did notfind psychotherapy to be harmful and, in fact,found that eclectic therapies were associatedwith fairly high rates of improvement. The criti-

cal question—whether improvement was causedby psychotherapy or resulted from other factors(e.g., spontaneous remission, placebo effects)—is not resolved by Eysenck’s work.

Meltzoff and Kornreich’s Analysis

According to Meltzoff and Kornreich (181),their work was stimulated by a belief that therehad been advances in the development of re-search on the effectiveness of psychotherapysince Eysenck (who initially published in theearly 1950’s), but that this work was underrep-resented in the literature. Meltzoff and Korn-reich, in their book-length discussion of the psy-chotherapy research literature, made an impor-tant departure from earlier reviews and clas-sified studies by methodological adequacy.Groups of studies were formed according totheir methodological quality and compared interms of the positiveness of findings. Meltzoffand Kornreich wanted to come up with a “fair”assessment of what was contained in the avail-able literature.

In their category of “adequate” researchdesigns, they included studies which used acontrol group condition and adequate outcomemeasures. In their “questionable” research de-sign category, they included studies with con-trol groups that may not have been comparableand/or used poor outcome measures or analysisprocedures. They also included in this lattercategory studies that used analog designs. Theresults of the studies they assessed were catego-rized as either “posit ive,” “negative,” or null.Since most studies included multiple outcomemeasures, their subdivision of outcomes was ajudgment of the balance of statistically signifi-cant findings. Their analysis included approx-imately 100 control group studies.

According to Smith, et al. (268), who tabu-lated the results of Meltzoff and Kornreich’s re-view, 80 percent of the controlled studies theyreported yielded positive results, while the other20 percent had null or negative results. Therewas also a positive relationship between re-search quality and positive findings. Thus, 8 4percent of Meltzoff and Kornreich’s adequatelydesigned studies yielded positive results about

Ch. 4–Literature on the Effect/vc)wss of Psychotherapy ● 41

the effectiveness of psychotherapy, while only33 percent of the studies of questionable designyielded positive results.

Meltzoff and Kornreich (181) also assessedthe degree to which results were “major” or “mi-nor.” The studies categorized as “adequately de-signed, with major outcomes” varied on a num-ber of dimensions. These studies included re-search such as Grace, Pinski, and Wolff’s (107)

assessment of group therapy for individualswith ulcerative colitis. In this study, patients inthe treatment group received psychotherapy toalleviate stress. Over a 2-year period, it wasfound across a variety of “hard” criteria (e.g.,operations required) that those who receivedgroup therapy were less likely to require medi-cal treatment than those who did not (there wasa lower morbidity rate for therapy patients, aswell).

A different type of study included by Meltzoffand Kornreich (181) was Morton’s (190) studyof college students who were referred to psycho-therapy for severe personal problems. A ran-domized control group design was implementedby having some subjects placed on a waitinglist. Raters, who were blind to informationabout the students’ treatment (they were thecounselors who made the actual referral), talkedwith students after 3 months, On a number ofmeasures, including those that assessed cogni-tive and behavioral variables, 93 percent of thetreatment group students and only 47 percent ofthe control group students showed improve-ment. The students who received psychotherapywere considered to have received a majorbenefit.

Although it is difficult to systematicallyanalyze the types of studies included in Meltzoffand Kornreich’s (181) review, most of the stud-ies they report appear not to be of severely dis-abled patients. In fact, almost all the studies inwhich there were adequate designs and majorpositive benefits were studies in which the prob-lems were relatively specific and the patientswere not institutionalized. The reason for thismay be connected with the problems of doing

control group research, as well as with the na-ture of psychotherapy. As noted in chapter 3, itis easier to conduct well-designed control groupresearch when problems are specific and the

potential harm of withholding treatment is notsevere. The hypothesis must be left open thatthe more general and unspecified the problem,the less efficacious is psychotherapy.

This point has been made by Malan (168)about Meltzoff and Kornreich’s work (181).Malan noted that, although he agreed with theMeltzoff and Kornreich conclusion about thebeneficial effects of psychotherapy, its general-izability for a variety of patient groups was notclear. There is also a problem, in generalizingfrom Meltzoff and Kornreich, of not knowingwhat specific techniques work with what specif-ic problems. Although their methodological fo-cus is useful for assessing hypotheses about therelation between research quality and outcome,it obscures the synthesis of findings aboutspecific treatments. From a more optimistic per-spective, it is comforting that their positive find-ings rate is a bit higher than that of Eysenck,and further, that these positive results were ob-tained in well-designed studies.

Bergin’s Reviews

Bergin’s 1971 review (19) of psychotherapeu-tic effectiveness was described briefly above inreference to his critique of Eysenck’s work.Bergin’s analysis is important (see also 18,21)because, unlike earlier critics of Eysenck, he ac-tually examined the studies used in the originalreport (72) and recomputed Eysenck’s treatmentremission rates. Bergin used different assump-tions and procedures for establishing improve-ment rates, and, as noted above, he found a dis-crepancy in Eysenck’s analysis of psychody-namic therapy. In addition, Bergin (19) calcu-lated a different remission rate for no-treatment(based on data not discussed by Eysenck), andfound that only about 30 percent of the patientswould have recovered had there not been psy-chotherapy. Within the framework of Eysenck’snonexperimental data base, there are limits toone’s certainty about any conclusions.

Bergin (19), in part to remedy this problem,reviewed 52 studies of psychotherapy outcome,He claimed that these studies represented a“cross-section” of the available literature (see276). Approximately half of the studies had con-trol groups, and Bergin categorized the studieson the basis of a number of variables, including

.

42 ● Background Paper #3 The Efficacy and Cost Effectiveness of Psychotherapy

the adequacy of the research design, duration oftherapy, type of therapy, and therapist experi-ence. Studies were judged and cross-tabulated interms of the results of the therapy: “positive,”“negative, “ or “in doubt. ”

Of the 52 studies, Bergin (19) judged 22 to bepositive (indicating psychotherapy was effec-tive), he judged 15 as having negative results,and 15 were judged “in doubt. ” Bergin con-cluded that “psychotherapy, on the average, hasmoderate positive results. ” Typical of the stud-ies Bergin classified as positive was Gottschalk,Mayerson, and Gottlieb’s (106) study of brieftherapy. Using a no-treatment control group de-sign, Gottschalk, et al. studied neurotics whowere given emergency outpatient therapy. Thetreated patients improved significantly morethan the controls on a variety of behavioral andcognitive measures. Another positive studycited by Bergin was Vorster’s (288) investigationof neurotics treated by eclectic therapies.Vorster found the treatment group significantlyimproved, although this study’s design did notemploy a randomized control group.

In a more recent extension of Bergin’s (19)review, Bergin and Lambert (21) discuss addi-tional evidence to support their hypothesis thatpsychotherapy is efficacious. They also reply tothe criticism (e.g., 222) of Bergin’s originalwork. Bergin and Lambert cite as one of the bestexamples of more recent research a study bySloane, Staple, Christol, Yorkston, and Whip-ple (266). That study used 90 outpatients, ofwhom two-thirds were neurotic and one-thirdhad personality conduct disorders. The patientswere randomly assigned (they were alsomatched with respect to sex and severity ofsymptoms) to short-term analytically orientedtherapy, behavior therapy, or a minimal treat-ment wait-list group. Treatment outcome wasassessed after 4 months. On average, all groupsimproved significantly across a set of targetsymptoms, but the two therapy groups (analyti-cally-oriented and behavior) improved signifi-cantly more than the minimal treatment wait-list group. No significant differences were foundin outcomes between the actual therapy groups.

Bergin and Lambert (21) conclude that, “Psy -choanalytic/insight therapies, humanistic or pa-

tient-centered psychotherapy, many behavioraltechniques and, to a lesser degree, cognitivetherapies rest on a reasonable empirical base. ”They contend that these therapies achieve re-sults that are superior to no-treatment and tovarious placebo treatment procedures. Berginand Lambert also assert that even if one acceptsEysenck’s “two-thirds remission within 2 years”formula, there is still positive evidence for psy-chotherapy. This is because treatment effects ofthe same magnitude are frequently obtained in 6months or less in formal psychotherapy, “. . . aconsiderable evidence of therapy’s efficiency/efficacy over no treatment. ” In effect, psycho-therapy hastens the recovery period and reducesboth suffering and the effects of disability.

Rachman’s Critique

As can be seen from the above two reviews,the early 1970’s was a fertile time for psycho-therapy assessments. Rachman (222), a frequentcollaborator of Eysenck’s, provided another re-view which served, in part, to refute Eysenck’scritics. Rachman, in particular, responded toBergin’s (19) reanalysis of Eysenck’s data and tothe dispute about what studies should be in-cluded in a review of psychotherapy efficacy.Rachman, in analyzing Bergin’s work, dis-allowed a number of studies (5 out of 14), be-cause the subjects were delinquents or had psy-chosomatic complaints, rather than being neu-rotic. If Rachman’s analysis had been extendedto Meltzoff and Kornreich’s (181) review, thesame problem would have existed.

Rachman (222) analyzed 23 studies which heconsidered to be relevant for assessing the ef-ficacy of “verbal” psychotherapy (i.e., therapiesbased on conversation between therapist andpatient—primarily, humanistic and psychody-namically based). Of these studies, Rachmanfound that only one provided tentative evidenceof the effectiveness of psychotherapy. Five stud-ies produced negative effects, where treatmentgroup results failed to exceed control groups orbase-line remission rates. It is important to notethat Rachman excluded 17 studies for a varietyof reasons, only 2 of which showed negative ef-fects. The reasons for excluding the 15 studiesthat showed positive results varied, and in-

Ch. 4–Literature on the Effectiveness of Psychotherapy ● 43

eluded the use of unacceptable outcome meas-ures (e. g., a projective test), the exclusion ofsubjects who left therapy before treatment wascompleted (which Rachman said inflated im-provement rates), and inconsistency across out-come measures (i. e., some measures showedpositive effects, while one or more measuresshowed negative effects). The two negativestudies were excluded because randomized con-trol group conditions were not employed.

A number of commentators (e.g., 257,268)

have reviewed Rachman’s (222) critique andanalysis, and they claim that he used inconsist-ent standards for evaluating evidence. In partic-ular, they assert that Rachman used criteria forassessing the verbal psychotherapies that weredifferent from the criteria he used for the behav-ioral therapy experiments. Smith, et al. (268)

have criticized Rachman because he selectivelychose studies to review. They indicate thatmany more studies were available to Rachmanthan he included, and add that it cannot be in-ferred from his discussion why particular stud-ies were not referenced. Smith, et al. also criti-cize Rachman for “ex post facto” exclusion ofstudies based on methodological criteria. Theyargue that he should have compared the goodand poor designs to determine whether or notthey yielded different kinds of conclusions, asMeltzoff and Kornreich (181) did in theiranalysis.

Luborsky, Singer, andLuborsky’s Review

In a more current assessment, Luborsky,Singer, and Luborsky (162) tried to evaluate allreasonably controlled studies of psychotherapyon “real” patients (i. e., analog studies were ex-cluded). Luborsky, et al. examined availablestudies that had assessed therapy for treatingrecognized problems of individuals who soughtpsychological treatment. Many of these studieswere ones that had been used by Bergin (19) andby Meltzoff and Kornreich (181). Although Lu-borsky, et al. ’s definition excluded some behav-ioral research (primarily because it was not con-ducted in actual clinical settings) and some pa-tient populations, their scope was wider thanthat of other reviewers.

Each of the studies Luborsky, et al. (162) as-sessed was categorized on a number of dimen-sions and then summarized in a “box-score”analysis (see ch. 3). One central dimension wasresearch quality, which was determined on thebasis of 12 criteria. The criteria included thestudy’s method for assigning subjects to com-parison groups, the procedures for dealing withpremature therapy termination, experience oftherapists, tailoring of outcome measures totherapeutic goals, and the adequacy of the sam-ple size. Luborsky, et al. summarized the cod-ings by “grading” each study on a 5-point scaleof research quality. They then categorized re-sults in terms of whether there were significantdifferences showing better effects for the treat-ment group ( + ), the comparison group ( – ), orno significance between the groups (0).

Luborsky, et al. (162) found 33 studies inwhich psychotherapy treatment groups werecompared with no-treatment control groups. Ofthese, 20 studies yielded psychotherapy treat-ment groups which were significantly better offthan control groups, and 13 showed no differ-ence. Luborsky, et al. did not find any instanceof a control group being better than a psycho-therapy treatment group. They found 19 studiesin which schizophrenic populations (i. e., severe-ly disabled individuals) were studied. Of these,11 yielded results in favor of the psychotherapycondition, and 8 yielded no differences. Lubor-sky, et al. also found that in a majority of com-parisons (13 out of 19), there were no significantdifferences in outcomes to patients between be-havior therapy and other psychotherapies.While the reviewers note the positive researchfindings on behavior therapies, they suggest theneed for more studies in which behavior ther-apies are applied to patients who have general-ized maladjustments.

Luborsky, et al. (162) conclude that controlstudies find that patients who go through psy-chotherapy do, in fact, gain. Because theyused “box scores” where effect sizes were notestimated (see 221), it is not possible to deter-mine how strong these effects will be. However,according to Smith, et al. (268), there has beenno published substantive criticism of Luborsky,et al. and (as is described below) Luborsky, et

—- ——-—— —-—--

44 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

al. ’s general finding has been substantiatedother reviews.

NIMH Synthesis

Perhaps the most comprehensive review

by

ofpsychotherapy outcome research has been con-ducted at the National Institute of MentalHealth by Parloff and his colleagues (208). Thisreview provided an assessment of psychosocialtreatments for mental disorders and was pre-pared for the Institute of Medicine (NationalAcademy of Sciences) as part of that Institute’swork for the President’s Commission on MentalHealth (219). The Parloff, et al. review differsfrom earlier works in that it uses a more nar-rowly defined treatment and is organized pri-marily according to disabling conditions. Thus,for each of a variety of mental disorders, theavailable evidence as to the effectiveness of psy-chotherapy was assembled and analyzed. Thereviewers tried to make some general statementsabout psychosocial therapies (psychotherapiesthat do not use drug treatments) and how thesetherapies are affected by other variables, such astherapist and patient characteristics. Parlofff etal. ’s work is very extensive, so a detailed sum-mary is not attempted here; only the work’s gen-eral conclusions are described below.

Parloff, et al. ’s (208) general finding (see also207) was that “patients treated by psychosocialtherapies show significantly more improvementin thought, mood, personality, and behaviorthan do comparable samples of untreated pa-tients. ” These reviewers found that spontaneousremission rates developed from separate sam-ples provide evidence that psychosocial treat-ments seem to result in greater improvementthan would be expected without psychothera-peutic treatment. Their finding is supportedmost clearly for disorders such as “anxietystates, fears, and phobias. ” Parloff, et al. ’srelatively positive assessment, however, was ac-companied by a number of caveats. For manydisorders, psychotherapy alone (i.e., withoutother treatments such as drugs) has not beendemonstrated to be effective; nor does it appearto be effective for particular populations (e.g.,children). Finally, the review notes, although ef-fectiveness evidence exists for a number of dis-

orders, only some types of therapies (with par-ticular therapists) may be effective.

The central aspect of Parloff, et al.’s (208)review was a summary, by each psychopatho-logical condition, of the available treatmentresearch evidence. To appreciate the complexityof this task, consider their discussion of severemental disorders such as schizophrenia. Forthese disorders, Parloff, et al. found that indi-vidual and group psychotherapy provide anambiguous amount of improvement for institu-tionalized patients; however, in conjunctionwith drug therapies and other psychologicaltreatments, they appear to have important ef-fects. The authors note that “. . . drugs do notteach individual social and interpersonal skills.”For such hospitalized populations, however,Parloff, et al. found considerable evidence thata specific type of therapy (behavior based) im-proves social adjustment (on a variety of socialand interpersonal variables). They also foundthat the return of severely disturbed patients totheir community had positive effects on treat-ment outcome, although this finding was lim-ited to patients with certain interaction skills,and under the condition that the patient returnsto a “good” family situation.

One important feature of the Parloff, et al.work, both as it was presented in the originalform (208) and as it was summarized by thePresident’s Commission on Mental Health(219), is that this report examines the evidencefor alternative hypotheses about the effects ofpsychotherapy. In particular, Parloff, et al.found a variety of reported spontaneous remis-sion rates; that is, different improvement ratesare obtained for disturbed patients who do notreceive therapy (see, e.g., 280). Despite the factthat patients improve to some extent withouttherapy, however, the hypothesis that suchspontaneous remission effects account forchanges in treated patients cannot be validated.Parloff, et al. (208) report that studies whichhave been controlled for placebo effects findthat changes associated with treatment aregreater than those associated with the placebo.Unfortunately, there is an inherent problem intheoretically identifying the nature of a psycho-therapy placebo (see ch. 3) and separating it

Ch. 4–Literature on the Effectiveness of Psychotherapy ● 45

from the effects of the treatment, so these resultsmust be regarded as tentative.

Parloff (207), in a discussion of the review(208), calls for the development of clinical trialresearch to assess more widely and systematical-ly the effects of psychotherapy. While Parloffadopts a relatively positive view of the effects ofpsychotherapy, he finds the current research lit-erature limited. He proposes the use of exper-imental designs to assess the specific conditions(including aspects of the treatment, therapist,and patient) under which therapy proceduresyield particular outcomes. Although one viewof such clinical trials would be to test differenttheories of psychotherapy, an implication ofParloff, et al. ’s work is that other factors need tobe incorporated in the design of these trials.

One additional note about Parloff, et al.’s(208) findings has to do with their review of be-havior-based therapies. Parloff, et al., alongwith Eysenck, Rachman, Bergin, and to someextent, Meltzoff and Kornreich, report clear-cutevidence that behavior-based therapies are ef-fective treatments for specified conditions.Although its generalizability has not beenestablished across the range of disorders, thisfinding suggests one important focus for futureresearch.

Smith, Glass, and Miller’sMeta-Analysis

The final review considered here, that bySmith, Glass, and Miller (268), has the potentialto be as controversial as Eysenck’s originalwork, and may stimulate an entire new set ofpsychotherapy outcome analyses. The meta-analytic methods used by Smith, et al. (see ch.3) are at the heart of the controversy about thiswork. A preliminary published report of themeta-analysis of 375 control group studies ofpsychotherapy (267) has already stimulated avariety of critiques (e.g., 73,90,218). Smith, etal. ’s goal was to determine the state of knowl-edge about the effects of psychotherapy, usingsystematic scientific procedures. Their meta-analytic procedure was used to integrate thefindings of a disparate set of studies on psycho-therapy. In addition, each of the studies in thesample was classified to enable determination ofthe factors that influence outcome findings.

In order to conduct their review, Smith, et al.(268) tried to include all controlled studies of theeffectiveness of any form of psychotherapy.Controlled studies were defined as investiga-tions where a treatment group received psycho-therapy, and another group which was compa-rable did not receive treatment. In some cases,receiving treatment meant a placebo or alternateform of psychotherapy. Smith, et al. considereda study to be relevant if the study investigatedtherapy that involved: 1) patients who wereidentified as having an emotional or behavioralproblem, 2) treatment that was psychological orbehavioral, and 3) therapists who were identi-fied professionals. The definition resulted in theanalysis’ including a variety of investigations oftreatments applied to problems of different de-grees of severity. Smith, et al. excluded sometypes of treatment, including those in whichpsychoactive drugs were used (these studieswere analyzed separately), those which wereprimarily educational, and those which werenot, essentially, psychosocial treatments. Theysurveyed a great number of sources to identifystudies, including published journals, disserta-tions, and clearinghouses that identify profes-sional publications. Their final sample included475 controlled studies of psychotherapy.

Probably the most important aspect of Smith,et al.’s (268) analysis was the way they classifiedthe research studies. Each study was coded on anumber of dimensions related to the character-istics of the researcher, therapist, and the pa-tient (including diagnosis); most importantly,the studies were classified on a series of meth-odological criteria. The methodological cate-gories included the nature of the assignment toconditions (e. g., random v. matching) and suchfactors as experimental mortality and internalvalidity (see 41,165). Smith, et al. ’s principaldependent measure was a standardized score forthe size of the effect. From the data reported ineach study, Smith, et al. calculated scores forthe size of the effect; to allow comparison acrossstudies, they computed each as a standardscore. Studies often included more than one out-come measure, so Smith, et al. ’s analysis treatedeach variable as a separate case. Thus, from the475 studies Smith, et al. analyzed, they found1,766 effect size measures. In addition, they

46 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

coded outcome measures in terms of the type ofmeasure, instrument, and its reactivity (i. e.,susceptibility to social desirability, faking, etc.).

Smith, et al.’s (268) principal finding wasthat, on the average, the difference betweenaverage scores in groups receiving psychothera-py and untreated control groups was 0.85 stand-ard deviation units (i.e., the effect size differencewas 0.85). According to Smith, et al., this aver-age effect size can be translated to indicate thatthe average person who receives therapy is bet-ter off than 80 percent of the persons who donot. They found little evidence for the existenceof harmful effects of psychotherapy (i. e., veryfew cases where the mean of the control groupwas higher than the treatment group). Smith, etal. found some significant differences across thetypes of therapies whose effects were studied(the range was 0.14 to 2.38), but these effects areconfounded by variables such as patient andtherapist characteristics which were distributedunequally among the therapies. Finally, theirmethodological categories proved not to corre-late with effect sizes; thus, for example, the bet-ter designed studies did not yield less positivefindings.

OTA has had an opportunity to reviewSmith, et al.’s (268) coded data of the 475 stud-ies. A sample of studies was drawn, and theirreliability was checked by comparing our blindcodings with their original data. In particular,effect size ratings for a dozen studies wererecomputed. This analysis indicated that theSmith, et al. codings were both easily replicableand apparently reliable. Their validity is moredifficult to establish.

A key question is whether the effect sizescores calculated by Smith, et al. (268) are avalid measure. There are reasons to suggest thattheir average effect size is inflated, but there arealso reasons to suggest that it is conservative.Inflation in the effect size measure may havecome about because only well-designed studieswere included in Smith, et al. ’s analysis. Thus,to the extent that the control group studies arecompleted only for successful psychotherapiesor for a limited range of psychotherapy treat-ments, the sample may be biased positivelywhen compared to psychotherapy research. It is

difficult to determine the exact nature of the biasthat may result from this problem, There arealso some aspects of Smith, et al. ’s procedureswhich suggest that their average effect sizemeasures may be conservative. Smith, et al.considered placebo treatments, as well as actualforms of therapy and counseling, in their treat-ment group means. These are all not legitimatetherapies, and, in fact, when separately ana-lyzed, showed lower effectiveness as comparedto actual psychotherapy.

Most critics of Smith, et al. (268), in par-ticular Eysenck (73), have disputed the meta-analysis approach (actually, the published cri-tiques refer to Smith and Glass (267)), because itlumps together too many things and includesstudies of poor design, as well as good design.This criticism is potentially justifiable, but aclose review of Smith, et al. indicates that theycontrol for this problem by their classificationof studies according to methodological criteria.Not only do they start with a group of relativelywell-designed studies (in terms of their defini-tion of controlled psychotherapy research), butthey provide analyses of the relationship be-tween effect size and classification variablessuch as internal validity (a measure of the quali-ty of the research design—see 41). The correla-tions are close to zero, indicating that studiesthat use randomized control group designs findthe same effect as studies that use poorerdesigns. There may still be a sampling problemin that the published literature only reportswell-controlled studies, but this problem is dif-ferent from that on which most of the criticismof Smith, et al. ’s meta-analytic procedures isbased.

At this point, it is difficult to know how toutilize the results of Smith, et al. (268). Theirwork is certainly more systematic than that ofother researchers; however, their methods arenot yet widely accepted and their work has notbeen available long enough for comprehensivereviews to appear. Perhaps, the most importantlimitation of such meta-analytic research is thatit relies on the existing literature. It seems clearthat much better research on psychotherapy canbe done, and a more definitive meta-analysismay have to await the completion of this newresearch.

Ch. 4—Literature on the Effectiveness of Psychotherapy ● 47

DISCUSSION

If one considers only the trend of findingsreported by scholarly reviews and analyses ofthe psychotherapy outcome literature, it wouldappear that psychotherapy treatments, undersome conditions, have been shown to be effica-cious (see, e.g., 287). Although the evidence isnot entirely convincing, the currently availableliterature contains a number of good-qualityresearch studies which find positive outcomesfor psychotherapy. There are also a largenumber of studies which report positive effects,but whose methods or generalizability are dif-ficult to assess. The quality of the evidencevaries in terms of the nature of the treatmentand the patient’s problem. One difficulty withthe available literature is that, while a host offactors have been identified as important to psy-chotherapeutic outcomes, the role of these fac-tors (e.g., characteristics of the patient, thera-pist, setting) has not been assessed in any defin-itive way.

Frank (85), for example, in commenting onthe state of psychotherapy outcome research,reports a disappointment that research has notproduced more specific understandings of whatoccurs during therapy. While he supports theconclusion that research demonstrates that psy-chotherapy is somewhat better than no therapy,he also finds that much of the literature in-dicates equivocal outcomes. One reason forthese equivocal findings, according to Frank, isthat the personal qualities of the therapist andpatient and their interaction may be more im-portant than the therapeutic method. Becausethese factors vary so widely and are not typical-ly controlled for, variability is built into

psychotherapy studies, making it more difficultto detect significant treatment effects. Frankurges more research that clearly assesses thesecomponents of therapeutic treatments.

It is likely, given the theoretical potential toconduct such research on the ingredients ofpsychotherapy, that a more extensive researchliterature could be developed (see also 207). Itprobably means that psychotherapy evalua-tions, unlike evaluations of new drugs, need topay equal attention to the conditions of treat-ment and the treatment per se. Research done onpsychotherapy in settings different from actualpractice will probably have limited utility andwill represent only one stage of a testing pro-gram. Although it would be hoped that psycho-therapy research could lead to better under-standings of the role of therapist characteristicsand skills, it might only be possible to assesshow these factors interact with treatments asthey are actually delivered. Finally, it shouldalso be noted that some forms of potential out-come research, such as program evaluations, donot seem to have made a substantial impact onthe literature (cf. 181,212). While program eval-uations have been done of a variety of processfactors in mental health delivery systems, thesestudies have less frequently measured outcomesusing experimental designs. In part, this reflectsthe developing nature of the methods for pro-gram evaluation (see ch. 3) and the slow shift tothe funding of such research. It is possible thatthis situation will change with increased pres-sures for accountability and with increased em-phasis on understanding complex sets of psy-chotherapy technologies.

5mMethods for AssessingCost Effectiveness and

Cost Benefit of Psychotherapy

5.Methods for Assessing

the Cost Effectiveness andCost Benefit of Psychotherapy

The preceding chapters illustrate some of thecomplexity of assessing the outcomes of psycho-therapy. Although such assessment is undoubt-edly difficult, it nevertheless seems possible toevaluate psychotherapy using scientific methodsof analysis. Assessing the costs and benefits ofpsychotherapy, and developing comparisonsamong effects, costs, and benefits—that is, con-ducting cost-effectiveness and cost-benefit anal-yses (CEA/CBAs)—is a natural next step in thisresearch process. To conduct such CEA/CBAsencourages the explicit analysis of the resourcesused in psychotherapy and the effects (positiveand negative) of different resource allocationdecisions.

Potentially, CEA/CBA is a set of proceduresthat can aid in decisionmaking about the use ofpsychotherapy. Increasingly severe economicconstraints, as well as the call for an expansionof mental health services (see 219), make it espe-cially important to understand how the effectsof psychotherapy are related to the resources itconsumes. There are a great number of compet-ing pressures for health care resources, and,ideally, CEA/CBA applied to psychotherapycan serve as an aid to resolving these conflicts.Although the application of CEA/CBA is, per-haps, not as well developed in psychotherapy asin other areas (see 203), the current policy con-troversy about mental health treatments hascreated increased interest in its use. The presentchapter describes the methods underlying theconduct of CEA/CBA in psychotherapy and in-dicates both the potential for its use and theproblems associated with its application to men-tal health treatments.

It is important to note that many of the issuesof CEA/CBA of psychotherapy are closely re-

lated to the problems of assessing efficacy. Eco-nomic analyses of psychotherapy are dependenton the quality of research data pertaining topsychotherapy’s effects. The unique problems ofCEA/CBA of psychotherapy have to do withthe difficulty of comprehensively assessing andvaluing the effects of psychotherapy (see 98,157,206). Such effects include the reduction ofpain and suffering and enhancement of “well-being. ” These effects are very difficult to meas-ure and even more difficult to value in monetaryterms. This difficulty may result in CBAs ofpsychotherapy consistently undervaluing thebenefits of psychotherapy. CEA, in contrast toCBA, does not require that such effects be ex-pressed in monetary units, but does rest on thepremise that they can be valued in some man-ner. Such difficulties restrict the usefulness ofcost analyses.

The methodological issues involved in the de-velopment of CEA/CBAs of psychotherapy aredescribed in the following sections. For a morecomplete description of CEA/CBA, the readershould consult OTA’s main report on CEA(203). ] The present analysis begins with a dis-cussion of the methods for assessing costs, andthat is followed by a discussion of the methodsfor assessing benefits. In the third section, theactual conduct of cost analyses is described. Thediscussion below emphasizes the relationship ofCEA/CBA to efficacy assessments and the use-fulness of CEA/CBA in aiding policymakingabout psychotherapy.

‘In addition to the main report ot OTA’S assessment of CEA,Backgroutlci Pup(Jr #1. Met}loJologicul Issues a~ld Literature Re-~Il[Ju) describes, in detail, the use of CEA , CBA methods.

51

52 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

COST ASSESSMENT

For the purpose of CEA/CBA, the cost of psy-chotherapy may be conceptualized as the valueof various resources consumed in the process oftherapy (see 6,157,173,237). These costs includethe value of a variety of resources used to pro-vide treatments, such as the value of the thera-pist’s time and the value of the use of the treat-ment facility. They may also include the valueof the patient’s time. Inevitably, decisions arerequired about whether to include or excludecosts and how to value resources appropriately.These decisions often reflect the subjective judg-ments of different interest groups involved inthe cost analysis (see 203,277) and the purposesfor which the analysis is being conducted. Justas different theoretical perspectives on psycho-therapy may result in different measures of ther-apeutic effectiveness, so too may different per-spectives yield alternative ways of defining ther-apy costs.

General Considerations

There are a number of general considerationsrelevant to assessing the costs of resources usedin psychotherapy. Some of these general issueshave to do with the data that are used in devel-oping cost estimates. Others concern the use ofprocedures to transform available data into use-ful indices of resources consumed. In a later sec-tion, the application of these procedures to thecollection of specific cost data is described.

Accounting Methods.—The most readily ac-cessible source of data to assess the costs ofpsychotherapy are the entries in the accountingrecords of a treatment facility (e.g., a practi-tioner’s office, a mental health center). Becausethey are usually highly organized and accessi-ble, accounting records give a ready definitionand a reasonably reliable record of the moneysrequired to deliver therapy. These direct costsare the ones that are most often referred to insimple cost analyses of psychotherapy. A caveatthat should be noted, however, is that the coststhat result from accounting tabulations may notinclude all or even most of the resources used bytherapy. Costs from such tabulations, therefore,

may not be an accurate reflection of the re-sources consumed.

In addition, a number of costs may not ap-pear in accounting records. Thus, for example,volunteered time and donated facilities mayplay a large, but unaccounted for, role in theprovision of psychotherapy. Therapies that at-tract more volunteers owing to location or typeof patient treated may appear less costly thantherapies located in areas where volunteers arescarcer (e.g., in impoverished neighborhoods)or than therapies treating less “attractive” per-sons (e. g., sex offenders compared to childrenwho are autistic). Resources contributed to ther-apy by the family or others connected with thepatient may also be considerable, but would notbe recorded in an accounting system.

Opportunity Cost. —A more correct ap-proach to assessing the costs of psychotherapythan using accounting costs involves the op-portunity costs concept. In using opportunitycosts, one calculates the value of a resource as ifit were applied to a best alternative use. Forpsychotherapy, determining opportunity costsallows one to consider more completely thevalue to society of various resources consumedby psychotherapy treatments and significantlyalters the analysis (25,40,283,305). Considera-tion of opportunity costs avoids problems cre-ated by different accounting procedures and, atleast conceptually, the problems due to the useof volunteers and donated facilities.

Discounting. —The actual calculation of op-portunity costs involves the use of discountingprocedures, which provide a present value of fu-ture costs. Thus, if costs will be incurred at afuture time, they will appear to be less costly ifvalued in the present. The discount rate is usual-ly based on the prevailing interest rate. Prob-ably, discounting procedures are more impor-tant for properly valuing benefits (which aremore likely to occur over time) them for valu-ing costs. Obviously, though, discounting pro-cedures are necessary when dealing with costssuch as those for a treatment facility.

Methods for Cost Assessment

The valuation procedures used to assess themost common cost elements in psychotherapyare considered in this section. The use of op-portunity cost procedures and the completeassessment of relevant cost elements are empha-sized. Also discussed is the assignment of coststo specific treatment.

Personnel Costs.—The cost of personnel isusually the largest therapy cost (between 60 and80 percent, according to Levin (157)), Personnelcosts can usually be estimated simply by multi-plying the sum of salaries and benefits of thepersonnel employed in the therapy process bythe time involved. Personnel may include pro-fessional therapists and paraprofessionals, aswell as support staff. In cases where salary dataare difficult to analyze or where time is volun-teered (e.g., a therapist donates time in a teach-ing facility), personnel costs can be estimated byother means. Thus, for example, hours of ther-apy can be multiplied by standard hourly salaryfigures.

Several “time accounting systems” have beenspecifically developed to collect personnel costdata in mental health treatment facilities (30,150,205). Some of these data are available in ac-counting records, but oftentimes they are notavailable in sufficient specificity (i. e., brokendown by tasks). In the time accounting systemdescribed by Carter and Newman (30), all per-sonnel, including volunteers, patients, andsalaried staff, were required to keep daily rec-ords of time spent in therapy-related activities.This information formed the basis of a cost-accounting system.

Facilities and Equipment Costs.—Also in-cluded in the valuation process are the costs ofoperating facilities, most correctly expressed interms of market rent, plus overhead such ascooling and building maintenance costs. Marketrent is used in order to value the resource interms of its “opportunity cost, ” discussedabove. This procedure corrects for the problemof valuing donated facilities and for valuinggovernment facilities which may be leased (tothe treatment agency) at artificially low rents.The procedure would include the estimated val-

ue of donated or loaned rooms and buildings. Itwould also adjust the rent or mortgage to assesstheir true, rather than paid-for, value.

Costs for the equipment and materials used inpsychotherapy also have to be calculated. Thesecosts, which are usually available in accountingrecords, include such resources as office sup-plies, food, laundry, and telephones. They mayalso include the value of such specialized mate-rials as psychological tests and computer scor-ing services. Usually, the difficulty with assess-ing the cost of these specialized resources is thelack of records as to their use.

Other Costs. —In addition to personnel andfacilities/equipment costs, some analysts in-clude the costs of the patient’s time and/or thecosts of therapy to a patient’s family. Althoughsuch costs can, alternatively, be considered as anegative benefit (i. e., subtracted from benefits),it is sometimes useful for them to be includedwith the actual costs of providing therapy.Thus, for example, when a patient loses timefrom work activities or when an employer hasto give release time to a patient for therapy, itmay be useful to value the patient’s time andconsider it as a cost.

Assessment of the costs of the patient’s time issimilar to the way other personnel costs are cal-culated. Usually, an accounting can be be madeof the amount of time that the patient spends intherapy. Assuming that this time could be usedproductively, it is multiplied by the patient’ssalary. Parallel calculations can be made forfamily members who become involved in thetherapy or are required to spend time with a pa-tient as a result of therapy. One problem withthis aspect of the costing process is the problemof equity, since some groups of people earnmore than others and, thus, their time could bevalued more highly.

While a number of other costs could be in-cluded, such as the psychological cost of ther-apy, there is no agreement as to whether theyshould be included or how they can be valued(see, e.g., 126,172). The development of psy-chological cost measures would involve theassessment of the suffering or pain of a patientas a result of therapy. However, such costs, as

well as the costs of mental illness are usuallyconsidered as a reduction in the benefit of atreatment.

Specifying Costs.—For a simple CEA orCBA, the sum of personnel, facilities, equip-ment, and materials costs may be adequate. Foran analysis which seeks to identify the proce-dures, therapists, patient types, or settings thatconsume more costs than others, however, pro-cedures have to be devised for assigning costs toindividual components of therapy. Analyses ofthis type may become even more complex whendistinctions have to be made among varioustypes of costs (157,248).

Cost data that can be used for analyses of theresources used for specific components of ther-apy can be collected either during therapy orafter it is completed. If collected after therapy,the analysis requires summary data to bebroken down into different costs correspondingto therapy components. Because many assess-ments of psychotherapy have been introducedonly after a therapy program has begun, thebreakdown method has received considerableattention. Analyses that examine the cost effec-tiveness of treating individual patients maydivide overhead costs such as salary, rent, andbasic supplies equally among patients, accord-ing to the amount of their therapy (e. g., numberof therapy hours).

Some cost assessment procedures assign ther-apist costs to patients according to the amountof time that therapists spend working with dif-ferent patients; then, the costs of overhead, per-sonnel, and other resources are divided equallyacross patients (e. g., 30,149,199,305). To com-pare the relative cost, cost effectiveness, or cost

benefit of different components of therapy pro-grams, the difference in direct costs of the var-ious treatment components is calculated. Insuch analyses, the overhead costs that are thesame for each component are ignored. Thesedirect costs are then used in CEA/CBAs, as de-scribed below.

Discussion of Cost Assessment

Despite some longstanding interest in assess-ing the cost of psychotherapy (e.g., 76) and theapparent ease of applying standard valuationmethods to psychotherapy, the assessment ofpsychotherapy’s cost is not widespread, norhas such assessment been evaluated. Certainly,techniques for the measurement of effectivenessare much better developed than those for meas-uring costs. Most psychotherapy research—per-haps 95 percent —neglects the cost of the treat-ment. The implications of this neglect for CEA/CBAs of psychotherapy are important to con-sider (16). It should be noted for example, thatno standards as to what should be included incost analyses have been developed. More im-portantly, the available studies may reflect anarrow range of treatments (probably thosetreatments which are either very costly or verylow in cost). In addition, cost data may bederived from existing studies even if not explic-itly included in the original analysis. Usually,the available information on numbers of thera-pists, patients, and treatment length is detailedenough for rough cost estimates to be made.Combined with secondary analysis procedures,the availability of these data may provide apromising opportunity for further R&D of psy-chotherapy CEA/CBAs.

BENEFIT ASSESSMENT

The valuation of “benefits” resulting from to do with selecting effects to be valued andpsychotherapy which can be used in CBA, un- determining appropriate ways of translating ef-fortunately, is even more problematic than cost fects into benefits (i.e., valuing effects in mone-assessment (e.g., 140,237). The translation of ef- tary terms). Below, these problems are dis-fects into benefits is necessary to provide a com- cussed in terms of the types of benefits producedmon metric for comparing resources used with by psychotherapy—to patients, to those asso-effects. The problems of benefit assessment have ciated with patients, and to society.

Benefits to Patients.—The most obvious ben-efits of psychotherapy accrue directly to the pa-tient, although different effects of therapy havesomewhat different problems attached to theirvaluation. Thus, while a change in earningsmay be valued in a relatively straightforwardway, other benefits of psychotherapy, such as areduction in pain or anxiety, are more difficultto measure and value. Because these intangiblesare so difficult to quantify in monetary units,any CBA of psychotherapy may undervalue thebenefits. It is also possible that some of thenegative benefits of psychotherap y (perhaps,the effects of stigmatization of being a patient)will not be calculated. To avoid this problem,many studies separately analyze tangible andintangible benefits, comparing only tangiblebenefits to costs.

From the perspective of some economists(e.g., 251), it is argued that a patient’s will-ingness to pay for psychotherapy reflects itsvalue to the patient. Thus, that which patientsare willing to pay for therapy is the net value ofthe expected health, social, and economicbenefits, minus the psychological suffering, losttime, and personal costs that are incurred as aresult of undergoing psychotherapy. Underlyingthe use of the “willingness-to-pay” concept is theassumption that the patient has made an in-formed decision to pay the required fee. How-ever, the willingness-to-pay assumption, whiletheoretically possible, assumes that the mentallydistraught person has made a rational decision.Since such individuals seldom have access toclear information about psychotherapy’s costsand benefits, this assumption is probably un-likely and has not really been applied.

Some economists, for example, Weisbrod(296), recognizing the inherent problem with ap-plying the willingness-to-pay criterion to psy-chotherapy, have proposed more direct assess-ment of patient benefits. In Weisbrod’s research,the variety of effects of treatment are identifiedand, to the extent possible, transformed intomonetary values. For some effects, such as im-proved quality of life or absence of mental ill-ness, no pecuniary value can be assigned. Thesefactors are therefore not included in an assess-

ment of benefits, but can be contrasted with thecosts of particular therapies.

Aside from such problems of valuing certaineffects, there is also a problem of identifyingwhich effects should be attributed to psycho-therapy. This is accomplished by comparingtreatment and control group data, To the extentvalid control group data are not available, re-sulting benefit estimates may be in error. Whilethe availability of appropriate effectiveness datais a problem for the assessment of any type ofbenefit, it is a particular problem for assessingthe effects on a patient. For most effects, therewill be a variety of other possible causes whichwill be difficult to separate without controlgroup data.

Benefits to Those Associated With Patients.—The effects of psychotherapy may extend be-yond the patient. Thus, the family and friendsof a patient may have their own quality of lifeimproved if therapy is successful. In a more tan-gible way, they may also achieve more produc-tivity in their own work and have more timeavailable for their own needs. The opportunityvalue of their improved productivity and timecan be calculated and, where appropriate, con-sidered as a benefit of psychotherapy.

As a result of therapy, patients may also bemore productive workers, and the benefits ofthis productivity may accrue to their employer(over and above the wages paid to the employ-ee). Absenteeism may be reduced, accidents befewer, and a host of other benefits are poten-tially the result of psychotherapy (e.g., 138).These benefits, of course, must be reduced bycosts for an employer to provide psychotherapyor to allow employees release time to undergopsychotherapy. As noted earlier, however,these costs can either be considered a direct costor subtracted from the benefits. It is importantto ensure that an analysis uses consistent proce-dures. It is also important that the same benefitsare not counted twice; thus, for example, theanalyst must be careful not to count the samewages as a benefit to the patient and to theemployer.

Benefits to Society .—Some of the most tangi-ble benefits of psychotherapy may accrue to so-

56 ● Background Paper #3, The Efficacy and Cost Effectiveness of Psychotherapy

ciety. Thus, the maintenance of employment ora reduction in criminal activities may yield asavings directly to society. These effects, re-flected in such savings as reduced unemploy-ment payments, may be over and above thebenefits to patients and employers. Althoughsuch outcomes may be relatively easy to value,a problem is that the benefits, if they exist,probably accumulate over a long period of time.In most cases, these benefits would have to bevery large to offset the impact of discountingtheir value over the time they are received.

It should be noted that most often the benefitsdescribed above will be in the form of expectedcost savings. Therapy-related benefits, such asreductions in work absenteeism, physicianvisits, drug abuse, and arrests, have each beenconsidered in CEA/CBA studies (e.g., 51,228,244,245). In these studies, the cost of each unitof service is estimated from average rates orfrom accounting records, and the reduction inuse of units of social services is multiplied by the

METHODS FOR CEA/CBA

The purpose of carefully measuring the costs,as well as the benefits, of psychotherapy is to beable to conduct CEAs and CBAs. CEA differsfrom CBA and it is essential to recognize the dif-ference (see 203). Cost-effectiveness studies re-quire only that the costs of psychotherapy bevalued in dollars. In CEA, the effects are notvalued in monetary terms and can be expressedin any units. In contrast, CBA requires thatboth the costs and outcomes be valued in mone-tary terms. (Theoretically, costs and benefitsneed only be expessed in the same unit for CBA,but this unit is nearly always monetary. ) Itshould also be noted that informal cost analysescan be conducted where a researcher selectivelypays attention to some aspects of resource useor benefits.

Although CEA usually requires a simpler setof procedures and yields less comparative in-formation than CBA, CEA is often considered amore appropriate tool; in other cases, when cer-tain comparative information is necessary andwhen valuation problems can be overcome,

unit cost to estimate monetary cost-savings ben-efits. Usually, the validity of control group datato estimate these savings is critical in order toseparate the effects of psychotherapy on thesevariables from other causes.

Discussion. —In several ways, developingbenefit measures is more difficult than the com-parable procedures for cost assessment. Becauseof the problems associated with using willing-ness to pay as the basis for valuing benefits,methods have to be developed to transform ef-fects into benefits. If errors are made, they willprobably result in some analyses’ not takingbenefits (especially the psychological ones) intoaccount and, thus, understating the benefits oftherapy. In addition, it is probably easy to errby not including some benefits. Because benefitspotentially accrue to a large number of peopleand societal agencies, often more so than thenumber of people or groups who incur costs,problems in analyses may result.

CBA is better suited to the problem (see, e.g.,98,157,206,305,307). In still other cases, whencomprehensive data are not available, actualCEA/CBA is not done, but the costs and bene-fits are compared informally. Most commonly,CEA and CBA have been used to compare the“worth” of different psychotherapies as pro-vided in different settings. Also frequently em-ployed are formal and informal CEA/CBAstudies designed to yield information to thosedirectly responsible for the therapy. Such anal-yses often try to assign costs and outcomes tospecific treatment processes, so that the optimalmixture of processes can be obtained (see 305).Below, some procedures associated with theconduct of CEA and CBA studies are described.

Cost= Effectiveness Analysis

When benefits and costs cannot be valued inthe same units, or when the outcomes of a treat-ment seem more valid when expressed in theirnatural units (e.g., reduction of a anxiety), thenCEA is probably more appropriate than CBA

(e.g., 157,237). In addition, when treatmentsare compared that have similar goals, CEA mayprovide an adequate and appropriate methodol-ogy. In conducting CEA, outcome data are di-vided by costs to form cost-effectiveness ratios.The procedure allows, for example, the compar-ison of several therapies to determine whichtherapy produces the greatest amount of changefor the least cost. A variety of examples of thisprocedure are given in chapter 6.

Tabulation and Matrix Methods.—One of thesimplest ways to analyze effectiveness and costrelationships is to display the data in an array.A simple tabulation model described by Krum-boltz (150) provides a rudimentary example ofhow the basic direct cost, processes, and effec-tiveness can be arrayed in a useful way. A tableis developed with the sideheadings “ProblemIdentification, “ “Method,” and “Outcome.” Un-der a supraheading, “Cost,” are the headings“Activity, “ “Hours,” and “Dollars.” Cost break-downs, problems to be worked on further, andthe other essential information for simple anal-ysis are contained here.

Newman (30,198) has developed a CEA pro-cedure that has been adopted in a number ofpsychotherapy settings. An instrument is usedto measure level of functioning along a range ofdimensions. Newman arrays his outcome datain a matrix, using dimensions such as level offunctioning before treatment and level of func-tion after therapy, The cells of the matrix arecompleted with the number of patients whofunctioned at that level before treatment andwho moved to another level of functioning bythe end of therapy (or who stayed in the samelevel). Next, the cost of treating each patientin that cell of the matrix is summed, and dividedby the number of patients whose functioningchange is described by the cell. The resultingcost-per-patient ratio reflects cost effective-ness in terms of its position in the matrix (seealso 270).

Linear Functions.—To describe and predictrelationships between effectiveness and cost, itis possible to develop equations that describecost-effectiveness relationships (see 305). Thetechniques that would achieve a given level ofeffectiveness also can be chosen from this

graphic model of the cost-effectiveness relation-ship. Not only can effectiveness be predicted,but the effectiveness of particular techniques fordifferent costs can be determined. In an actualsituation, a number of possibilities exist for afunction to describe this relationship (e.g.,linear or exponential). Ideally, if this methodwere chosen before therapy, several plausiblemodels would be chosen, such as the linear andexponential, and the average of predictionsgenerated from them would be used in decisionsuntil further information supported one modelover the other.

Linear Programing.—Finally, some applica-tions of CEA techniques attempt to incorporateinformation on cost limits, as well as on the fac-tors that determine the effectiveness of psycho-therapy. The basic concept underlying linearprograming is to consider not only the factorsthat contribute most to therapy outcomes, butalso the cost of less effective factors and budgetrestrictions. Linear programing is a statisticalprocedure used to find the exact mixture of themost contributory factors that are possiblewithin budget constraints.

The equations for linear programing bringtogether information on which therapy tech-niques, delivery systems, or therapists workbest and on the amounts of each resourceneeded to implement each technique or deliverysystem, or to hire each therapist. Equations canbe used to minimize the total costs of achievinga prescribed degree of effectiveness or benefit.The equations can also be manipulated furtherto discover which cost constraint could be fittedto yield the maximum improvement of effective-ness or benefit. A number of psychotherapy re-searchers have advocated use of these and re-lated techniques to conduct cost analyses in thehuman services delivery such as psychotherapy(e.g., 1,14,72,115,193,310).

Cost= Benefit Analysis

In CBA, benefits are summed using the sameunits (e. g., dollars, person-hours) and costs aresummed using the same units as benefits. Aratio is then derived by dividing total benefitsby total costs. If benefits exceed costs, the ratiois larger than 1, and if benefits are less than

58 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

costs, the ratio is less than 1. Such benefit/costratios provide a convenient economic index ofthe net benefit of an activity.

One reason that CBA may be more usefulthan CEA is that decisions which compare ben-efits to costs of a treatment program are seldommade in isolation. Often, benefits and costs of agiven treatment are most useful when comparedwith benefit/cost ratios of other treatmentscompeting for the same funds. Thus, an alterna-tive treatment program may be available whichcould generate superior benefit-to-cost ratios.The important factors are the difference in thecosts of alternative treatments, the difference inbenefits of alternative treatments, and the ratioof benefit to cost. An argument can be madethat effectiveness measures should be used, insome cases, instead of benefit measures, becauseeffectiveness data probably retain more ac-curate and valid information on treatment out-comes. That very much depends on the situ-ation, however, and it may be necessary to pre-sent both types of data.

To illustrate the problem, consider the appli-cations of CEA methods to psychotherapy byHalpern and his colleagues (22,113,114). Theirapproach views improvement in therapist rat-ings of patient functioning as a monetizable in-crement in the economic value of the patient.This increment is contrasted, in simple benefit/cost ratios, to the monetary cost of treating thepatient. Potter, Binner, and Halpern (216) havemade their models somewhat more sophisti-cated by considering benefits according to theamount of time the patient stays in the com-munity as well as the improvement noted by theend of therapy. Although perhaps useful, thismethod suffers from potential bias in therapistratings of improvement in patient functioningand in assignment of a somewhat arbitraryvalue (e. g., $10,000) to each functioning unitimprovement. It may be more accurate to usethe actual functioning unit scores in a CEA.

Depite such problems, there are still reasonsto prefer benefit measures, especially when theunits in which benefits have been measured aremore directly meaningful than those used byHalpern, et al. (22). Ratios of effectiveness di-vided by cost may not provide as much infor-

mation as benefit divided by marginal cost, be-cause if the latter is greater than 1, the addi-tional benefits of one of the alternative treat-ments can be said to be “worth” the additionalcosts it requires. It is difficult to make a similarstatement about effectiveness/cost ratios.

Fishman’s research (77,78), developed to as-sess the effects of a community mental healthcenter, illustrates the use of similar CBA proce-dures. Fishman views CBA as an experiment inwhich outcome and cost data are gathered usingthe research designs for assessing efficiencydescribed earlier. If the effectiveness of one pro-gram or program component is shown by statis-tical analysis to be significantly superior to theeffectiveness or benefit of another program, andthe costs of the two do not differ significantly,then the program with superior effectiveness ismore cost effective. If the costs of alternativeprograms differ significantly, but their effec-tiveness or benefit does not, then the least costlyprogram is adopted. Fishman acknowledgesthat his model breaks down in situations wherethe significantly more effective or beneficialprogram is also significantly more costly. Thequestion as to whether the increment in effec-tiveness or benefit is “worth” the increment incost is, as noted earlier, a question for marginalbenefit-cost analysis.

Net Benefit Analysis.—It should be recog-nized that ratios of benefit to cost or effec-tiveness to cost do not really yield informationabout the absolute amount of benefits and totalcosts involved. This information may be impor-tant in decisions that must deal with limits onthe maximum cost allowable and on the mini-mum benefit that should be produced. Theamount that benefits exceed costs also may be ofconcern. A benefit/cost ratio of 2 can be pro-duced by a benefit of $200 and a cost of $100 forone program, or by a benefit of $200,000 and acost of $100,000 for another program, but thesetwo programs are not the same. If no more than$2,000 can be spent, the former treatment pro-gram is the only feasible one; if benefits must ex-ceed $2,000, then the second treatment programis the only one possible. To aid in maximizingbenefits, analysts often calculate net benefits,which are present-valued benefits minus pres-

ent-valued costs. This is one of the advantagesin expressing outcomes and costs in the sameunits. If the net benefit is negative, then the pro-gram is not worthwhile; if net benefits exceedzero, the program is worthwhile. This informa-tion may be more useful than ratios in many in-stances, although it might also be desirable toconsider calculations of net benefit per patient.

Sensitivity Analyses.—In any CBA, it is im-portant to consider the impact of alternatively

valuing benefits and the impact of measurementerror. For example, if a benefit of $100,000 is inerror by + 10 percent and the cost of $90,000 inerror by + 10 percent, the benefit/cost ratiomight vary from 0 . 9 1 ( $ 9 0 , 0 0 0 ) / ( $ 9 9 , 0 0 0 ) , t o1 . 3 6 ( $ 1 1 0 , 0 0 0 ) / ( $ 8 1 , 0 0 0 ) , instead of the 1.11($100,000)/($90,000) calculated originally. Pro-viding these alternative calculations would givethe interpreter of the CBA an idea of the possi-ble error.

SUMMARY

There exist a variety of methods to assess thecosts and benefits of psychotherapy and to com-pare the data generated by various studies. Al-though in the case of psychotherapy, unique dif-ficulties arise (in particular, having to do withthe valuation of benefits), the problems of con-ducting CEA/CBAs are not necessarily uniqueto assessments of psychotherapy. In every in-stance, the usefulness of such analyses is very

Statistical analyses can also be applied to suchproblems, Thus, for example, one can calculatestandard error scores which provide a precisestatistical measure of error (see, e.g., 201).Statistical procedures can also be used to test thesignificance of different cost/benefit ratios (30).These ratios are typically calculated for thetreatment as a whole, in which case possibleerror in measurements of costs, effectiveness,and benefits cannot be treated as variance abouta mean, but instead as an absolute error. In suchanalyses, the degree to which measurementerror may influence the benefit /cost ratio can beinvestigated only by first establishing a reason-able range of possible error and next calculating(benefit given error) /(highest cost given error),and (highest benefit given error) /(lowest costgiven error). Some of these procedures are alsoapplicable to CEA.

much dependent on the quality and availabilityof outcome data. It would seem, however, thatmuch more methodological development needsto take place with respect to CEA/CBA beforethese techniques can be used with known reli-ability and validity in psychotherapy assess-ments. The substantive literature describing theapplication of these methods to psychotherapyis examined in the next chapter of this report.

. . . .

6.Illustrative Cost-Effectiveness

and Cost-Benefit Analyses

6Illustrative Cost- Effectiveness

and Cost= Benefit Analyses

As noted in the previous chapter, the applica-tion of cost-effectiveness and cost-benefit analy-sis (CEA/CBA) to psychotherapy is fairly recentand probably far less developed than otherareas of psychotherapy research. Not surpris-ingly, therefore, there are a number of problemswith the available CEA/CBA research. Some ofthese problems have to do with the methodolog-ical limitations of the ways in which psycho-therapy’s outcomes have been assessed (see chs.2 and 3); additional problems relate to the wayCEA/CBA methods have been employed. CEA/CBAs of psychotherapy, for example, focussolely on the costs of treatment and pay mini-mal attention to outcomes. Others use single orinsensitive measures of outcome. Still others failto measure costs or outcomes comprehensively,and many CEA/CBA studies use accountingcosts, rather than opportunity costs. CEA/CBAstudies that use only accounting costs potential-ly underestimate the actual amount of resourcesrequired to provide psychotherapy treatmentsand the benefits that may accrue from their ap-plication. At present, CEA/CBA analyses ofpsychotherapy seem to have neither the meth-odological rigor, nor the breadth of application,to be centrally useful in assessing psychother-apy’s value (see 2).

Despite the problems with CEA/CBA re-search, however, it does not follow that CEA/CBA studies of psychotherapy cannot or shouldnot be conducted. As described in chapter 5,there are methods for valuing the resources usedby psychotherapy; in addition, methods forvaluing the outcomes of psychotherapy, thoughless well developed, can be applied. To the ex-tent that CEA/CBA is a useful adjunct to thehealth policy decisionmaking process (see 203),its application to psychotherapy seems as rea-sonable (see, e.g., 179) as its application toother health problems. The use of CEA/CBAstudies may provide information about the use

and impact of psychotherapy that is otherwiseunavailable. At the very least, such studies mayhelp structure the questions and data aboutpsychotherapy in a way that should aid policy-makers in choosing among alternative pro-grams. CEA/CBA may also be useful in stimu-lating better research about the effects ofpsychotherapy. When resource-effect relation-ships are considered as in CEA/CBA, aspects ofpsychotherapy are highlighted which otherwisemight be ignored; thus, for example, cost analy-ses may suggest assessing a wider range of vari-ables than would usually be considered theoret-ically relevant.

Although CEA/CBA studies of psychother-apy currently appear to have limited policyusefulness, it should be recognized that datafrom such studies sometimes provide helpfulpolicy information. There are reliable and validdata which indicate that some conditions of of-fering psychotherapy are more cost effective orcost beneficial than others. Although these datamay have limited generalizability, they may behelpful in making specific resource allocationdecisions (e.g., reimbursement policy for mentalhealth treatments). The data also suggest prom-ising areas for future research.

The review below describes selected aspectsof a number of recent cost-effectiveness andcost-benefit studies of psychotherapy. The focusis on individual studies, because syntheses of theCEA/CBA literature in psychotherapy are notyet available. In general, the available CEA/CBA studies do not compare different psycho-therapy treatments—instead, they deal with thefactors that affect the provision of psychother-apy treatments. Thus, a major emphasis ofCEA/CBA studies conducted to date has beendifferences among treatment settings. Anotheremphasis has been the analysis of conditionsunder which either low-cost therapies or poten-tially high-benefit treatments have been used.

63

64 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

Because of the difficulties associated withmeasuring psychotherapy’s benefits (see ch. 5),the literature is probably biased toward assess-ing low-cost treatments. This characteristic ofthe available studies should be kept in mind.

The attempt in the present review is not toprovide a comprehensive analysis of the CEA/CBA literature, but to illustrate the types ofstudies available and their findings. A numberof the studies reported here were included, insome form (e.g., were considered within a re-view that was analyzed), in the discussion of ef-fectiveness in chapter 4. They are described inmore detail in the present chapter because oftheir explicit consideration of resource issues(i.e., assessment of cost and/or benefits). In-cluded in the present chapter are a number of

studies that assess effectiveness poorly and con-tain other methodological flaws. These studiesare included to suggest the potential contribu-tion of CEA/CBA research on psychotherapyand to identify some of the problems of con-ducting CEA/CBA.

The discussion below is organized accordingto the treatment factors and patient conditionsto which CEA/CBA has been applied. First, thereview considers a number of treatment-relatedcharacteristics that have been subjected toCEA/CBA. Next, it considers therapist factorsas they relate to costs of treatment and out-comes. Finally, several specific patient condi-tions and CEA/CBA studies of psychotherapytreatments applied to these conditions arereviewed.

TREATMENT-RELATED CHARACTERISTICS

As noted above, a focus of psychotherapyCEA/CBAs conducted to date has been charac-teristics of the treatment system in which psy-chotherapy is provided. Some studies have fo-cused on the provision of psychotherapy undervarious forms of institutional care, and somehave compared, for example, institutional andoutpatient/community-based treatments. Theresults of these comparisons are not easy tosummarize, although it does seem clear that theconditions of treatment can vary widely interms of the resources they use and their impact.

Inpatient Therapy

One area in which CEA/CBAs of psychother-apy have frequently been done concerns institu-tionalization of mental patients and the provi-sion of inpatient psychotherapy. The costs ofinstitutionalizing a patient are high, and thecosts of providing therapy (in addition to caringfor the patient) are potentially even higher. Anumber of studies have examined the costs andbenefits of various treatments offered patients,in particular, the provision of therapy alongwith custodial or milieu care. The conclusion ofa number of these studies is that the provisionof psychotherapy can provide more cost-effec-

tive or cost-beneficial treatment (e.g., 24,130,175,273,289). These findings were obtained be-cause therapy may reduce the amount of hospi-talization or achieve larger benefits in someother area.

McCaffree (175), in one study of the benefitsof “intensive” inpatient treament, reviewed thecosts and results of treatment in the WashingtonState hospital system during two periods. Dur-ing the two periods, there was a shift in thetreatment of institutionalized patients. In thefirst period, only custodial care was provided;in the second period, various forms of psycho-therapy (which was called “intensive therapy”)were provided as well. McCaffree assessed thecosts for both types of treatment. He includedboth public costs (e.g., subsistence and treat-ment) and private costs (e. g., loss of patient in-come). McCaffree found that the costs per pa-tient in intensive therapy were about 50 percentless than the costs of custodial care. Moneyswere saved in intensive therapy primarilybecause patient stays were much shorter (22 v.42 days). Unfortunately, McCaffree’s data arenonexperimental, and some of his data indicatethat the intensive therapy patients may havebeen less disabled than the custodial care group.

Ch. 6–Illustrative Cost-Effectiveness and Cost-Benefit Analyses ● 65

In addition, McCaffree ignores psychologicalmeasures of effectiveness. It is possible that theintensive therapy group was no better afterreceiving treatment, but that criteria for patientdischarge had been altered between the two datagathering periods. Also important is the factthat not only was more psychotherapy offeredin intensive treatment, but (see 173) drug treat-ment was introduced at the same time.

A later study by May (173) assessed the costeffectiveness of milieu care (see also 24) and fourother treatment methods. A patient in milieucare is provided a therapeutic environment(nurses and others who deal with the patient arespecially trained). May used experimental pro-cedures to assign schizophrenic patients to treat-ment conditions and assessed both costs and ef-fectiveness. He found that milieu therapy wasalmost as costly as psychotherapy, but was theleast effective of the five treatments. Drugswithout psychotherapy were the least costlytreatment, and drugs with psychotherapy werethe most effective. It is difficult to determinewhether May’s results are generalizable to otherpatient conditions and psychotherapy/chemo-therapy as they are currently provided.

Community= Based VersusInstitutional Therapy

Although it appears to be more cost beneficialand cost effective to provide inpatients with in-tensive treatment, other alternatives have beenstudied. Thus, a major focus of CEA/CBA stud-ies has been community-based approaches forproviding psychotherapy. Community care hasthe potential to provide major cost savings,because such care reduces the costs not directlyassociated with treatment (i. e., the costs ofhousing, feeding, and supervising an institution-alized patient are higher than the costs of thesame services provided in the community).

Thus, for example, Binner, Halpern, and Pot-ter (22) used data from almost 600 patientswho had received either inpatient or outpatienttreatment. Binner, et al. quantified intensity ac-cording to a factor-analyzed combination ofresource utilization, therapeutic involvement,support from the therapeutic environment, and

length of stay (see 247). Binner, et al. found thatless intensive treatment resulted in higher bene-fit/cost ratios. Categorizing patients by severityof dysfunction at admission, and stratifyingtherapy intensity at four levels, Binner, et al.showed that benefit/cost ratios were greaterthan 1 (benefits exceeded costs) for all combina-tions of impairment and intensity, but were sig-nificantly larger for low-intensity therapy pro-vided to less impaired patients. Average bene-fit/cost ratios ranged from 2.28 for patientstreated more intensively to 5.23 for patientsgiven the least intensive therapy. These resultssuggest that therapy may not be effective forseverely impaired populations and that re-sources should be directed to outpatienttreatment.

Although Binner, et al.’s (22) study collecteddata on outcomes as well as cost, it is neverthe-less difficult to generalize from it. One problemis that patients were not randomly assigned todifferent levels of therapy intensity. The classi-fication scheme for intensity, though mathe-matically sophisticated, cannot control forpossible differences in the type of patient whowas assigned to more versus less intensivetherapy. It is also not clear whether the patientswho participated in community-based treat-ment had dysfunctions as severe as those of theinpatients. Furthermore, the outcome measureused by Binner, et al. was a gross measure basedon a therapist rating. The valuing of this benefitwas done by assigning an arbitrary dollaramount to changes in therapists’ ratings. Final-ly, Binner, et al. used operations costs, ratherthan opportunity values, and this may have ledto either an underestimate or overestimate. It isdifficult to determine what bias was incorpo-rated and how this relates to the procedure theyused to value therapist-rated changes.

In another study of an outpatient treatmentprogram, Washburn, Vanicelli, Longabaugh,and Sheff (292) studied a “daycare” treatment.Daycare is actually a modified form of inpatienttreatment where patients return to their ownhomes (or families) each evening. The research-ers collected a variety of effectiveness data, in-cluding checklists completed by patients, those

66 Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

who knew them well, and their therapists. Pa-tients were assigned randomly to daycare orstandard inpatient care. Analyses of effective-ness data collected at 6-month intervals duringtherapy and at a 2-year followup found only afew significant differences. Daycare patientsevidenced somewhat less subjective distress,functioned better in the community, and wereless of a burden to their families than patientswho received standard inpatient therapy. Costdata also collected at 6-month intervals, how-ever, showed significantly lower costs for day-care. The lower cost of daycare treatment seemsto be the primary determinant of cost effective-ness, although daycare also seems to have im-proved effectiveness to some degree. What ismissing from Washburn, et al. ’s study, unfor-tunately, is a no-treatment or a placebo treat-ment condition, which would allow inferencesabout how much of the outcome is caused byfactors not related to treatment procedures (i.e.,spontaneous remission), or by the patient’sexpectations.

In another comparative study of communityversus institutional treatment, Murphy andDatel (192; see also 57) projected the costs andbenefits for 52 mentally ill and mentally re-tarded patients who were placed in the commu-nity from State institutions. The costs of com-munity care included housing and subsistence,as well as the costs of community treatment.Benefits included the cost savings of not havingto provide institutional care and the wages/fringe benefits received from jobs. Costs andbenefits were adjusted for present value (a 0.08discount rate was used) and inflation. Murphyand Datel’s results were organized in terms of 12patient categories. Their findings indicated that10-year projected benefits exceeded 10-year pro-jected costs of community care, yielding bene-fit/cost ratios of between 0.99 to 11.86. T h eaverage ratio was substantially greater than 1,indicating that community care was superior.

Murphy and Datel’s (192) study, although itconsidered a broader array of benefits and coststhan most CBA studies and used present valueand inflation adjustment procedures, does havesome methodological problems. Although itwas recognized that there are psychological ben-

efits to patients and their families and data onsuch things as marriage, normality of appear-ance, mobility in the community, and employ-ment were presented, these data were not usedin the benefit/cost ratios. Valuing such benefitsis admittedly difficult, but Murphy and Datel’sanalysis, as presented, may underestimate ther-apy’s benefits. Another problem with this studythat limits its validity is that patients who re-fused community placement were not includedin the analysis. These individuals were consid-ered treatment failures and excluded from theanalysis. If a significant number of these pa-tients returned to the institution, costs of estab-lishing them in the community (“set-up” costs)and of readmitting them to institutions shouldhave been incorporated in the CBA. The studyalso didn’t consider cost ratios for an alternativeprogram. As discussed in the preceding chapter,there always is an implicit alternative to which acost-benefit calculation is compared and not allprograms with benefit/cost ratios exceeding 1should be continued. An obvious comparisongroup for the Murphy and Datel study wouldhave been a randomly selected group from thepotentially deinstitutionalizable population thatwas retained in the institution or was released toanother program.

Cassell, Smith, Grunberg, Bean, and Thomas(32) obtained cost and effectiveness data foralmost 500 deinstitutionalized patients, most ofwhom had been diagnosed as chronic schizo-phrenics. These patients had resided in mentalhospitals for an average of 18.2 years prior todeinstitutionalization, and the costs of hospitali-zation had totaled $70,000 per patient beforerelease. After deinstitutionalization, the costs ofwelfare, followup, drugs, and rehospitalizationfor those who could not adapt to communityliving was only $1,575 annually, or 2.44 timesless than the cost of institutionalization. Al-though very few of the patients would havebeen discharged had their deinstitutionalizationprogram not begun, 49 percent of the men and38 percent of the women under age 65 were em-ployed at least 3 months during the 2-yearpostrelease period. More than 20 percent of themen and 10 percent of the women were em-ployed for at least 13 months following their

Ch. 16—Illustrative Cost-Effectiveness and Cost-Benefit Analyses ● 67

release. Utilization of medical care by the dein-stitutionalized patients was also significantlyless than for normally adjusted persons in simi-lar groups. In sum, the cost was less and effec-tiveness greater for noninstitutional comparedto institutional care provided to severely dis-turbed patients. Other studies of reinstitutional-ization, which similarly have tracked patientsafter release, have yielded similar findings (see,e.g., 79,156,191).

An important methodological limitation ofthese deinstitutionalization studies is their useof nonexperimental designs. Typically, theseCEA/CBA studies follow deinstitutionalized pa-tients, accounting for the resources they require.This is potentially problematic given the factthat new treatments were simultaneously beingintroduced within mental hospitals (see 173,175)and the fact that patients who were reinstitu-tionalized are likely to be less disabled thanthose who remain in the hospital. There is also aproblem of adequately assessing the costs ofcommunity care. Although most studies includemajor housing and support costs for mainte-nance of patients in the community, a compre-hensive accounting of community costs may bedifficult (especially in comparison to the ob-viousness of many costs borne in a hospital). Asa result of these factors, the high benefit/costratios for community treatment may be overlyoptimistic.

The study reported by Weisbrod (296; seealso 297) illustrates the implications of consider-ing these factors comprehensively in a CBA.Weisbrod compared traditional therapy in amental institution against a community careprogram using random assignment of patients.Therapy in the mental institution was brief(usually less than 1 month) and was followed bycommunity care delivered by local mentalhealth agencies. The usual stay in the hospitalwas 17 days, but many patients returned to theinstitution. Therapy in the alternative commu-nity-based program involved relatively littletime in institutions (i.e., every effort was madenot to hospitalize patients). During this 14-month “community living” trial, patients had tocope with normal living conditions and were re-quired to take responsibility for the problems

that might, in other circumstances, result intheir return to an institution.

Sixty-five patients were included in eachtherapy condition and data were collected forthe first 12 months of treatment. Weisbrod’s(296) cost assessment procedures were compre-hensive and included a variety of direct and in-direct costs of treatment. Included also were lawenforcement costs and family burden costs (e. g.,lost wages caused by the patient). Weisbrod’sresults indicated that community care wasslightly more costly, but yielded higher benefitthan the institutional treatment program. Thecosts were higher in the community program be-cause patients were closely supervised and re-ceived intensive therapy. The benefits of com-munity treatment were higher primarily becausethe outpatients earned more from employment.

Despite the use of random assignment andcomprehensive assessment of cost and benefits,there are important limitations to the general-izability of Weisbrod’s (296) findings. Althoughdata were collected over a relatively long periodof time (4 years, from 1972 to 1976), each pa-tient participated for only 14 months. That maybe too short a time to assess the effects of treat-ment. In addition, only a rather small numberof patients were included (less than 150), allfrom the same geographic area. It is not clearwhether the program actually studied a repre-sentative sample of those eligible for reinstitu-tionalization. There is also no evidence thateither community or institutional treatment issuperior to no treatment, since a no-treatmentcontrol condition was not included. Of course,the patients in Weisbrod’s study were severe-ly dysfunctional, and withholding treatmentwould have been difficult.

Residential Versus InstitutionalTherapy for Problem Children

Most of the aforementioned studies involvedschizophrenics and other severely disturbedadults. Another typical application of CEA/CBA in assessing mental health treatments arestudies which have examined the use of psycho-therapy for “problem populations” (see 63). Forexample, there has been great interest in study-

68 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

ing the use of psychotherapy with juvenile delin-quents. There is evidence that a substantialnumber of serious crimes are committed by ju-veniles under 14 years of age (see 286), and thereis considerable correlational evidence that crim-inal behavior in youths is related to adult crimeand mental dysfunction (see, e.g., 233). T h emagnitude of the costs associated with theseproblems is very great.

A number of studies have investigated the useof behavior modification procedures with trou-bled youths. Phillips (213), for example, pre-sented data on the cost and effectiveness of aresidential token economy therapy for juveniledelinquents called Achievement Place (see also243). Token economy therapy is an applicationof learning theory to establish a therapeuticmilieu. Along with the direct costs of treatment,Phillips assessed effectiveness data on severalvariables, including police and court contacts,school attendance, and grades before, during,and after therapy. Before therapy, AchievementPlace youths averaged about 3.8 contacts peryear with the police or courts. After 1 year oftherapy, Achievement Place delinquents aver-aged only 0.75 police and court contacts and nocontacts 2 years after therapy. In comparison toa traditional program for juvenile delinquents (a

THERAPIST VARIABLES

Because the major proportion of resourcesused in psychotherapy relates to the cost of per-sonnel time, a number of cost comparisons ofdifferent therapists have been conducted. In-cluded in these studies is an analysis of the dif-ferences in fees charged by various professionalsand paraprofessionals and the impact of thera-pist variables on outcome.

Professional Therapists

Karen and VandenBos (132) report one of thefew available studies on the cost effectiveness ofdifferent professional therapists. A small num-ber of patients diagnosed as schizophrenics weretreated by psychologists, by psychiatrists, or by

special school), Achievement Place was muchmore effective and one-third as costly ($6,000 v.$20,000 to $30,000 per delinquent).

Although Phillips, et al. (213) provide verypositive cost-effectiveness data, it should benoted that their findings are in contrast to thoseof Powers and Witmer (217). In what has be-come a classic study (known as the Cambridge-Somerville youth study), 650 youths werematched and randomly assigned to either long-term counseling and supervision or no mentalhealth treatment. The participants in this studywere tracked from 1939 to 1976, and data on avariety of psychological and social dimensionswere collected (see also 178). Unfortunately, nolong-term benefits can be demonstrated, andthere is some disturbing evidence that the ther-apy-treated youths have faired worse than thosewho received no treatment (on such criteria asalcoholism and later mental health). It is impor-tant to recognize, however, that the treatmentreceived by these youths may be far differentfrom that provided by current standards (see269). In some respects, the Cambridge-Somer-ville approach is not comparable in rigor totreatments such as Achievement Place (whichwas based on a clear-cut theoretical rationale).

drugs without therapy. Karen and VandenBosfound that the cost of therapy provided by psy-chologists was substantially less than the cost oftherapy provided by psychiatrists, primarilyowing to lower hospitalization and drug use intherapy provided by psychologists. The averagecost per patient treated by psychologists was$7,813, but the average cost per patient was$12,221 when psychiatrists provided treatmentand $17,234 when drugs alone were used intherapy. Long-term costs due to recidivism alsofavored psychologists over psychiatrists. Pa-tients treated by psychologists were hospitalizedan average 7.2 days in a 2-year followup, com-pared to 93.5 days for patients treated by psy-chiatrists and 99.8 days for patients treated bydrugs alone.

Although these findings suggest that psy-chologists may be more cost-effective therapistsfor schizophrenics than psychiatrists, the gener-alizability of these data is hard to determine. Feeschedules for psychologists vary widely, and thespecific conditions of Karen and VandenBos’ re-search setting may have affected the results inunknown ways. One important outcome wasthat therapy given by either type of professionalwas less costly than drugs alone: 17 percent lesswhen provided by psychiatrists, and 33 percentless when provided by psychologists. While,again, it is difficult to know the generalizabilityof these findings, these data suggest an area forfurther CBAs.

In a related study, Karen and VandenBos(133) report that therapy provided by bettertrained therapists is more effective, thoughmore costly, than therapy provided by lesstrained therapists. The reduction in costs resultsfrom generally shorter lengths of stay and othersavings in types of services required. This find-ing was obtained despite a gross underestimateof the actual cost of hospitalization (132). Thegeneralizability of such studies, however, maybe limited to the type of patient they used.

There is evidence, for example, that hospital-ized neurotics can be treated by nurse therapistswith better cost/benefit ratios than with thera-pists who have more training (i.e., a doctorate),Ginsberg and Marks (95) assessed the effec-tiveness of treatment by nurse therapists forbrief behavioral psychotherapy (patients usedan average of 9 sessions) for neurotic andphobic patients. Ginsberg and Marks used datafrom 1 year prior to therapy and 1 year post-therapy to estimate the impact of therapy. Thedata indicated that therapy significantly re-duced symptomology and resulted in a numberof tangible (i. e., valued) benefits. The benefitsincluded reduced use of medical services and im-proved work productivity. Since the study didnot employ comparison groups, it is impossibleto estimate the comparative effects of nurse-therapists; however, the study does suggest apotentially useful direction for future analyses.

Gabby and Leavitt (89) report similar findingsfor therapy on a population of neurotics. A self-supporting nonprofit clinic provided long-term

therapy and was able to charge an average of 50percent of the fees charged by private practi-tioners in the area ($13.50 to $15/hour v.$19.95/hour at that time). A key factor in theclinic’s being able to offer lower fees was its ex-tensive use of nonpsychiatric staff. Psychiatristswere called in only when necessary for consulta-tion. Although Gabby and Leavitt provide anexample of how costs can be reduced, they donot report outcome data in any detail, it is un-clear whether the effects of treatment in theclinic setting were as good as those in traditionalsettings. In addition, it is unclear whether pa-tients at the clinic were more or less disturbedthan other patients in the community. There isalso some reason to suspect that they attracted aselect population of patients.

Nonprofessionals

One focus of CEA/CBA studies has been thetraining of parents to provide therapy to dis-turbed children (see, e.g., 111,211,290). In onestudy, Rubenstein, Armentrout, Levin, andHerald (243) placed 36 emotionally disturbedchildren in normal homes. Parents residing inthese homes had received training in child man-agement skills and received salaries and expensereimbursement for caring for the disturbedchildren. Lengths of stay ranged from 9 to 26months, and the patients were supervised byother “parent therapists. ” Professional mentalhealth specialists also met weekly with parents.Psychometric tests, grades, and a behaviorchecklist showed that children treated by parenttherapists were as improved as comparablechildren treated in two residential programs.Costs of parent-therapist treatment were onlyhalf of residential treatment costs ($30.60 v.$63.77 per child per day).

What seems to have reduced costs was thedistribution of professional expertise over anumber of “helpers. ” This simple modificationof the traditional way of providing psychother-apy (direct contact between professional thera-pists and patient) would be expected to reducecosts. However, the cost reductions suggestedby the Rubenstein, et al. (243) study probablyare overestimated because they do not includemany of the costs of training parents. The cost

70 ● Background Paper #3: The Efficacy and cost Effectiveness of Psychotherapy

findings also are limited by Rubenstein, et al.’suse of “comparable” rather than randomly se-lected comparison groups. Perhaps the childrenassigned to the parent-therapist program wereless severely disturbed or had characteristicsthat predisposed them to benefit more fromparent therapy than other children. As is typicalof many CEA and CBA studies, costs used inRubenstein, et al.’s study were accounting costs,rather than opportunity costs. These costs maywell underestimate the use of resources.

Other CBA and CEA studies have explicitlycompared the use of professional versus para-professionals. Yates (306), for example, reportsa study of paraprofessionals who conductedtherapy for obese patients. The therapy was lessthan one-tenth as costly as therapy provided bypsychologists and psychiatrists, but appeared tobe equally effective. Effectiveness was measureddirectly in terms of the number of pounds lost.Yates’ study, however, used a quasi-experimen-tal design in which patients selected their owntherapy, and it may be that those who selectedthe paraprofessional treatment condition be-lieved they needed less help to lose weight (i.e.,were most motivated). Also, while it can be as-sumed that nonprofessionals are less costly thanprofessionals, costs were not directly reported(see also 282).

Self-directed therapy, with or without con-sultation with a professional, has also begun toreceive some attention (see 96). Many of thesestudies have focused on dysfunctions typically

not treated by professional psychotherapy, butthere are some examples with depression, anxie-ty, phobias, and sexual dysfunctions. Such ther-apies involve the use of a book or manual todirect the individual and have been mainlybased on learning and phenomenological theo-ries. Case studies of therapies directed bymanuals have yielded results indicating thatpositive effects can be obtained with such low-cost therapies (e. g., 294,299), but a thoroughevaluation of the effectiveness and cost effec-tiveness of these procedures has not been done.The lack of comprehensive cost data in suchstudies further limits any conclusions about costeffectiveness or cost benefit.

One formal cost-effectiveness study of self-directed therapy was conducted by Marston,Marston, and Ross (170). Marston, et al. mailedobese patients weekly weight-reduction read-ings (“bibliotherapy”) and had them respond towritten questions about the readings andweight-loss problems. They found their form oftherapy to be as effective as weekly visits with aprofessional therapist and much less costly. Un-fortunately, the range of cost and effectivenessdata collected was limited; in addition, theseprograms have not been compared with tradi-tional psychotherapy treatments. Further stud-ies have shown that some contact with a profes-sional or paraprofessional therapist is necessaryfor these bibliotherapies to be effective com-pared to no-treatment groups, even if this con-tact is made by phone or mail (305,307).

DRUG ADDICTION AND MEDICAL UTILIZATION

The studies cited above principally focus onCEA and CBA of different ways of offering psy-chotherapeutic treatments. CEA and CBA re-ports have also been developed to study treat-ments for problems such as alcohol and heroinaddiction. These are high-cost problems to soci-ety and are also problems that have highly vis-ible costs and benefits/effects. Overutilizationof medical services, also a costly problem, hasalso received considerable attention (e. g., 60,129), and it has a number of implications forhow and under what conditions psychotherapy

should be evaluated in a national health insur-ance program. Below several CEA/CBA studiesof treatment for these high-cost problems aredescribed.

Psychotherapy for Drug Addictions

The obvious relationship between alcoholand opiate addictions and subsequent psycho-logical dysfunction, coupled with the highsocietal costs of such addictions, has stimulateda number of CEA and CBA studies of therapies

Ch. 6–Illustrative Cost-Effectiveness and Cost-Benefit Analyses ● 71

for drug addictions. Involvement of multidisci-plinary teams in therapy for drug addictionsseem to have generated more comprehensiveCEA and CBA in this area than in any otherand, in general, the findings of these studies isthat therapy is beneficial. This conclusion mustbe tempered, however, by recognizing thattreatments for drug addiction almost always in-volve more than psychotherapy.

Rufener and his colleagues (244,245), for ex-ample, conducted a CBA of five different thera-pies for heroin addiction: 1) methadone mainte-nance, 2) therapeutic community, 3) outpatientdrug-free treatment, 4) outpatient detoxifica-tion, and 5) inpatient detoxification. Benefitswere calculated by estimating social costs in-curred directly and indirectly as a result of her-oin use (e. g., crime, incarceration, court costs),the reduction in costs produced by rehabilitat-ing a heroin abuser, and adjustment of benefitsfor the relative effectiveness of different pro-grams. Costs were assessed from accountingrecords of providing each therapy. Rufener, etal. considered benefits under three assumptionsof the size of heroin abuser population and threedifferent discount rates for present-valuing. Theresulting ratios were all greater than 1 (benefitsexceed costs) and showed that outpatient drug-free therapy to be the most cost beneficial.Drug-free therapy yielded a 12.82 benefit/costratio for the intermediate abuser population size(and included discount rate assumptions). Apossible problem of this has to do with esti-mated adjustment cost savings for differentialrelapse rates which may have been overly opti-mistic. In addition, the study did not use ran-dom assignment of patients to different treat-ment techniques, and it is difficult to determinethe possible effects of the research design.

In another study, Hall, Bass, Hargreaves, andLoeb (112) report a 20-percent reduction in theuse of opiates and barbiturates for outpatientdetoxification patients. Random assignment ofsubjects to behavior therapy and no treatmentconditions was accomplished, and effectivenesswas assessed by urine tests. The behavior ther-apy consisted of reinforcing patients (by payingthem up to $10/day) for drug-free urines. De-pending on the day urines were collected, 40 to

50 percent of the paid subjects had not useddrugs. Since most estimates of the daily cost tosociety of heroin use are five or more times thatcost, this reinforcement contingency seemspromisingly cost beneficial, even when person-nel, facility, and testing costs are added. Otherdata suggest that patients did not use theirpayments to purchase illegal drugs.

Sirotnik and Bailey (262), in a similar study,conducted a CBA of heroin addiction therapies.Their investigation involved 285 patients over a1.5-year period. They found that comprehen-sively defined benefits exceeded costs by morethan a 2.5 ratio under each of a range of cost-savings assumptions. This analysis probably isconservative, because it did not consider ben-efits that might accrue to patients and societyafter the program. Unfortunately, nonrandomassignment to therapies and the absence of acontrol group limit, the implications of thisstudy.

Even larger benefit/cost ratios may be pro-duced by long-term therapy for addicts, al-though there is much debate on this point. Aronand Daily (8), for example, found that long-term therapy was more effective and, in total,less costly than short-term therapy for drug ad-diction when costs of re-entry to therapy andlong-run effectiveness were assessed. Cost-effec-tiveness ratios were $4,624 per successfullytreated addict in long-term therapy, but $5,988per successful patient in short-term therapy.

Maidlow and Berman (167) contrasted thecost effectiveness and cost benefit of a drug-free therapeutic community and methadone ap-proaches to treating heroin addiction. The aver-age stay of 4 years in drug-free communities wasfound to generate a 65-percent success rate,resulting in a direct cost per successful patient of$17,760. Methadone or other drug substitutionhad a higher, 87-percent success rate at a life-time cost effectiveness of $45,000 per successfulpatient. Considering the probability that formeraddicts would leave therapy, and present-valu-ing future benefits and costs (thus reducing life-time cost of drug substitution), it would seemdrug substitution is more cost beneficial thandrug-free therapeutic communities. However,Maidlow and Berman’s study, although it uses

72 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

sound CBA methodology, did not use randomassignment or a no-treatment control group andthere may be important biases in who selectsparticular treatment.

Goldschmidt (101) described a comprehensivemodel of CEA for health care, applying it toanalysis of heroin addiction therapies. Sampling1,640 patients over a 6-month period he found1,241 who could be interviewed and used datafrom them to compare the cost effectiveness ofdrug substitution (methadone) to the therapeu-tic community. Using several variables to assesseffectiveness and operations cost for the twoapproaches, high variability was found in non-recidivism measures of effectiveness (10.9 t o33.7 percent for methadone therapy, depend-ing largely on length of stay, v. 12.5 to 47 per-cent for therapeutic communities). Goldschmidtfound that the annual cost per patient of thetherapeutic community was about four timesthe cost of drug substitution. Because thera-peutic communities were found to treat morepatients and have somewhat more persistent ef-fects, the average cost/successful patient ratiowas 1.7 times higher for therapeutic commu-nities than for drug substitution, thus favoringdrug substitution as more effective for its cost.This finding was maintained for both “nor-malcy” (i. e., some use of drugs, but normalfunctioning) and “heroin-free” criteria of effec-tiveness. This study shows the impact that treat-ment system costs can have on CEA findings.Consideration of lifetime costs of methadonemaintenance, however, might reverse the direc-tion or magnitude of the findings. No informa-tion was provided on opportunity value andcomprehensive costs and, as well, no-treatmentcontrol groups were not evaluated.

To summarize the status of CEA and CBA fordrug addictions, while this area has receivedmuch attention, there are still serious short-comings with the available research. Theseshortcomings have to do primarily with the re-search designs used. Although some of the find-ings are impressive in that consistent cost/bene-fit ratios greater than unity are found, theremay be other explanations for these results. Inaddition, it should be recognized that drugabuse remains a serious problem and it is not

clear that the available studies are representa-tive of the treatment programs being conducted.

Psychotherapy and Overutilization ofMedical Services

Depending on the definition used, it has beenestimated that medical services are overused (orin some other way abused) by between 20 and60 percent of those who seek them (e.g., 54,174,182). Various researchers and analysts haveconcluded that medical services often are usedto ameliorate mental problems, rather than thetreatment of physical dysfunctions. Regier, etal. ’s (225) data on the role of primary carephysicians in the delivery of mental health serv-ices is one indication of this problem (see ch. 1).It has been suggested that psychotherapy maybe used to reduce individuals’ dependence onmedical services. Thus, for example, a 4-yearstudy by Cavanaugh (cited in 119) found thathospitalization for physical ailments was re-duced from an average 111 to an average 53days by psychotherapy, resulting in a $1.1 mil-lion savings (which was greater than psycho-therapy costs).

Cummings and Follette (53; see also 99) re-ported that a single session of psychotherapy re-duced utilization of medical services for highutilizer patients by 60 percent over a 5-yearperiod. Additional sessions further reducedmedical care utilization: patients attending twoto eight sessions subsequently decreased medicaluse by 75 percent. Continuing their study on useof medical services in the large Kaiser-Perma-nente health insurance plan, Cummings (51)used findings from Cummings (50) and Cum-mings and Follette (52,53) to contrast the bene-fits of medical cost savings to costs of four dura-tions of psychotherapy.

Cummings formed benefit/cost ratios by di-viding medical care utilization (number of vis-its) for the year preceding therapy by the sum ofsubsequent medical care and psychotherapy vis-its. Very brief psychotherapy (one to four ses-sions) generated “cost-therapeutic effectiveness”(actually crude benefit/cost) ratios of 2.59. Psy-chotherapy lasting from 1 to 15 sessions had asimilarly positive benefit/cost ratio of 2.11.

—

Long-term therapy (more than 16 sessions) andinterminable therapy were found to be less cost-beneficial (benefit/cost ratios of 1.14 and 0.91,respectively). Not receiving any psychotherapy,however, generated the worst ratio of all: 0.88.

Other investigators have found specific re-ductions in use of physician services, especiallylaboratory and X-ray services. In one report,Goldensohn and Fink (100) found that these ef-fects followed after the administration of psy-chological tests. A number of other studiesshow similar, and often monetized, reductionsin medical care utilization following a variety offorms of psychotherapy, including brief treat-ment (e. g., 125). In many of the studies, reduc-tions in medical utilization have been directlyrelated to reduced costs.

Jones and Vischi (129), at the National Insti-tute of Mental Health, have conducted a com-prehensive review of the literature on the “off-set” benefits of psychotherapy treatments. Theyreviewed 25 studies that examined the impact oftherapy for mental illness, alcohol abuse, anddrug abuse on medical utilization. Their conclu-sion was that, under certain circumstances,medical care utilization does “appear” to bereduced as a result of therapy. In particular, fortreatments of mental illness, they found 12 stud-ies (out of 13 available studies) which demon-strated reduced medical care utilization follow-ing therapy. The median reduction was 20 per-cent, and the range (for the 12 studies) was 5 to85 percent. The reduction in medical care util-ization was accompanied by parallel reduction

SUMMARY

This chapter has provided a number of illus-trations of the use of CEA/CBA methods forassessing a variety of psychotherapy treatmentalternatives and mental health problems. Theconduct of CEA/CBA studies of psychotherapyresearch is not widespread, and many importantareas and types of psychotherapy have beenignored to date. The review in this chapter sug-gests that, for some treatment system character-istics and for some problems, psychotherapy

in cost (mental health treatments being typicallyless costly than medical treatment). The investi-gators report a variety of benefit/cost ratios,which range from 0.95 to 2.11.

Jones and Vischi (129) also reported a varietyof studies in which psychotherapeutic treat-ments were applied to alcoholism and drugabuse. These studies report similar findings (ofreduced medical care utilization) although theytended to have more methodological problemsthan the mental illness studies. A particularproblem for all of the studies was the frequentlack of appropriate control group conditions.Another problem was the selection of time peri-ods for analysis. Typically, patients enteredthese studies after a period of high use of serv-ices and it is difficult to attribute the reductionsolely to the mental health treatment (it couldreflect a natural change in medical needs). Jonesand Vischi, although optimistic about the use ofpsychotherapy to reduce medical utilization,recommend more rigorous research that in-cludes better cost-benefit data.

In a number of cases, psychotherapy has beenemployed as an adjunct to medical services(e.g., 143,204,215,238). Although there is notyet a substantial literature describing the effec-tiveness of psychotherapeutic treatments used inthis way, such fields as “behavioral medicine”are being developed. It would seem that as theeffectiveness literature on such behavioral-mental treatment matures, cost and benefit datashould also be assessed.

appears to be cost beneficial and that it can bemade more efficient (improved cost effective-ness). These findings must be considered tenu-ous, however, given the methodological prob-lems inherent in many of these studies. What isneeded is a more systematic application ofrigorous research designs that employ no-treat-ment and placebo treatment conditions, as wellas more comprehensive measures of outcomeand cost.

74 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

It would seem that we possess the scientific potential to improve well-being and to be costtools to assess psychotherapy and to use this in- effective. Although one will probably never beformation in making policy decisions about totally satisfied with the answers provided bysocietal support for these treatments. It would research, that seems a poor reason not to at-also appear that we have substantial knowledge tempt” more rigorous and comprehensive scien-which indicates that these treatments offer the tific analyses.

Appendixes

.*‘

Appendix A.— Description ofOther Volumes of the Assessment

The overall OTA assessment, The lmplications ofCost-Effectiveness Analysis of Medical Technology,consists of a main, policy-oriented report plus fivebackground papers. The present volume, The Ef-ficacy and Cost Effectiveness of Psychotherapy, isone of the background papers. The main report andthe other background efforts are briefly describedbelow.

The main report examines three major issues: 1)the general usefulness of CEA/CBA in decisionmak-ing regarding medical technology, 2) the methodo-logical strengths and shortcomings of the technique,and 3) the potential for initiating or expanding theuse of CEA/CBA in six health care programs (reim-bursement coverage, health planning, market ap-proval for drugs and medical devices, ProfessionalStandards Review Organizations, R&D activities,and health maintenance organizations), and most im-portantly, the implications of any expanded use.

The prime focus of the report is on the applicationof CEA/CBA to medical technology (i. e., the drugs,devices, and medical and surgical procedures used inmedical care, and the organizational and support sys-tems within which such care is provided). With theexception of the present background paper on psy-chotherapy, the report does not address psychosocialmedicine. Other aspects of health, such as the envi-ronment, are not directly covered either. The find-ings of the assessment, though, might very well applyto health care resource decisionmaking in general,and with modification, to other policy areas such aseducation, the environment, and occupational safetyand health.

The main report contains chapters on methodol-ogy, general decisionmaking, each of the six healthprograms mentioned above, and the general useful-ness of CEA/CBA. It contains appendixes covering asurvey of current and past uses of CEA/CBA byagencies (primarily Federal), a survey of the resourcecosts involved in conducting CEA/CBAs, a discus-sion of ethical issues and CEA/CBA, and a brief dis-cussion of legal issues.

Background Paper #I: Methodological Issues andLiterature Review, includes an in-depth examinationof the decisionmaking context and methodology dis-cussions presented in this report. A critique of the lit-erature, a bibliography of over 600 items, and ab-stracts of over 70 studies and other articles are alsoincluded.

In order to help examine the applicability of tech-niques to assess the costs and benefits of medical

technology, 19 case studies were prepared. All 19 areavailable individually. In addition, 17 of the cases areavailable collectively in a volume entitled Back-ground Paper #2: Case Studies of Medical Technol-ogies. Some of the cases represent formal CEAs (e.g.,the case on bone marrow transplants), and some rep-resent net cost or “least cost” analysis (e. g., the caseon certain respiratory therapies). Other cases illus-trate various issues such as the difficulty of conduct-ing CEA in the absence of adequate efficacy and safe-ty information (e.g., the case on breast cancer sur-gery), or the role and impact of formal analysis onpolicymaking (e.g., the case on end-stage renal dis-ease interventions). The 17 case studies in Back-ground Paper #2 and their authors are:

Artificial HeartDeborah P. LubeckJohn P. Bunker

Automated Multichannel Chemistry AnalyzersMilton C. WeinsteinLaurie A. Pearlman

Bone Marrow TransplantsStuart O. SchweitzerC. C. Scalzi

Breast Cancer SurgeryKaren SchachterDuncan Neuhauser

Cardiac Radionuclide ImagingWilliam B. StasonEric Fortess

Cervical Cancer ScreeningBryan R. Luce

Cimetidine and Peptic Ulcer DiseaseHarvey V. FinebergLaurie A. Pearlman

Colon Cancer ScreeningDavid M. Eddy

CT ScanningJudith L. Wagner

Elective HysterectomyCarol KorenbrotAnn B. FloodMichael HigginsNoralou RoosJohn P. Bunker

End-Stage Renal Disease InterventionsRichard A. Rettig

Gastrointestinal EndoscopyJonathan A. ShowstackSteven A. Schroeder

.-/,

78 . Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

Neonatal Intensive CarePeter BudettiPeggy McManusNancy BarrandLu Ann Heinen Nurse PractitionersLauren LeRoy

Orthopedic Joint Prosthetic ImplantsJudith D. BentkoverPhilip G. Drew

Periodontal Disease InterventionsRichard M. SchefflerSheldon Rovin

Respiratory TherapyRichard M. SchefflerMorgan Delaney

The 18th case study is the present volume, Back-ground Paper #3: The Efficacy and Cost Effectivenessof Psychotherapy, The 19th case study was preparedby Judith Wagner and is published separately asBackground Paper #5: Assessment of Four CommonX-Ray Procedures.

Background Paper #4: The Management of HealthCare Technology in Ten Countries is an analysis ofthe policies, programs, and methods, including cost-effectiveness and cost-benefit techniques, that nineindustrialized nations other than the United Statesuse to manage the effects of medical technology. Theexperience of these nine countries in managing medi-cal technology is compared to that of the UnitedStates. The paper on the United States and the com-parative analysis were prepared by OTA staff, as-

sisted by Louise Russell. The authors of the papers onthe nine foreign countries are:United Kingdom

Barbara StockingCanada

Jack NeedlemanAustralia

Sydney SaxJapan

Joel BroidaFrance

Rebecca FuhrerGermany

Karin A. DumbaughNetherlands

L.M.J. GrootIceland

David GunnarsonDuncan vB. Neuhauser

SwedenErik H. G. GaenslerEgon JonssonDuncan vB. NeuhauserA related report prepared by OTA and reviewed

by the Advisory Panel to the overall assessment is AReview of Selected Federal Vaccine and Immuniza-tion Policies. That study, published in September of1979, examined vaccine research, development, andproduction; vaccine efficacy, safety, and cost effec-tiveness; liability issues; and factors affecting the useof vaccines. Pneumococcal vaccine was used as acase study, and a CEA/CBA was performed.

Appendix B.— Health Program Advisory Committee,Acknowledgments

HEALTH PROGRAM ADVISORY COMMITTEE

Frederick C. Robbins, ChairmanDean, School of Medicine, Case Western Reserve University

Stuart H. AltmanDeanFlorence Heller SchoolBrandeis University

Robert M. BallSenior ScholarInstitute of MedicineNational Academy of Sciences

Lewis H. ButlerHealth Policy ProgramUniversity of California, San Francisco

Kurt DeuschleProfessor of Community MedicineMount Sinai School of Medicine

Zita FearonConsumer Commission on the Accreditation of

Health Services, Inc.

Rashi FeinProfessor of the Economics of MedicineCenter for Community Health and Medical CareHarvard Medical School

Melvin A. GlasserDirectorSocial Security DepartmentUnited Auto Workers

Patricia KingProfessorGeorgetown Law Center

Sidney S. LeeAssociate DeanCommunity MedicineMcGill University

Mark LepperVice President for Znter-institutional AffairsRush-Presbyterian Medical SchoolSt, Luke’s Medical Center

Frederick MostellerProfessor and ChairmanDepartment of BiostatisticsHarvard University

Beverlee MyersDirectorDepartment of Health ServicesState of California

Mitchell RabkinGeneral DirectorBeth Israel Hospital

Kerr L. WhiteRockefeller Foundation

Acknowledgments

Appreciation is expressed to Dr. Brian Yates (American University) and Dr. Frederick Newman (EasternPennsylvania Psychiatric Institute), who provided materials which were useful in preparing portions of thisreport. In addition, thanks are expressed to the following students and staff from Boston University and the Uni-versity of Pittsburgh, who assisted in various aspects of the report’s preparation: Alice Grossman, GaryMenster, Amy Sonka, and Lois Weithorn.

Finally, appreciation is expressed to the individuals listed below, who, along with members of the OTA Ad-visory Panels on CEA and on Psychotherapy, provided reviews of various drafts of the report:Ellen A. Andruzzi Patrick H. DeLeon Charles Kiesler

Psychiatric Nurse Office of Senator Inouye Carnegie-Mellon UniversityArthur H. Auerback David Ehrenfried Daniel Koretz

University of Pennsylvania Blue Cross/Blue Shield Congressional Budget OfficeIrving Chase Michelle Fine Marvin Kristein

National Mental Health Association Columbia University Economic Research Bureau

79

Mack Lipkin, Jr. Peter D. Richman Mary L. SmithRockefeller Foundation Clinical Social Worker University of Colorado

Beryce MacClennan Jeanne M. Safer David UptonGeneral Accounting Office Postgraduate Center for Mental Health Psychiatrist

John C. NemiahBeth Israel Hospital, Boston

References

References

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

Alberts, D. S,, A Plan for Measuring the Per-formance of Social Programs: The Applicationof Operations Research Methodology (NewYork, N. Y.: Praeger Publications, 1970).Alcohol, Drug Abuse, and Mental Health Ad-ministration, National Data Book, ]anuary2980, DHEW publication No. (ADM) 80-938(Washington, D. C.: U.S. Government PrintingOffice, 1980).Allen, K. E., et al., “Effects of Social Reinforce-ment on Isolate Behavior of a Nursery SchoolChild, ” Child Dev. 35:310, 1964.Alley S., et al. (eds. ), Paraprofessionals inMental Health: Theory and Practice (NewYork, N. Y.: Human Sciences Press, 1979).American Medical Association, Physical Dis-tribution and Medica/ Licensure in the U. S,(Chicago, 111.: AMA, 1977).American Psychological Association, EthicalPrinciples in the Conduct of Research With Hu-man Participants (Washington, D. C.: APA,1973).

Standards for Providers of Psycholog-ical Se~ices (Washington, D. C.: APA, 1977).Aron, W, S., and Daily, D., “Short- and Long-Term Therapeutic Communities: A Follow-Upand Cost-Effectiveness Comparison, ” Znt. ],Addic. 9:619, 1974.Attkisson, C. C., et al., “Evaluation: CurrentStrengths and Future Directions, ” in Evaluationof Human Service Programs, edited by C. C.Attkisson, et al. (New York, N. Y.: AcademicPress, 1978).Bandura, A., “On Paradigms and Recycled Ide-ologies,” Cog. Theory & Res. 2:79, 1978.

Principles of Behavior Modification(New York, N. Y.: Holt, Rinehart and Winston,1969. )

Social Learning Theory (EnglewoodCliffs, N. J.: Prentice-Hall, 1977).

, et al., “Relative Effectiveness of Desen-sitization and Modeling Approaches for In-ducing Behavioral, Affective, and AttitudinalChanges,” J, Pers. Soc, Psycho/. 13:173, 1969.Barnhart, G., “Social Design and OperationsResearch,” Pub, Health Rep. 85:247, 1979.Beck, A. T., Diagnosis and Management ofDepression (Philadelphia, Pa.: University ofPennsylvania Press, 1973).Beck, N., “Cost-Benefit and Cost-EffectivenessAnalysis in Psychotherapy Outcome Re-search, ” manuscript submitted for publication,University of Missouri, 1979.

17.

18.

19.

20.

21,

22.

23.

24.

25.

26.

27.

28.

28.

30.

31.

Bergin, A. E., “The Effects of Psychotherapy:Negative Results Revisited, ” J. Colin. Psychol.10:244, 1963.

“An Empirical Analysis of TherapeuticIssues, ‘; in Counseling and Psychotherapy: AnOverview, edited by G. Arbuckle (New York,N. Y.: McGraw Hill, 1967).

, “The Evaluation of Therapeutic Out-comes, ” in Handbook of Psyclzotherapy andBehavior Change, edited by A. E. Bergin and S.L. Garfield (New York, N, Y.: Wiley, 1971).

“Some Implications of PsychotherapyResear;h of Therapeutic Practice, ” ], Abn.Psycho/. 71:235, 1966.

/ and Lambert, M. J., “The Evaluationof Therapeutic Outcomes, ” in Handbook ofPsychotherapy and Behavior Change: An Em-pirical Analysis, edited by S. L. Garfield andA. E. Bergin (New York, N. Y.: WiIey, 1978).Binner, P. R., et al., “Patients, Programs, andResults in a Comprehensive Mental HealthCenter,” ]. Cons, C/in, Psychol. 41:148, 1973.Blanchard, E, B,, and Epstein, L. H., A Bio-feedback Primer (Reading, Pa.: Addison-Wes-ley, 1978).Boudewyns, P. A., “Is ‘Milieu Therapy’ ina Short-Term Inpatient Psychiatric SettingWorth the Money?” Znt. Merit . Health Res.News. 16;7, 1974.Bowman, M. J., “The Costing of Human Re-source Development, ” in The Economics ofEducation, edited by E. A. Robinson and J. E.Vaizey (New York, N. Y.: St, Martins, 1966).Budman, S. H,, et al., “Adult Mental HealthServices in a Health Maintenance Organiza-tion, ” Am. }. Psychi. 136:392, 1979.Bunker, J. P., et al., Costs, Risks, and Benefitsof Surgery (New York, N. Y.: Oxford Universi-ty Press, 1977).Burgess, J., System Approaches to Public Ser-v-ice (Teaneck, N, J.: Fairleigh-Dickenson Uni-versity Press, 1978).Campbell, D. T., and Stanley, J. C., Experi-mental and Quasi-Experimental Designs for Re-search (Chicago, Ill.: Rand McNally, 1963).Carter, D. E., and Newman, F. L., A Client-Oriented System of Mental Health Service De-livery and Program Management: A Work-book and Guide, DHEW publication No.(ADM) 76-307 (RockvilIe, Md.: National Insti-tute of Mental Health, 1976).Cartwright, D. S., “Patient Self-Report Meas-ures, ” in Psychotherapy Change Measures,

83

84 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

32.

33.

34.

35.

36.

37.

38.

39.

40.

41.

42.

43.

44.

45.

DHEW publication No. (ADM) 74-120, editedby I. E. Waskow and M. B. Parloff (Washing-ton, D. C.: DHEW, 1975).Cassell, W. A., et al., “Comparing Costs ofHospital and Community Care,” Hosp,Comm. Psychi, 23:197, 1972.Claiborn, W. L., and Kayton, R., “Analysis ofMental Health Service Delivery: Profession,Patient, Treatment, ” 1. Comm. Psychol. 2:18,1974.

, and Stricker, G,, “Professional Stand-ards Organization, Peer Review, and CHAM-PUS, ” Prof. Psychol. 631, 1979.

and Zaro, J. S., “The Development ofa Peer ‘Review System: The APA/CHAMPUSContract, ” in Psychology and National HealthInsurance: A Sourcebook, edited by C. A.Kiesler, et al. (Washington, D. C.: AmericanPsychological Association, 1979).Cochrane, A. L., Effectiveness and Efficiency:Random Reflections on Health Services (Lon-don: Nuffield Provincial Hospitals Trust,1972).Cohen, J., and Cohen, P., Applied Research/Correlation Analysis for the Behavioral Sci-ences (Hillsdale, N. J.: Lawrence Erlbaum Asso-ciates, 1975).Cohen, M., and Ewalt, P. L,, “An IntensiveProgram for Severely Retarded Children, ” Soc.Casework 56:337, 1975,Cole, S., et al., “Peer Review and the Supportof Science, ” Sci. Am, 237:34, 1977.Conley, R. W., et al., “An Approach to Meas-uring the Cost of Mental Illness, ” Am. /.Psychi. 124:63, 1967.Cook, T. D., and Campbell, D. T., Quasi-Ex-perimentation: Design and Analysis Issues forField Settings (Chicago, Ill.: Rand McNally,1979).Cooper, B. S,, and Rice, D. P., “The EconomicCost of Illness Revisited, ” Soc. Sec. Bull.,39(2):21, 1976.Cooper, E. M., Guidelines for a Minimum Sta-tistical and Accounting System for CommunityMental Health Centers, series C, No. 7 (Rock-ville, Md.: National Institute of Mental Health,1977).Cowen, E, L., “Some Problems in CommunityProgram Evaluation Research, ” ]. Cons. Clin.@cho1. 46:792, 1978.

“Some Problems in Community Pro-gram Evaluation Research, ” }. Cons. Clin.Psychol. 46:792, 1978.

46.

47.

48.

49.

50.

51.

52.

53.

54.

55.

56.

57.

58.

59.

60.

Craighead, W. E., et al., Behavior Modifica-tion: Principles, Issues, and Applications (Bos-ton, Mass.: Houghton Mifflin, 1976).Cronbach, L. J., Designing Evaluations (Stan-ford, Calif.: Stanford Evaluation Consortium,1978).

et al., The Dependability of BehavioralMeasurements: Theory of Generalizability forScores and Profiles (New York, N. Y.: Wileyr

1972).Crow, W. J., “The Effect of Training on Accu-racy and Variability in Interpersonal Percep-tion, ” ]. Abn. Soc. Psychol. 55:355, 1957.Cummings, N. A., “The Health Model as En-tree to the Human Services Model in Psycho-therapy, ” Clin. Psychol. 29:19, 1975.

“Prolonged (Ideal) Versus Short-Term(Realis~ic) Psychotherapy, ” Prof. Psycho/.9:491, 1977.

, and Follette, W. T., “Brief Psychother-apy and Medical Utilization: An Eight-YearFollow-Up, ” in The Professional PsychologistToday: New Developments in Law, Health In-surance, and Health Practice, edited by H.Dorken and Associates (San Francisco, Calif.:Jossey-Bass, 1976).

, and Follette, W. T., “Psychiatric Serv-ices and Medical Utilization in a Prepaid HealthPlan Setting (Pt. 2),” Med. Care. 5:31, 1968.CupIan, R., and Davies, B., “Psychiatric Illnessat a Medical and a Surgical Outpatient Clinic, ”Comp. Psychi. 1:228, 1960.Dahlstrom, W. G., and Welsh, G. S., MMPZHandbook: A Guide to Use in Clinical Practiceand Research (Minneapolis, Minn.: University

of Minnesota Press, 1960).Dahms, W. T., et al., “Treatment of Obesity:Cost-Benefit Assessment of Behavioral Thera-py, Placebo, and Two Anorectic Drugs, ” Am,]. C/in. Nut. 31’:774, 1978.Datel, W. E., and Murphy, J. G., “A Service-Integrating Model for Deinstitutionalization, ”Admin. Merit. He. 3:3s, 197s.Dawes, R. M., “The Robust Beauty of Im-proper Linear Models in Decision Making, ”Am. Psychol. 34:s71, 1979.Delaney, J. A., et al., “Crisis Intervention andthe Prevention of Institutionalization: An In-terrupted Time Series Analysis, ” Am, ].Comm. Psychol, 6:33, 1978.DeLeon, P. H., and VandenBos, G. R., “Politi-cal Factors Related to the Coverage of MentalServices in Federal Programs: Implications for

References ● 85

61.

62.

63.

64.

65.

66.

67.

68.

69.

70.

71.

72.

73.

74.

75.

76.

77.

the Reimbursement of Psychotherapy, ” in Psy-chotherapy: From Practice to Research in Pol-icy, edited by G. R. VandenBos (Beverly Hills,Calif.: Sage Publications, in press).Denker, P. G., “Results of Treatment of Psy-choneuroses by the General Practitioners, ”N. Y. State ]. Med. 46:2164, 1946.Department of Health, Education, and Wel-fare, National Health Survey, Runaway Youth:A Status Report and Summary of Projects(Washington, D. C.: DHEW, 1976).Diagnostic and Statistical Manual of MentalDisorders (3d cd., Washington, D. C.: Amer-ican Psychiatric Association, 1980).Dorken, H., “CHAMPUS Ten-State Claim Ex-perience for Mental Disorders: Fiscal Year197s, ” in Psychology and National Health in-surance: A Sourcebook, ed i ted by C . A .KiesIer, et al. (Washington, D. C.: AmericanPsychological Association, 1979).Durlak, J. A., “Comparative Effectiveness ofParaprofessional and Professional Helpers, ”Psychol. Bull. 36:80, 1979.Edwards, A. L., and Cronback, L. J., “Experi-mental Design for Research in Psychotherapy, ”J. Clin. Psychol. 8:51, 1952.Endicott, J., et al., “The Global AssessmentScale, ” Arch. Gen. Psychi. 33:766, 1976.Epstein, S., “The Stability of Behavior: I. OnPredicting Most of the People Much of theTime,” ]. Per. Soc. Psychol. 37:1097, 1979.Eysenck, H. F., The Effects of Psychotherapy(New York, N. Y.: International Science Press,1966).

“The Effects of Psychotherapy, ” inHandb;ok of Abnormal Psychology, edited byH. J. Eysenck (New York, N. Y.: Basic BooksInc., 1961).

“The Effects of Psychotherapy, ” Int. }.Psychi.’ 1:97, 1965.

“The Effects of Psychotherapy: AnEvalua~ion, ” ). Cons, Psychol. 16:319, 1952.

“An Exercise in Megasilliness, ” Am.Psycho’l. 33:517, 1978.

I “Note on ‘Factors Influencing the Out-come of Psychotherapy ’,” Psychol. Bull. 78:403, 1972.Fairweather, G. W., et al., Community Life forthe Mentally 11/: An Alternative to InstitutionalCare (Chicago, Ill.: Aldine, 1969.Fein, R., Economics of Mental Zlbzess (NewYork, N. Y.: Basic Books, 1958.Fishman, D. B., A Cost Effectiveness Method-

78.

79.

80.

81.

82.

83.

84.

85,

86.

87.

88.

89.

90.

91.

92.

ology for Rational Decision-Making in MentalHealth Services, paper presented at the Meetingof the American Psychological Association,San Francisco, Calif., Aug. 26, 1977.

“Development of a Generic Cost-Effec-tivenes~ Methodology for Evaluating the Pa-tient Services of a Community Mental HealthCenter, ” in Evaluation in Alcohol, DrugAbuse, and Mental Health Service Programs,edited by J. Zusman and C. R. Wurster (Lex-ington, Mass.: D. C. Heath and Co., 1975).Foreyt, J. P., et aI., “Benefit-Cost Analysis of aToken Economy, “ Prof. Psychol. 6:26, 1975.Fox, H., “Toward an Understanding of Opera-tions Research Concepts, ” Mgt. Sem, 7:23,1970.Fox P, D., and Kuldau, J. M., “Expanding theFramework for Mental Health Program Evalu-ation, ” Arch. Gen. Psychi, 19:538, 1968.Foxx, R. M., and Azrin, N. H., “Dry Pants: ARapid Method of Toilet Training Children, ”Behav. Res. Ther. 11:43s, 1973.Frank, J. D., “Methods of Assessing the Resultsof Psychotherapy, ” in Ciba Foundation Sym-posium on the Role of Learning in Psychother-apy, edited by R. Porter (Boston, Mass.: LittleBrown and Co., 1968).

Persuasion and Healing (2d cd., Balti-more, Md.: Johns Hopkins, 1973).

, “The Present Status of Outcome Stud-ies,” 1, Cons, Clin. Psychol. 47:31o, 1979.

“Therapeutic Components of Psycho-therap~, ” ]. Nerv. Merit. Dis. 159:325, 1974.Frankl, V. E., “Basic Concepts of Logother-apy, ” Confina Psychiatrical, 4:99, 1961.Freud, S., Standard Edition of the CompletePsychological Works of Sigmund Freud, editedand translated by J. Strachey (vol. 17, London:Hogarth, I$IS.S). (Originally published, 1918. )Gabby, J. I., and Leavitt, A., “Providing LowCost Therapy to Middle Income Patients, ”Comm. Merit. Health]. 6:210, 1970.Gallo, P. S., Jr., “Meta-Analysis-A MixedMeta-Phor?” Am. Psychol. 33:515, 1978.Garfield, S. L., “Research on Client Variablesin Psychotherapy, ” in Handbook of Psycho-therapy and Behavior Change: An EmpiricalAnalysis, edited by S. L. Garfield and A. E.Bergin (2d cd., New York, N. Y.: Wiley, 1978).

and Bergin, A. E. (eds. ), Handbook ofPsycho’therapy and Behavior Change: An Em-pirical Analysis (2d cd., New York, N. Y.:Wiley, 1978).

86 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

93.

94.

95.

96.

97.

98.

99.

100.

101.

102.

103.

104.

105.

106.

, and Kurtz, R., “A Survey of ClinicalPsychologists: Characteristics, Activities, andOrientation,” C]irz. Psycho]. 28:7, 1974.Gilbert, J. P., et al., “Assessing Social Innova-tions: An Empirical Basis for Policy, ” in Evacu-ation and Exper-irnent, edited by Bennett andLumsdaine (New York, N. Y.: Academic Press,1975) .Ginsberg, G., and Marks, L., “Costs and Ben-efits of Behavioral Psychotherapy: A PilotStudy of Neurotics Treated by Nurse-Thera-pists, ” Psychol. Med. 7:685, 1977.Glasgow, R. E., and Rosen, G. M., “BehavioralBibliotherapy: A Review of Self-Help BehaviorTherapy Manuals, “ Psychol. Bull. 85:1, 1978.Glass, G. V., et al., Design and Analysis ofTirne-Ser-ies Experiments (Boulder, Colo.: Lab-oratory of Educational Research Press, 1973).Glass, M. G., and Goldberg, S., “Cost-BenefitAnalysis and the Evaluation of PsychiatricServices,” Psycho/, Med. 7:701, 1977.Goldberg, I., et al., “Effect of a Short-TermOutpatient Psychiatric Therapy Benefit on theUtilization of Medical Services in a PrepaidPractice Medical Program, ” Med. Care 8:423,1970.Goldensohn, S. S., and Fink, R., “MentalHealth Services for Medicaid Enrollees in a Pre-paid Group Practice Plan, ” Am. ]. Psychi. 136:160, 1979.Goldschmidt, P. G., “A Cost-EffectivenessModel for Evaluating Health Care Programs:Application to Drug Abuse Treatment, ” in-quiry 13:29, 1976.Goodwin, I., and Rosenblum, A., “A Methodof Measuring and Comparing Cost in MentalHealth Clinics, ” H o s p . Comm. Psychi. 23:47,1972.Gottfredson, G. D., and Dyer, S. E., “HealthService Providers in Psychology, ” Am. Psychi.33:314, 1978.Gottman, J. M., and Leiblum, S. R., How ToDo Psychotherapy and How To Evaluate It(New York, N. Y.: Holt, Rinehart and Winston,1974).

and Markman, H. J., “ExperimentalDesign; in Psychotherapy Research, ” in Hand-book of Psychotherapy and Behavior Change:An Empirical AnaZysis, edited by S. L. Garfieldand A. E. Bergin (2d cd., New York, N. Y.:Wiley, 1978).Gottschalk, L. A., et al., “Prediction and Eval-uation of Outcome in an Emergency Brief Psy-chotherapy Clinic, ” ]. Nerv. Merit, Dis. 144:77,1967.

107.

108.

109.

110.

111.

112.

113.

114.

115.

116.

117.

118.

119.

120.

Grace, W. J., et al., “The Treatment of Ulcera-tive Colitis: II, ” Gastroent, 26:462, 1954.Green, B. L., et al., “Relationships AmongDiverse Relationships of Psychotherapy Out-come, ” J. Con, Clin, Psychol, 43:689, 1975.Green, P. E., and Carmone, F. J., Multidimen-sional Scaling and Related Techniques in Mar-keting Analysis (Boston, Mass.: Allyn andBacon, 1970.Gruber, A. R., “The High Cost of DeliveringServices, ” Soc. Work 18:33, 1973.Guzzetta, R. A., “Acquisition and Transfer ofEmpathy by the Parents of Early AdolescentsThrough Structured Learning Training, ” ].Coun. Psychol. 23:449, 1976.Hall, S. M., et al., “Contingency Managementand Information Feedback in Outpatient Hero-in Detoxification, “ Behav. Ther. 10:443, 1979.Halpern, J . , “Program Evaluation, SystemsTheory, and Output Value Analysis: A Bene-fit/Cost Model,” in Program Evaluation forMental Health, edited by R. D. Corusey (NewYork, N. Y.: Grune and Stratton, 1977).

and Binner, J. R., “A Model for anOutput’ Value Analysis of Mental Health Pro-grams,” Admin. Merit, He. 1:40, 1972.Halpert, H., “Models for the Application ofSystems Analysis to the Delivery of MentalHealth Services, ” in Community Psychologyand Community Mental Health, edited by D.Cooks (San Francisco, Calif.: Holden-Day,1970).Hargreaves, W. C., et al., Resource Materialsfor Community Mental Health Program Evalu-ation: Part 3—Evaluating Effectiveness of Serv-ices, DHEW publication No. (ADM) 75-222(Rockville, Md.: National Institute of MentalHealth, 1975).Hiebert, S., “Who Benefits From the Pro-gram?” in Evaluation of Behavioral Programs:in Community, Residential, and School Set-tings, edited by P. O. Davison, et al. (Cham-paign, 111.: Research Press, 1974).Hogan, D. B., The Regulation of Psychothera-pists: A Study in the Philosophy and Practiceof Professional Regulations (vols. 1-111, Cam-bridge, Mass.: Ballinger, 1979).Holden, C,, “Senators Hear Case for Psycho-therapy, ” Science 201:794, 1978.Hollon, S., and Beck, A. T., “Psychotherapyand Drug Therapy: Comparisons and Combi-nations, ” in Handbook of Psychotherapy andBehavior Change: An Empirical Analysis,edited by S. L. Garfield and A. E. Bergin (2dcd., New York, N. Y.: Wiley, 1978).

121.

122.

123.

124.

125.

126.

127.

128.

129.

130.

131.

132.

133.

134.

135.

136,

Hospital Statistics (Chicago, Ill.: AmericanHospital Association, 1979).Hupmans, J. H., The lmplernentation of Oper-ations Research (New York, N. Y.: Wiley,1970) .Imber, S. D., “Patient Direct Self-Report Tech-niques, ” in Psyc]zotherapy Change Measures,edited by 1. E. Waskow and M. B. Parloff,DHEW publication No. (ADM) 74-120 (Wash-ington, D. C.: DHEW 1975).James, L. R., and Jones, A. P., “OrganizationalClimate: A Review of Theory and Research, ”Psychol. BL{ll. 81:1096, 1974.Jameson, J., et al., “The Effects of OutpatientPsychiatric Utilization on the Costs of Pro-viding Third-Party Coverage, ” Research Series28, Blue Cross of Western Pennsylvania, 1976.Jeffrey, D. B., “Treatment Evaluation Issues inResearch on Addictive Behaviors, ” Addic.Behav. 1:23, 1975.Jeffrey, R. W., et al., “Behavioral Treatment ofObesity: The State of the Art, 1976,” Behau.T{ler. 9:189, 1978.Joint Commission on Accreditation of Hospi-tals, Principles for Accreditation of Communi-ty Mental Health Service Programs (Chicago,Ill.: JCAH, 1976).Jones, K., and Vischi, T., “Impact of Alcohol,Drug Abuse, and Mental Health Treatment onMedical Care Utilization: Review of the Re-search Literature, ” Med. Care (suppl. ) 17:12,1979.

, et al., “Cost and Efficiency in MentalHospitals, ” The Hospital 57:23, 1961.Jung, C., Collected Works, edited by H. Read,et al. (New York, N. Y.: Pantheon, 1953).Karen, B. P., and VandenBos, G. R., “Cost/Benefit Analysis: Psychologist Versus Psychia-trist for Schizophrenics, ” Prof. Psycho/. 7:107,1976.

, and VandenBos, G. R., “TreatmentCosts of Psychotherapy Versus Medication forSchizophrenics, ” Prof. Psycho/. 6:293, 1975.Karush, A., et al., “The Response to Psycho-therapy in Chronic Ulcerative Colitis. I. Pre-treatment Factors, ” Psychosorn, Med. 30:225,1968.Kazdin, A. E,, “Evaluating the Generality ofFindings in Analogue Therapy Research, ” ].Cons. Clin. Psychol. 46:673, 1978.

“Therapy Outcome Questions Requir-ing Co~trol of Credibility and Treatment-Gen-erated Expectancies, ” Behav. Ther. 10:81,1979.

137.

138.

139.

140.

141.

142.

143,

144,

145.

146.

147.

148.

149.

150.

151.

152.

Kelly, G., The Psychology of Personal Con-structs (New York, N. Y.: Norton, 1955).Kennecott Cooper Corp., Insight, unpublishedreport (Salt Lake City, Utah: Utah CopperDivision, 1975).Kiresuk, T. J., and Lung, S. H., “Goal Attain-ment Scaling, “ in Evaluation of Human ServicePrograms, edited by C. C. Attkisson, et al,(New York, N. Y.: Academic Press, 1978).Klasman, H. E., “Application of Cost-BenefitAnalysis to Health Systems Technology, ” ].Occup. Med. 16:172, 1974,Klein, D. F., “A Proposed Definition of MentalIllness, ” in Critical Issues in Psychiatric Diag-nosis, edited by R. L. Spitzer and D. F. Klein(New York, N. Y.: Raven Press, 1978).Klerman, G. L., et al., “Treatment of Depres-sion by Drugs and Psychotherapy, ” Am. ].Psychi. 131:186, 1974.Knapp, T. J., and Peterson, L. W., “BehaviorManagement in Medical and Nursing Practice, ”in Behavior Modification: Principles, Issues,and Applications, edited by W. E. Craighead,et al. (Boston, Mass.: Houghton Mifflin, 1976).Kolb, L. C., Modern Clinical Psychiatry (Phil-adelphia, Pa.: Saunders, 1968).

Noyes’ Clinical Psychiatry (7th cd.,Philad~lphia, Pa,: Saunders, 1968).Krantz, D., et al., Foundation of Measurement(vol. 1, New York, N. Y.: Academic Press,1972).Krapfl, J. E., “Accountability Through Cost-Benefit Analysis, ” in Behavior Analysis andSystems Analysis: An Integrative Approach toMental Health Programs, edited by D. Harsh-barger and R, F. Maley (Kalamazoo, Mich.:Behaviordelia, 1974).Krowinski, W. F., and Fitt, D. X., “Day Treat-ment vs. Brief Hospitalization: A ControlledStudy of Clinical and Cost Effectiveness, ” Am,). Comm. Psycho/., in press.

, and Fitt, D. X., “A Model for Evaluat-ing Mental Health Programs: The FunctionalBaseline System, ” Admin. Merit, He. 6:22,1978.Krumboltz, J. D., “An Accountability Modelfor Counselors, ” Pers. Guid. ]. 52:639, 1974.Lachenmeyer, C. W., “Experimentation-AMisunderstood Methodology in Psychologicaland Socio-Psychological Research, ” Am. Psy -chol. 25:617, 1970.Lalonde, M., A New Perspective on the Healthof Canadians (Ottawa, Ontario: Governmentof Canada, 1974).

d

88 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

153.

154.

155.

156.

157.

158.

159.

160.

161.

162.

163.

164.

165.

166.

167.

168.

169.

Landis, C., “A Statistical Evaluation of Psy-chotherapeutic Methods, ” in Concepts andProblems of Psychotherapy, edited by L. E.Hinsie (New York, N. Y.: Columbia, 1937).Lazarus, A., Behavior Therapy and Beyond(New York, N. Y.: McGraw Hill, 1972).

“Group Therapy of Phobic Disordersby Sys~ematic Desensitization, ” ], Abn. SocialPsychol. 63:504, 1961.Levenson, A. J., et al., “Acute Schizophrenia:An Efficacious Outpatient Treatment Ap-proach as an Alternative to Full-Time Hospi-talization, ” Dist. New. Sys, 38:242, 1977.Levin, H. M., “Cost-Effectiveness Analysis inEvaluation Research, ” in Handbook of Evalua-tion Research, edited by M. Guttentag and E.Struening (vol. 2, Beverly Hills, Calif.: SagePublications, 1975).Levine, D. S., and Winner, S. G., The Cost ofMental Illness, 2974, DHEW publication No.(ADM) 76-158 (Rockville, Md.: National Insti-tute of Mental Health, 1976).Liptzin, B., et al., “A Federal View of MentalHealth Program Evaluation, ” Prof. Psycho/.8:543, 1977.Luborsky, L., “A Note on Eysenck’s Article‘The Effects of Psychotherapy: An Evalua-tion’,” Bri, ]. Psychi. 45:129, 1954.

, and Bachrach, H., “Factors InfluencingClinical Judgment of Mental Health,” Arch.Gen. Psychi. 31:292, 1974.

, et al., “Comparative Studies of Psy-chotherapies: Is It True That ‘Everybody HasWon and All Must Have Prizes’?” Arch, Gen,Psychi. 32:995, 1975.Macklin, R,, “The Medical Model in Psycho-analysis and Psychotherapy, ” Comp, Psychi.14:49, 1973.Mahoney, M. J., Cognition and Behavior Mod-ification (Cambridge, Mass.: Ballinger, 1976).

“Experimental Methods and OutcomeEvalua~ion,” ]. Cons. C/in. Psycho/. 46:660,1978.

“The Self-Management of Covert Be-havior: A Case Study,” Behav. Ther. 2:575,1971.Maidlow, S. T., and Berman, H., “The Eco-nomics of Heroin Treatment, ” Am. ]. Pub.Health 62:1397, 1972,Malan, D. H., “The Outcome Problem in Psy-chotherapy Research: A Historical Review, ”Arch. Gen. Psychi, 29:719, 1973.Marks, I. M., “Empirical PsychotherapeuticMethods,” Psychoth. & Psychosom. 24:222,1974.

170,

171.

172.

173.

174.

175.

176.

177.

178.

179.

180.

181.

182.

183.

184.

185.

Marston, A. R., et al., “A CorrespondenceCourse Behavioral Program for Weight Reduc-tion, ” Obesity/Bariatric Med. 6:140, 1977.Maslow, A. H., The Farther Reaches of HumanNature (New York, N. Y.: Viking, 1971).Mason, P, E,, and Stitham, M. D., “The Ex-pensive Dalliance: Assessing the Cost of Pa-tient-Therapist Sex, ” Bull. Am. Acad. Psychi.Law 5:450, 1977.May, P. R., “Cost Efficiency of Treatments forthe Schizophrenic Patient, ” Am, ], Psychi.127:118, 1971.Mazer, M., “Psychiatric Disorders in the Gen-eral Practices of an Island, ” Med. Care 7:372,1969.McCaffree, K. M., “The Cost of Mental HealthCare Under Changing Treatment Methods,”Am, ], Pub. Health 56:1013, 1966.McCardell, J., and Murray, E. J., “NonspecificFactors in Weekend Encounter Groups, ” ].Cons. Clin, Psychol. 42:337,1974.McClean, P. D., and Hakstian, A. R., “ClinicalDepression: Comparative Efficacy of Outpa-tient Treatments, ” ]. Cons. Clin. Psychol., inpress.McCord, J., “A Thirty-Year Follow-Up ofTreatment Effects, ” Am. Psycho/. 33:284,1978.McGuire, T., National Health Insurance andIssues and Evidence (Cambridge, Mass.: Ball-inger, forthcoming).Meichenbaum, D., Cognitive Behavior Modifi-cation: An Integrative Approach (New York,N. Y.: Plenum Press, 1977).Meltzoff, J., and Kornreich, M., Research inPsychotherapy (New York, N. Y,: Atherton,1970).Mendelssohn, F., “Long-Term PsychotherapyCoverage Under National Health Insurance, ”Soc. Sec. Bull. 18:1, 1976.Mills, D. H,, et al., “The National RegisterSurvey: The First Comprehensive Study of AllLicensed/Certified Psychologists, ” in Psychol-ogy and National Health Insurance: A Source-book, edited by C. A. Kiesler, et al. (Washing-ton, D. C.: American Psychological Associa-tion, 1979).Mintz, J., et al., “Measuring the Outcomes ofPsychotherapy: Findings of the Penn Psycho-therapy Project, ” ~, Cons. Clin, Psychol.47:319, 1979.Mischel, W., Introduction to Personality (2dcd., New York, N. Y.: HoIt, Rinehart and Win-ston, 1976).

References ● 89

186.

187.

188.

189.

190.

191.

192,

193.

194.

195.

196.

197.

198.

199.

200.

201.

Personality and Assessment (NewYork: Wiley, 1968).

“Toward a Cognitive Social LearningReconc’eptualization of Personality, ” Psychol.Rev, 80:252, 1973.Moos, R., Coping With Physical Illness (NewYork, N. Y.: Plenum Press, 1977).Morganstern, K. P., “Cigarette Smoking as aNoxious Stimulus in Self-Managed AversionTherapy for Compulsive Eating: Techniqueand Case Illustration, ” Behav. Ther. 5:255,1974.Morton, R. B., “An Experiment in Brief Psy-chotherapy,” Psych. Monogr. 69(386):1, 1955.Mosher, L. R., et al., “Soteria: Evaluation of aHome-Based Treatment for Schizophrenia, ”Am. J. Orthopsychi. 4s:4ss, 197s.Murphy, J. G., and Datel, W. E., “A Cost-Benefit Analysis of Community Versus Institu-tional Living, ” H o s p . Comm. Psychi. 27:165,1976.Nagel, S. S., and Neef, M., Operations Re-search Methods (Beverly Hills, Calif.: SagePublications, 1976).National Health Insurance Resource Book,House Committee on Ways and Means (revisededition, Washington, D. C.: U.S. GovernmentPrinting Office, 1976).National Institute of Drug Abuse, ManagementEffectiveness Measures for NZZ3A Drug AbuseTreatment Programs, Volume 1: Cost-BenefitAnalysis, Volume 2: Costs to Society of DrugAbuse (Washington, D. C.: NIDA, 1977).National Institute of Mental Health, Draft Re-port: The Financing, Utilization, and Qualityof Mental Health Care in the United States(Washington, D. C.: DHEW, Office of ProgramDevelopment and Analysis, 1976).

, Mental Health Statistical Note No.239, August 1977.Newman, F. L., and Rinkus, A. J., “Level ofFunctioning, Clinical Judgment, and MentalHealth Services, ” Eval. & the H e a l t h Prof.1:175, 1978.

, et al., “Program Analysis Using theClient-Oriented Cost Outcome System, ” Eval.& Z%og. P!ng. 1:19, 1978.Noble, J. H., Jr., “The Limit of Cost-BenefitAnalysis as a Guide to Priority-Setting in Reha-bilitation,” Eval. Qtrly. 1:347, 1977.Nunnally, J, C,, and Wilson, W. H., “Methodand Theory for Developing Measures in Eval-uation Research, “ in Handbook of EvaluationResearch, edited by E. L. Streuning and M.

202.

203.

204.

205.

206.

207.

208.

209.

210.

211.

212.

213.

214.

215.

Gutenberg (vol. 1, Beverly Hills, Calif.: SagePublications, 197s).Office of Technology Assessment, U.S. Con-gress, Assessing the Efficacy and Safety ofMedical Technologies, GPO stock No. 052-003-00593-0 (Washington, D. C.: U.S. Govern-ment Printing Office, September 1978).

U.S. Congress, The Implications ofCost-E~fectiveness A n a l y s i s o f M e d i c a lTechnology, GPO stock No. 052-003 -00765-7(Washington, D. C.: U.S. Government PrintingOffice, August 1980).Olbrisch, M. E., “Psychotherapeutic Interven-tions in Physical Health: Effectiveness and Eco-nomic Efficiency,“ Am. Psychol. 32:761, 1977.O’Neil, H. F., et al., “A Systems Analysis of aCampus Community Mental Health Facility: APreliminary Investigation, ” Am. 1. Comm.Psychol. 1:362, 1973.Panzetta, A. F., “Cost-Benefit/Studies in Psy-chiatry,” Comp. Psychi. 14:451, 1973.Parloff, M. B., “Can Psychotherapy ResearchGuide the Policymaker? A Little KnowledgeMay Be a Dangerous Thing, ” Am. Psycho/,34:296, 1979.

, et al., “Assessment of PsychosocialTreatment of Mental Health Disorders: Cur-rent Status and Prospects, ” report to the Na-tional Academy of Sciences, Institute ofMedicine, Washington, D. C., 1978.Patterson, G. R., “Intervention for Boys WithConduct Problems: Multiple Settings, Treat-ments, and Criteria, ” ]. Cons. Clin. Psychol.42:471, 1974.Payne, F. D., and Wiggins, J. S., “MMPI Pro-file Types and the Self-Report of PsychiatricPatients,” J. Abn. Psychol. 79:1, 1972.Perkins, J. A., and Wicas, E., “Group Counsel-ing With Bright Underachievers and TheirMothers,” ). Coun, Psychol. 18:273, 1971.Perloff, E. and Perloff, R., “Selected Processesfor Evaluating Service Delivery Programs:Overview,” Prof. Psychol, 8:389, 1977.Phillips, E. L., “Achievement Place: Token Re-inforcement Procedures in a Home-Style Reha-bilitation Setting for ‘Pre-Delinquent’ Boys, ” ].Appl. Behav. Anal, 1:213, 1968.Piasecki, J. R., et al., Determining the Costs ofCommunity Residential Services for the Psy-chosocially Disabled, DHEW publication No.(ADM) 7-5041 (Rockville, Md.: National In-stitute of Mental Health, 1977).Pomerleau, O. F., “Behavioral Medicine: TheContributions of the Experimental Analysis of

90 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

216.

217.

218.

219.

220.

221.

222.

223.

224.

225.

226.

227.

228.

229.

230.

231.

232.

Behavior to Medical Care, ” Am. Psychol. 34:654, 1979.Potter, A., et al., “Readmission Discount Fac-tors in Program Evaluation: An Output ValueAnalysis of an Adult Psychiatry Program, ”Am. ]. Comm. Psychol. 3:3o3, 197s.Powers, E., and Witmer, H., An Experiment inthe Prevention of Delinquency (New York,N. Y.: Columbia University Press, 1951).Presby, S., “Overly Broad Categories ObscureImportant Differences Between Therapies, ”Am. Psychol. 33:514, 1978.President’s Commission on Mental Health, Re-port to the President, GPO stock No. 040-000-00390-8 (vol. 1, Washington, D. C.: U.S.Government Printing Office, 1978).Prest, A. R., and Turvey, R., “Cost-BenefitAnalysis: A Survey, “ Econ, ]. 75:683, 1965.Rachman, S., “Double Standards and SingleStandards, ” Bull, Brit. Psycho/. Soc. 30:295,1977.

“The Effects of Psychological Treat-merit, ” ‘in Handbook of Abnormal Psychology,edited by H. Eysenck (New York, N. Y.: BasicBooks, 1973).

The Effects of Psychotherapy (Oxford,England: Pergamon Press, 1971).Rathus, S. A., and Nevid, J. S., BehaviorTherapy: Strategies for Solving Problems inLiving (New York, N. Y.: Signet, 1977).Regier, D. A,, et al,, “The De Facto U.S. Men-tal Health Services System, ” Arch. Gen.Psychi. 35:685, 1978.Reiss, B., “Changes in Patient Income Concom-itant With Psychotherapy, ” ]. Cons. Psychol.31:130, 1967.Reiss, M. L., et al., “Behavioral CommunityPsychology: Encouraging Low-Income Parentsto Seek Dental Care for Their Children, ” ].Appl. Behav. Anal. 9:387, 1976.Rice, D. P., Estimating the Cost of Mental Ill-ness, Health Economics Series No. 5 (Washing-ton, D. C.: DHEW, 1966).Riecken, H. W., “Principal Components of theEvaluation Process, ” Prof. Psycho/. 8:392,1977.

and Boruch, R. F. (eds. ), Social Experi-mentat;”on: A Method for Planning and Eval-uating Social Intervention (New York, N. Y.:Academic Press, 1974),Rivlin, A. M., Systematic Thinking for SocialAction (Washington, D. C.: The Brookings In-stitution, 1971).

t and Timpane, P. M. (eds. ), Ethical and

233.

234,

235,

236.

237,

238.

239.

240.

241.

242.

243.

244.

245.

246.

247.

Legal Issues of Social Experimentation (Wash-ington, D. C.: The Brookings Institution, 1975).Robins, L. N., Deviant Children Grownup(Baltimore, Md.: Williams and Wilkins, 1966).Rogers, C. R., Client-Centered Therapy: ZtsCurrent Practice, Implications, and Theory(Boston, Mass,: Houghton Mifflin, 19s1).

“Client Oriented Psychotherapy, ” inComprehensive Textbook of Psychiatry, editedby A. M. Freedman, et al. (vol. 1, 2d cd., Bal-timore, Md.: Wilkins and Wilkins, 1975).

On Personal Power: Inner Strengthand Zt; Revolutionary Impact (New Y o r k ,N. Y.: Delacorte, 1977).Remans, J. T., “The Economic Evaluation ofMental Health Programs, ” Znt. ], Merit. Health2:38, 1973.Rosen, J. C., and Wiens, A. N., “Changes inMedical Problems and Use of Medical ServicesFollowing Psychological Intervention, ” Am,Psycho/. 34:420, 1979.Rosenthal, R., “Combining Results of Inde-pendent Studies, “ Psychol. Bull. 85:185, 1978.

Experimenter Effects in Behavioral Re-search ‘(New York, N. Y.: Appleton-Century-Crofts, 1966).

, “The ‘Filedrawer Problem, ’ and Toler-ance for Null Results, ” Psychol. Bull. 86:638,1979.Rosenzweig, S., “A Transvaluation of Psycho-therapy—A Reply to Hans Eysenck, ” J, Abn.Soc. Psycho/. 49:298, 1954.Rubenstein, J. S., et al., “The Parent-TherapistProgram: Alternative Care for EmotionallyDisturbed Children, ” A m . ], Orthopsychi.48;654, 1978.Rufener, B. L., et al., Management Effective-ness Measures for NIDA Drug Abuse Treat-ment Programs, Volume 1: Cost Benefit Anal-ysis, GPO stuck No, 017-024-00577-1 (Wash-ington, D. C.: National Institute of DrugAbuse, 1977).

, et al., Management EffectivenessMeasures for NIDA Drug Abuse TreatmentPrograms, Volume 2: Costs to Society ofDrug Abuse, GPO stock No. 017-024-00578-9(Washington, D. C.: National Institute of DrugAbuse, 1977).Rush, A. J., et al., “Comparative Efficacy ofCognitive Therapy and Pharmacotherapy inthe Treatment of Depressed Outpatients, ” Cog,Ther, Res. 1:17, 1977.Rutledge, L., and Binner, P. R., “Readmissionto a Community Mental Health Center, ”Comm. Merit, Health J. 6:136, 1970.

References ● 91

248.

249.

250.

251.

252.

253.

254.

255.

256.

257.

258.

259.

260.

261.

262.

Salsbery, D. L., Accounting Guidelines forCommunity Mental Health Centers and Re-lated Facilities, DHEW publication No. (HMS)73-9068 (Rockville, Md.: National Institute ofMental Health, 1972).Saxe, L., and Fine, M., Social Experiments:Methods for Design and Evaluation (BeverlyHills, Calif.: Sage Publications, forthcoming).Scheff, T. J., Being Mentally 111: A SociologicalTheory (Chicago, Ill.: Adeline, 1966).Schelling, T. C., “The Life You Save May BeYour Own, ” in Problems in Public Expendi-tures, edited by S. B. Chase, Jr. (Washington,D. C.: The Brookings Institution, 1968).Schulberg, H, C., “Issues in the Evaluation ofCommunity Mental Health Programs, ” Prof.Psychol. 8:560, 1977.

“Quality-of-Care Standards and Pro-fession~l Norms, ” Am. ], Psychol. 133:1047,1976.Seres, J. L., and Newman, R. I., “Results ofTreatment of Chronic Low-Back Pain at thePortland Pain Center, ” ]. Neuro. 45:32, 1976.Shanteau, J., and Phelps, R. H., “Judgment andSwine: Approaches and Issues in Applied Judg-ment Analysis,“ in Human Judgment and Deci-sion Processes: Applications in Problem Set-tings, edited by M. F. Caplan and S. Schwartz(New York, N. Y.: Academic Press, 1977).Shapiro, A. K., and Morris, L. A., “The Place-bo Effect in Medical and Psychological Thera-pies, ” in Handbook of Psychotherapy and l?e-havior Change: An Empirical Analysis, editedby S. L. Garfield and D. E. Bergin (2d cd., NewYork, N. Y.: Wiley, 1978).

Shapiro, D., and Shapiro, D., “The ‘DoubleStandard’ in Evaluation of Psychotherapies, ”Bull. l?rit. Psychol, Soc. 30:209, 1977.Sharfstein, S. S., “Third-Party Payers: To Payor Not To Pay, ” A m , J, Psychi. 135:1185,1978.

and Nafziger, J. C., “CommunityCare: Costs and Benefits for a Chronic Pa-tient, ” Hosp. Comm. Psychi. 27:170, 1976.Shephard, R. N., et al. (eds. ), Multidimen-sional Scaling: Theory and Application in the13ehavioral Sciences (vols. I and 2, New York,N. Y.: Academic Press, 1972).Siegert, F., and Yates, B. T., “Behavioral-ChildManagement: A Comparison of Individual In-Office, Individual In-Home, and Group Deliv-ery Systems, ” Eual. & the Health Prof. 3:123,1980.Sirotnik, K. A., and Bailey, R. C., “A Cost-Benefit Analysis for a Multimodality Heroin

263,

264.

265.

266.

267.

268.

269.

270.

271.

272.

273.

274.

275.

276.

277.

278.

279.

Treatment Project, ” lnt. ]. A d d i c t . 10:443,1975.Skinner, B. F., About Behaviorism (New York,N, Y.: Knopf, 1974).

Beyond Freedom and Dignity (NewYork, N. Y.: Knopf, 1971).

Science and Human Behavior (NewYork, N. Y.: MacMillan, 1953).Sloane, R. B., et al., Psychotherapy VersusBehavior Therapy (Cambridge, Mass.: Har-vard University Press, 1975).Smith, M. L., and Glass, G. V., “Meta-Analy -sis of Psychotherapy Outcome Studies, ” Am.Psychol. 32:752, 1977.

et aI., The Benefits of Psychotherapy(Baltimore, Md.: Johns Hopkins UniversityPress, in press).Sobel, S. B., “Throwing the Baby Out With theBathwater: The Hazards of Follow-Up Re-search, ” Am. Psychol. 33:290, 1978,Sorenson, J. E., and Grove, H. D., “UsingCost-Outcome and Cost-Effectiveness Analy-ses for Improved Program Management andAccountability, ” in Evaluation of Human Seru-ice Program, edited by C. C. Attkisson, et al.(New York, N. Y.: Academic Press, 1978).

and Phipps, D. W., “Cost-Finding: ATool fir Managing Your Community MentalHealth Center,” Admin. Merit, He. 1:68, 1972.

and Phipps, D. W., Cost-Finding andRate-S; tting for Corr,munity Mental HealthCenters, DHEW publication No, (ADM)76-291 (Rockville, Md.: National Institute ofMental Health, 1975),Stern, B. E., and Stern, E. S., “Efficiency ofMental Hospitals, ” Brit. ]. Prezl. & Soc. Med.17:111, 1963.Strupp, H. H., “Psychotherapy Research andPractice: An Overview, ” in Handbook of Psy-chotherapy and Behavior Change: An Empiri-cal Analysis, edited by S. L. Garfield and A. E.Bergin (2d cd., New York, N. Y.: Wiley, 1978).

, “Success and Failure in Time-LimitedPsychotherapy, ” Arch. Gen. Psychi. 37:595,1980.

and Bergin, A. E., A Bibliographyof Res;arch in Psychotherapy (Washington,D. C.: NIMH, 1969).

and Hadley, S. W., “A TripartiteModel ‘of Mental Health and Therapeutic Out-comes, ” Am. Psychol. 32:187, 1977.

, et al., Psychotherapy for Better orWorse: The Problem of Negative Effects (NewYork, N. Y.: Jason Aronson, 1977).Stuart, R. B., Trick or Treatment: How and

92 ● Background Paper #3: The Efficacy and Cost Effectiveness of Psychotherapy

280,

281,

282.

283.

284.

285.

286.

287.

288.

289.

290.

291.

292.

293.

294.

When Psyclzotherapy Fails (Champaign, Ill.:Research Press, 1970).Subotnik, K. L. “Spontaneous Remission: Factor Artifact, ” Psychol. Bull. 77:32, 1972.Tennov, D., Psychotherapy: The HazardousCure (Garden City, N. Y.: Anchor Press, Dou-bleday, 1976).Tharp, R. G., and Wetzel, R. J., Behav io rModification in the Natural Environment (NewYork, N. Y.: Academic Press, 1969).Thomas, J. A., The Productive School: A Sys-tems Analysis Approach to Educational Ad-ministration (New York, N. Y.: Wiley, 1971).Thoresen, K. E., et al., “Learning House: Help-ing Troubled Children and Their Parents HelpThemselves,“ in Progress in Behavior TherapyWith Delinquents, edited by J. S, Stumphauzer(Springfield, Ill.: Thomas, 1979).Ullmann, L. P., and Gurel, L., “Validity ofSymptom Ratings From Psychiatric Records, ”Arch. Gen. Psychi. 7:130, 1962.United States Bureau of the Census, Character-istics of American Children and Youth, 1976(Washington, D. C.: U.S. Department of Com-merce, Bureau of the Census, 1978).VandenBos, G. R., and Pines, C. D., “Re-search on the Outcome of Psychotherapy, ” inPsychotherapy: From Practice to Research inPolicy, edited by G. R. VandenBos (BeverlyHills, Calif.: Sage Publications, in press).Vorster, D., “Psychotherapy and the Results ofPsychotherapy,” S. Afr. Med. ]. 40:934, 1966.Wadsworth, W. V., et al., “Cost of Treatmentof Affective Disorders: A Comparison of ThreeHospitals, ” Lancet 273:533, 1957.Walder, L. O., et al., “Teaching BehavioralPrinciples to Parents of Disturbed Children, ” inPsychotherapeutic Agents: New Roles for Non-Professionals, Parents, and Teachers, edited byB. G. Guerney (New York, N. Y.: Holt, Rine-hart and Winston, 1969).Wallace, J., and Sechrest, L., “Frequency Hy-pothesis and Content Analysis of ProjectiveTechniques,” ]. Cons. PsychoZ. 27:387, 1963.Washburn, S., et al., “A Controlled Compari-son of Psychiatric Day Treatment and Inpa-tient Hospitalization, ” 1. Cons. Clin, Psychol.44:665, 1976,Waskow, I. E., and Parloff, M. B. (eds. ), Psy-chotherapy Change Measures, DHEW publica-tion No. (ADM) 74-120 (Washington, D. C.:DHEW, 1975).Watson, D. L., and Tharp, R. G., Self-DirectedBehavior: Self-Modification for Personal Ad-justment (Belmont, Calif.: Brooks/Cole, 1972).

295.

296.

297.

298.

299.

300.

301,

302.

303.

304.

305.

306.

307.

308.

309.

Watts, C. A., et al., “Cost-Effectiveness Analy-sis: Some Problems of Implementation, ” Med.Care 17:430, 1979.Weisbrod, B. A., A Guide to Benefit-CostAnalysis, as Seen Through a Controlled Experi-ment in Treating the Mentally Zll, discussionpaper SS9-79 (Madison, Wis.: Institute for Re-search on Poverty, University of Wisconsin,1979).

, et al., “An Alternative to the MentalHospital—Benefits and Costs, ” paper presentedat the meeting of the American PsychiatricAssociation, Miami Beach, Fla., April 1976.Weiss, C. H., Evaluation Research: Methods ofAssessing Program Effectiveness (EnglewoodCliffs, N. J.: Prentice-Hall, 1972).Williams, R. L., and Long, J. D., Toward aSelf-Managed Life Style (2d cd., Boston,Mass.: Houghton Mifflin, 1979).Wolberg, L. R., The Technique of Psychother-apy (New York, N. Y.: Grune and Stratton,1977).Wolins, M., A Manual for Cost Analysis in In-stitutions for Children (New York, N. Y.: ChildWelfare League of America, 1962).Wolpe, J., Psychotherapy by Reciprocal Inhi-bition (Stanford, Calif.: Stanford UniversityPress, 1958).Wortman, P. M., “Evaluation Research: APsychological Perspective,” Am. Psycho/,30:562, 1975.Yates, B, T., “Benefits and Costs of Communi-ty-Academia Interaction in a ParaprofessionalTraining Course,” Teach. Psychol. 7:8, 1980.

Improving Effectiveness and ReducingCosts ‘in Mental Health (Springfield, 111.:Thomas, 1980).

“Improving the Cost-Effectiveness of .Obesit~ Programs: Three Basic Strategies forReducing the Cost per Pound, ” lnt. ], Obes.2:249, 1978.

, and Newman, F., “Approaches to CostEffectiveness Analysis and Cost Benefit Analy-sis in Psychotherapy, ” in Psychotherapy: FromPractice to Research in Policy, edited by G, R.VandenBos (Beverly Hills, Calif.: Sage Publica-tions, in press).

, et al., “Toward More Cost-EffectiveDelivery Systems in Behavioral Medicine: SelfManagement Institute’s Obesity, Insomnia,and Smoking Divisions, ” in Behavior GroupTherapy, 1981: An Annual Review, edited byD. Upper and S. M. Ross (Champaign, Ill.: Re-search Press, in press).Young, F. W., “Nonmetric Multidimensional

References ● 93

Scaling: Recovery of Metric Information, ” Psy - Counselor-Training Systems: An Operant andckometrika. 35:455, 1 9 7 0 . Operations Research Perspective, ” Coun. Psy-

3 1 o . Zifferblatt, S . M., “Analysis and Design of chol, 3:12, 1972.