Strategies for Medical Technology Assessment

Strategies for Medical Technology Assessment

September 1982

NTIS order #PB83-113274

Library of Congress Catalog Card Number 82-600606

For sale by the Superintendent of Documents,U.S. Government Printing Office, Washington, D.C. 20402

Foreword

This report, Strategies for Medical Technology Assessment, analyzes the presentsystem of identifying and testing medical technologies and of synthesizing and dissemi-nating assessment information. OTA began the study in July 1980, at the request ofthe House Committee on Energy and Commerce.

The report focuses on the flow of information that is central to an efficient assess-ment system. Methods for testing technologies and for synthesizing information are ex-plored, and a compendium of data and bibliographic sources are included. The reportalso describes the innovation process for medical technologies, the effects that Federalpolicies have on that process, and the needs those policies generate for technology assess-ment information. It critiques the current system of assessment and provides policyoptions, both legislative and oversight, for Congress to improve the system.

During the course of this assessment, both the House Committee on Energy andCommerce and the Senate Committee on Labor and Human Resources requested thatOTA study several specific areas in more depth. In response to these requests, OTAis publishing three other volumes: 1) a report on medical technology under proposalsto increase competition in health care, 2) a report on the postmarketing surveillanceof prescription drugs, and 3) a technical memorandum on MEDLARS (the NationalLibrary of Medicine’s Medical Literature Analysis and Retrieval System) and health in-formation policy. Another paper, funded as part of this assessment, concerns the potentialrole of Professional Standards Review Organizations in medical technology assessment.

In preparing this report, OTA consulted with members of the advisory panel forthe assessment, with contractors and special consultants, and with numerous other ex-perts in industry, medicine, economics, pharmacology, ethics, information science, andhealth policy.

Drafts of the final report were reviewed by the advisory panel chaired by Dr. LesterBreslow, by the Health Program Advisory Committee chaired by Dr. Sidney S. Lee,and by approximately 100 other individuals and groups representing a wide range ofdisciplines and perspectives. We are grateful for their many contributions. As with allOTA reports, however, the content is the responsibility of the Office and does not con-stitute consensus or endorsement by the advisory panel or the Technology AssessmentBoard.

JOHN H. GIBBONSDirector

. . .///

Advisory Panel for Strategies for Medical Technology Assessment

Lester Breslow, Panel ChairmanSchool of Public Health, University of California, Los Angeles

Morris CohenDirector of Technology AssessmentKaiser Permanence Medical Group

Richard CooperWilliams & Connolly, Inc.

D. V. d’ArbeloffChairman and Chief Executive OfficerMillipore Corp.

Harvey FinebergHarvard School of Public Health

Jerome D. FrankThe Henry Phipps Psychiatric ClinicThe Johns Hopkins Hospital

William GoffmanSchool of Library ScienceCase Western Reserve University

Leon GreeneVice PresidentNew Product TechnologySmith, Kline & French Laboratories, Inc.

David B. HomerOrthopedic Surgeon

Stanley B. JonesVice PresidentBlue Cross/Blue Shield Association

F. Wilfrid LancasterGraduate School of Library and

Information ScienceUniversity of Illinois

Louise B. RussellSenior FellowThe Brookings Institute

Herbert SemmelPresidentConsumer Coalition for Health

Robert M. VeatchKennedy Institute of EthicsGeorgetown University

Richard W. VilterAmerican College of PhysiciansCollege of MedicineUniversity of Cincinnati

Kenneth E. WarnerSchool of Public HealthUniversity of Michigan

Richard N. WatkinsStaff PhysicianGroup Health Cooperative

Carol WeissGraduate School of EducationHarvard University

Kerr L. WhiteDeputy Director for Health SciencesRockefeller Foundation

iv

OTA Project Staff—Strategies for Medical Technology Assessment

H. David Banta, Assistant Director, OTAHealth and Life Sciences Division

Clyde J. Behney, Health Program Manager

Bryan R. Luce, F’reject DirectorDale A. Carlson, Analyst

John C. Langenbrunner, AnalystGloria Ruby, Analyst

Kerry Britten Kemp, Editor

Virginia Cwalina, Administrative AssistantMary E. Harvey, SecretaryPamela Simerly, Secretary

Lorraine G. Ferris, Secretary*Nancy L. Kenney, Secretary*

Other Contributing Staff

Ann Rose, Senior AnalystArthur Kohrman, Congressional Fellow

Hellen Gelband, AnalystMark Hochler, InternLisa Scheffler, InternHarvey Sepler, Intern

Principal Contractors

Rand Corp.John P. Bunker, Stanford University

Lawrence MiikeJohn B. Reiss, Baker and HostetlerLeonard Saxe, Boston University

Paul M. Wortman, University of Michigan

OTA Publishing Staff

John C. Holmes, Publishing Officer

John Bergling Kathie S. Boss Debra M. Datcher Joe Henson

‘Until January 1982,● *Unti] September 1981.

Health Program Advisory Committee

Sidney S. Lee, ChairmanVice President, Michael Reese Hospital and Medical Center

Stuart H. AltmanFlorence Heller SchoolBrandeis University

Robert Ball’Institute of MedicineNational Academy of Sciences

Lewis N. Butler*Health Policy ProgramSchool of MedicineUniversity of California, San Francisco

Kurt DeuschleMount Sinai School of Medicine

Zita Fearon*Consumer Commission on the Accreditation

of Health Services, Inc.

Rashi FeinCenter for Community Health and Medical CareHarvard Medical School

Melvin A. GlasserCommittee for National Health Insurance

Patricia KingGeorgetown Law Center

Joyce C. LashofSchool of Public HealthUniversity of California,

Mark Lepper

Berkeley

Vice President for Inter-Institutional AffairsRush-Presbyterian-St. Luke’s Medical Center

Margaret MahoneyPresidentThe Commonwealth Fund

Frederick MostellerDepartment of Health Policy and ManagementHarvard School of Public Health

Beverlee Myers*DirectorDepartment of Health ServicesState of California

Mitchell RabkinGeneral DirectorBeth Israel Hospital

Dorothy P. RiceDepartment of Social and Behavioral SciencesSchool of NursingUniversity of California, San Francisco

Richard K. RiegelmanSchool of MedicineGeorge Washington University

Walter L. RobbVice President and General ManagerMedical Systems OperationsGeneral Electric

Frederick C. RobbinsPresidentInstitute of MedicineNational Academy of Sciences

Rosemary StevensDepartment of History and Sociology of ScienceUniversity of Pennsylvania

Kerr WhiteDeputy Director for Health SciencesRockefeller Foundation

“Former members of the Health Program Advisory Committee whose terms expired prior to the completion of this study

vi

Contents

1.

2.

3.

4.

5.

6.

7.

8.

Page

Introduction and Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Information for Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Evaluating Health and Economic Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Social Values in Technology Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Synthesis, Cost-Effectiveness Analysis, and Decisionmaking . . . . . . . . . . . . . . . 57

Factors Affecting the Development, Diffusion, and Use ofMedical Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Critique of the Current System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Policy Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

Appendixes

A. Compendium of Statistical Data Sources for Medical TechnologyAssessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

B. Compendium of Bibliographic Data Bases for Medical TechnologyAssessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

C. Assessment of Medical Technology: Methodological Considerations . . . . . . . 127

D. Medical Technologies and Innovation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

E. Case Studies of Medical Innovations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

F. Model for an Institute for Health Care Evaluation . . . . . . . . . . . . . . . . . . . . . . . 185

G. Method of the Study and Description of Other Volumes . . . . . . . . . . . . . . . . . 192

H. Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

I. Glossary of Acronyms and Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .....................-.....205

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....................223

vii

Introduction and Summary

Knowledge advances by steps, and not by leaps.

—Thomas Babbington Macaulay

Need for a Strategy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Dimensions of the Need and the Problem . . . . . . . . . . . . . . . .

Government . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .The Public . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Health Professionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Industry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Nature of the Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Conceptual Framework for Medical Technology AssessmentIdentification: Technologies Needing Assessment . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .Testing: Types of Information Needed and Mechanisms for Testing . . . . . . . . . . . . . . . . .

Health Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Economic Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Social Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Mechanisms for Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Synthesis: Using Information as the Basis for Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . .Synthesis of Research Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Synthesis of Health, Economic, and Social Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Synthesis of Opinion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Dissemination of Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Government Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Other Mechanisms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Major Conclusions of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Identification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Emerging Technologies . . . . . . . . . . . . . . . . . .New Technologies . . . . . . . . . . . . . . . . . . . . . .Existing Technologies . . . . . . . . . . . . . . . . . . .New Applications of Existing Technologies

Testing .~;. . . . . . . . . . . . . . . . . . . . . . . . . . . . .Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Dissemination. . . . . . . . . . . . . . . . . . . . . . . . . .

Policy Options. . . . . . . . . . . . . . . . . . . . . . . . . . .Legislative Options . . . . . . . . . . . . . . . . . . . . .Oversight Options . . . . . . . . . . . . . . . . . . . . . .

Organization of the Remainder of the Report

List of Figures

. .

. .

. .

. .

. .,.. .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .,...., . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Page

344455678

10101112121314141515151616161617171717171718181819

Figure No. Page

l. Process of Assessing Medical Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72. Lifecycle of a Medical Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1

Introduction and Summary

NEED FOR A STRATEGY

Several reasons for assessing medical technolo-gies have been presented in previous OTA reports,Assessing the Efficacy and Safety of Medical Tech-nologies and The Implications of Cost-Effective-ness Analysis of Medical Technology. The mainreasons are to help ensure that medical technolo-gies are safe, efficacious, and appropriately used.Whether current policies and practices for medicaltechnology assessment achieve these and relatedobjectives is the subject of this report. Havingstudied both the methods of medical technologyassessment and the dissemination of informationdeveloped by technology assessment, OTA findsthat a strategy is needed to implement the assess-ment process to make it more effective. OTA alsofinds that greater attention to assessment of socialand ethical values is needed for policymaking.

A medical technology, as used in this report,is a drug, device, or medical or surgical procedureused in medical care. (The term may also applyto the organizational and supportive systemswithin which medical care is delivered, but thosesystems are not the focus of this report. ) Med-ical technology assessment is, in a narrow sense,the evaluation or testing of a technology for safetyand efficacy. In a broader sense, it is a processof policy research that examines the short- andlong-term consequences of individual medicaltechnologies and thereby becomes the source ofinformation needed by policymakers in formu-lating regulations and legislation, by industry indeveloping products, by health professionals intreating and serving patients, and by consumersin making personal health decisions. Unfortunate-ly, that process currently has deficiencies thatcause or allow confusion to exist at all decisionpoints.

Historically, medical technology assessment hasdeveloped incrementally as responses to specificdemands. Taken singly, some of these responseshave been coherent (e.g., the Food and Drug Ad-ministration’s (FDA’s) premarketing approval

process which was developed to protect the publicfrom unsafe and inefficacious new drugs). Takenin combination, however, these various responsesdo not constitute a coherent system for assessingall classes of medical technologies. The presentapproach is characterized by multiple participantsfrom the public and private sectors, and by unco-ordinated activities. Complicating matters furtheris the large number of medical technologies in use,with thousands of new technologies appearingevery year. The result is an overload and confu-sion among decisionmakers and consumers.

OTA finds that a strategy is needed to guidethe selection and implementation of componentsthat would constitute a coordinated system ofmedical technology assessment. The basis of thestrategy should be the values and available re-sources in a free-market economy, coupled withthe social responsibility to make available safe,effective medical care. The vehicle of the strategyshould be a systematic process of information de-velopment, dissemination, and use. The targetshould be to address the confusion deriving fromthe lack of information available to decisionmak-ers.

Minimally, the following components of an as-sessment system must be considered in develop-ing a strategy:

1. the values of individuals and of society con-cerning medical technologies and their use;

2. the goals and appropriate role of medicaltechnology assessment in society;

3. the types of assessment information neededfor decisionmaking;

4. the methods and technologies for develop-ing and acquiring the information; and

5. mechanisms for disseminating and applying

the information, including programs thatwill use the

A strategy formust consider not

information.

assessing medical technologiesonly the methods of assessment,

3

4 ● Strategies for Medical Technology Assessment

but also the needs, demands, and resistances ofpotential participants in the process of assessment.Specifically, the public itself as consumers; healthcare professionals as users; industry as innovators,producers, and reimbursers; and the Federal Gov-ernment simultaneously as purchaser and guard-ian must be informed and active in setting mutual-ly compatible goals for technology assessment.Each sector has health, social, and economic val-

ues underlying its decisionmaking. Clarifyingthose values and realistically accommodatingthem will require developing not just more, butalso more reliable information about the safety,efficacy, cost effectiveness, and social and ethicalimplications of all classes of medical technologies.The inconsistencies and contradictions in availableinformation are reflected in the inconsistent andcompeting pressures from the various sectors.

DIMENSIONS OF THE NEED AND

As an illustration of the mutual involvementof all sectors with a medical technology and asan illustration of the waste and potential threatresulting from premature adoption of an unas-sessed technology, consider the medical procedureof gastric freezing. In the mid-1950’s, a clinicianresearcher at a university medical school, in con-junction with a private corporation, developeda device to treat peptic ulcer disease with gastriccooling. The procedure involved circulating alco-hol at –15° C through a nasogastric tube to aballoon inserted into the stomach. He first triedthe procedure on dogs, then on a dozen humanpatients, and reported in 1962 the followingresults: no serious side effects, reduced stomachacid output, immediate relief of ulcer pain, andradiographic evidence of ulcer healing. By the endof 1963, 1,000 devices had been sold, and 15,000procedures had been performed nationwide, aidedfinancially by third-party reimbursers. In 1964,other published reports concluded that acid sup-pression was limited or was unrelated to pain re-lief, symptomatic improvement was short-livedor due to placebo effects, and serious risks werepresent. By 1966, the technique was rarely used.

As an extreme example—that is, a technologythat did not work—gastric freezing makes obviousthe useless expenditure of money, time, and hu-man emotion. The questions about most technol-ogies, however, are more subtle. Most medicaltechnologies have a therapeutic or diagnostic val-ue for specific problems under appropriate cir-cumstances. The difficulty is determining forwhom and under what circumstances use of atechnology is valid or worth the tradeoff of risksand benefits. Mammography and radical mastec-

THE PROBLEM

tomy, for example, have a place in the detectionand treatment of breast cancer, but understandingexactly what that place is may take years and acertain amount of trial and error.

Government

The Federal Government’s interest in develop-ing clear policies and an effective strategy for as-sessing medical technology derives from its tradi-tional role as guardian of the public’s safety andof social equity and from its concerns about eco-nomic issues. As protector of the public, the Gov-ernment seeks to ensure that health care is notonly safe but also efficacious. As the single larg-est buyer of health services, the Government seeksto ensure that all citizens, especially the poor,have health care available to them; but the Gov-ernment is also concerned about rising health carecosts in general and specifically about those it paysfor directly through programs of service or reim-bursement (Medicare, for example) and throughbiomedical and other health research. Any policiesthe Government sets will affect not only the Gov-ernment itself, but the public and private industry,and such policies must especially be justifiablewhen the public and private industry make self-interested demands.

The Public

The gastric freezing incident, though occurring15 years ago, is still representative of current is-sues. As more recent technologies receive wide-spread attention (e.g., mammography, laetrile,and electronic fetal monitoring), the public be-comes more vocal and involved. The public is of-

Ch. 7—Introduction and Summary ● 5

ten neither fully informed about the safety andefficacy of individual technologies nor educatedabout the issues of cost and social values that mustbe considered in the adoption of a technology.The public mixes facts with beliefs, hopes, andfears and translates those into confused, contra-dictory, and often impossible demands.

For example, the public hears of a drug, perhapsone used in another country, and wants it imme-diately available to patients in the United States,especially when available therapies are ineffective.The desperation individuals feel tends to outweighthe fear of any risks that might be involved, andthey demand the right to take personal responsi-bility for use of the drug. Perhaps assuming thatif a therapy is used in a European country, it hasalready met rigorous assessment standards, thepublic perceives itself as being denied a cure forno valid reason. The recent laetrile issue is perhapsthe most emotionally dramatic example.

Simultaneously to demanding speedy availabil-ity and personal responsibility, however, the pub-lic demands protection against all forms of un-safe medical practice and is prepared to sue formistakes. Perhaps because of the rigor which FDAapplies to approval of new drugs and because ofGovernment safety standards applied to so manynonmedical products, the public assumes that itis likewise protected in undergoing any medicalor surgical procedure recommended.

The confused demands of the public can beviewed either as irrational or as a frustrated reflec-tion of the deficiencies that do exist in the Nation’sapproach to assuring the availability of safe, ef-fective, and cost-effective medical technologies.Numerous needs and values are implied in the de-mands of the public and must be taken into ac-count when planning a strategy of policies andprocedures for medical technology assessment.

Health Professionals

Health professionals often find themselves incircumstances that require decisions based on in-adequate information. They take action or advisepatients who must make decisions about use ofdrugs, devices, or medical and surgical proce-dures. Although at the time of decision the pa-

tient may be willing to assume responsibility forthe decision, later, if harm occurs, the patienttends to hold the physician responsible. The flawsin the information flow to physicians and otherhealth professionals are numerous: there is notenough information available about the safety,efficacy, costs, and social values of medical tech-nologies; much existing information is of dubiousquality and is therefore unreliable; the practicalsignificance of data is usually not interpreted forclinicians; and easy access to the appropriate in-formation is rare.

Furthermore, medical education typically doesnot train physicians and other health care profes-sionals to make decisions based on a considera-tion of values. They are trained to seek the mostreliable technique to produce a desired physiologi-cal response. As an illustration, in the issue of sav-ing the lives of extremely premature babies in in-cubators, physicians, by training, would tend tobe concerned mainly with choosing the technol-ogy that would support life. Physicians would lesslikely know or be concerned about the implica-tions of the survival of the deformed or retardedinfant—implications for the infant itself, for thefamily, and for society. Thus, developing and sup-plying the right kind of assessment informationto health care professionals is essential to a strat-egy for medical technology assessment.

Industry

From the point of view of the private sector,of producers of technologies and of third-party

payers, the assessment of medical technologies isboth advantageous and disadvantageous. Govern-ment’s involvement in the assessment processraises primarily financial issues for the privatesector.

Industry, which invests money in research anddevelopment (R&D), is willing to do so if thereis a potential market for the device or drug; how-ever, excessive regulation or the wrong kind ofregulation by the Government could discourageinnovation if companies fear that assessment willultimately preclude marketing their product ormaking a profit from it.


Private third-party payers, on the other hand,might welcome shifting the entire burden ofassessment to the Federal Government. They mustmake decisions about reimbursement—whetherto reimburse for specific procedures and if so howmuch—but they have little incentive to conducttheir own assessments of procedures because ofthe expense. Assessment information tends to bewidely available and not proprietary; the in-surance companies cannot profit individuallyfrom conducting assessments. The failure of in-dustry members to adequately conduct assessmentactivities on their own puts a heavy responsibili-ty in the Government domain.

Nature of the Challenge

The market for medical technologies is moder-ated by individual consumer tastes and financialconstraints. To perhaps a greater degree, it is in-fluenced by policies that determine what kinds ofresearch will be supported, what regulations re-strict market entry, and which technologies willbe reimbursable by Government or private pro-grams.

No policy decision has isolated effects in justone sector; repercussions occur throughout the en-tire social and economic fabric of the Nation. Aregulatory decision to require extensive, expen-sive assessment of a medical device in a develop-mental phase, but not to offer industry assistancein the assessment, for example, could lead to adecision by industry never to begin the innova-tion phase. An idea might never be realized whicheventually could have best served the public. Infact, current policies and procedures for assess-ment are not adequate to fully serve the publicinterest. No consistent policy or system exists forassessing all classes of medical technologies, noreven for various technologies within a class.

The principles of competition and of supply anddemand which ordinarily control prices and con-sumer choices in the market do not operate effi-ciently in the provision of medical care, especial-ly because of reimbursement policies. Typically,for example, after a dramatic new procedure be-comes routine, requiring less time and skill andincurring fewer risks, fees increase rather than de-crease. Hospitals invest in services and new equip-

ment which, like the hospitals themselves, areoften underutilized. Third-party coverage of med-ical care, both Government and private, is a ma-jor cause of this inflated purchasing and cost. Forthis reason, the 1972 amendments to the SocialSecurity Act limited the amounts Medicare couldpay institutions and physicians.

Reimbursement decisions also influence the in-novation and adoption of medical and surgicalprocedures. Although new procedures tend to beadopted and reimbursed without adequate assess-ment, in the case of truly innovative procedures,third-party payers sometimes refuse reimburse-ment. While encouraging new applications, slightmodifications, and excessive use of existing tech-nologies, the present reimbursement system maydiscourage radical innovations.

The challenge in developing a strategy for as-sessment is to develop a system that will serve thepublic interest by encouraging the developmentand appropriate use of needed and safe medicaltechnologies without unnecessarily discouraginginnovation and production. The practical ques-tions are: What information is needed to makedecisions about medical technologies in the bestinterest of the public and how can that informa-tion best be generated and disseminated? Canclear knowledge be developed that will enable pol-icymakers and decisionmakers to act in the bestinterest of the social and economic elements ofthe Nation?

But there are also philosophical considerations.What the role of Government is and how strongthat role should be is the subject of a perennialdebate. Should the role be regulatory or over-sight? To what degree? Should industry be leftto its own incentives or pressured by the Gover-nment with directed incentives? How, in otherwords, can the Federal Government move thecountry toward a more efficient and equitable sys-tem of ensuring that useful and timely informa-tion is available to those who need it, withoutadversely affecting the innovation process andhealth care services?

The next section of this chapter presents the ma-jor components, drawbacks, and considerationsin the existing process of medical technology as-sessment.

Ch. I—Introduction and Summary ● 7

CONCEPTUAL FRAMEWORK FORMEDICAL TECHNOLOGY ASSESSMENT

Medical technology assessment involves numer-ous components and subcomponents at variousstages of the process. Though these do not existas a coherent system, discussion of them is facili-tated by describing a systematic framework. Themultiple components of the medical technologyassessment process can be conceptualized as aninformation flow associated with the followingfour stages of assessment (see fig. 1):

● Identification. —Monitoring technologies, de-termining which need to be studied, and de-ciding which to study.

• Testing. —Conducting the appropriate anal-yses or trials.

● Synthesis. —Collecting and interpreting ex-isting information and the results of the test-ing stage, and, usually, making recommen-dations or judgments about appropriate use.

. Dissemination. —Providing the synthesis ofinformation, or any other relevant informa-tion, to the appropriate parties who use med-ical technologies or make decisions abouttheir use.

Figure 1 .—Process of AssessingMedical Technologies

I Identification

. I . .-

Testing

Dissemination I

SOURCE: Off Ice of Technology Assessment.

This four-stage process is applicable to the threeclasses of medical technologies mentioned earlier—namely, drugs, devices, and medical and surgi-cal procedures. It is also applicable to any technol-ogy in any of four typical stages of development,loosely defined as follows:

●

●

●

●

Emerging technology. —A technology in thephase prior to adoption.New technology. —A technology in the phaseof adoption.Existing technology. —A technology in gen-eral use.New application of an existing technol-ogy. —A-new application of a technology ingeneral use.

Visualizing the lifecycle of a hypothetical tech-nology (see fig. 2) makes obvious some of the deci-sion points at which assessment information is es-sential. If an emerging technology is a drug or de-vice, industry must decide whether to commit re-sources to develop it; must later decide whetherto market it; and must ultimately decide whetherto maintain, alter, or discard it. If a new drug ora certain class of device is to be marketed, FDAmust decide whether to grant market approvalbased on safety and efficacy criteria. If the newtechnology is to be used in medical practice, some-one must decide whether to pay for it. In somecases, the Health Care Financing Administration(HCFA) must decide whether to include a newtechnology or a new use of an existing technologyas a reimbursable expense for Medicare benefici-aries. Private insurers, such as Blue Cross/BlueShield and health maintenance organizations,must make similar decisions. Hospitals must de-cide whether to purchase, and practitioners andtheir patients must decide whether to use, the tech-nology. Finally, all users and payers at times needto review the usefulness of existing technologies.And, in some cases, existing technologies find newuses or are modified, and the process begins allover.

In contrast to drugs and devices, medical andsurgical procedures and their variations are ordi-narily developed by clinicians and researchers and

— - —

SOURCE: Office of Technology Assessment.

therefore seldom require investment decisionmak-ing by industry. Furthermore, under the presentsystem, medical and surgical procedures are notregulated for safety and efficacy by FDA and thustend to escape the regulatory decisions. Neverthe-less, decisions about reimbursement of such pro-cedures must be made.

Many medical technologies in use have not beenadequately evaluated. If all medical technologieswere adequately assessed as emerging or newtechnologies, there would be less need for assess-ing existing technologies.

In addition to considering the stages of the as-sessment process and the classes and developmen-tal stages of technologies, an assessment systemrequires the measuring of specified effects. De-pending on the technology, the effects to be con-sidered are health (safety, efficacy, and effective-ness), economic, or social. Once the categories ofeffects to measure have been determined, testingand analysis may begin. Throughout the assess-ment process, all information and decisions mustbe balanced against the moral and ethical valuesof society.

IDENTIFICATION: TECHNOLOGIES NEEDING ASSESSMENT

A decision to conduct a technology assessmentmust be preceded by the identification of tech-nologies that should be assessed and the settingof priorities among candidate technologies. Iden-tification procedures may vary with the type oftechnology, but basically can be classified as oneof three types: 1) routine mechanisms, 2) priority-

setting mechanisms, and 3) mechanisms of oppor-tunity. Routine mechanisms systematically iden-tify a class of technologies, usually in relation toa specific event—e.g., FDA requires that all drugsand devices be registered before they can be mar-keted or tested in humans. Priority-setting mech-anisms are used, as needed, to apply implicit or

Ch. I—Introduction and Summary ● 9

explicit criteria to determine which technologiesshould be assessed-e. g., HCFA and the NationalInstitutes of Health (NIH) set research agendas.Mechanisms of opportunity are not formalizedbut are valuable in identifying technologies as theysurface or become important—e.g., patient out-come data may bring the need for analysis to theattention of researchers or the public.

Identifying medical technologies for priority-setting and assessment is an important responsibil-ity primarily of several agencies within the De-partment of Health and Human Services (DHHS):FDA, the National Center for Health Services Re-search (NCHSR), NIH, and HCFA. The NationalCenter for Health Care Technology (NCHCT),while it was funded, also identified technologiesfor assessment.

FDA identifies new drugs and medical devicesthrough its premarket approval authority. To testpromising new drugs in humans, drug sponsors(e.g., manufacturers) must notify and receive per-mission from FDA through a “notice of claimedinvestigational exemption for a new drug” (IND).If the drug successfully passes this premarket test-ing, the sponsor may file for a “new drug applica-tion” (NDA), which is a request for FDA’s permis-sion to market the drug. Since 1962, when thisregulatory mechanism was instituted, FDA hasreviewed over 13,500 applications for INDs andhas approved about 1,000 NDAs. Since 1976,FDA also has an expanded responsibility for regu-lating medical devices. In the first 4 years of im-plementing the 1976 Medical Device Amend-ments, about 98 percent of the listed devices inthe 10,540 premarket notifications received wereclaimed to be “substantially equivalent” to pre-existing devices. In 1981, FDA estimated that2,300 premarket notifications would be reviewed.New applications of existing drugs and devicesmust also meet premarket approval requirements,but the initiative for these new applications re-mains with the manufacturer, not with FDA. FDAdoes support some monitoring activities of exist-ing drugs and requires manufacturers to reportadverse reactions, but these postmarketing sur-veillance activities are focused on the safety as-pects of these drugs, not on refinements in use ornew uses.1 Nevertheless, postmarketing surveil-

IThis topic is explored in greater depth in OTA’S report entitledPostmarketing Surveillance of Prescription Drugs.

lance has the potential of becoming an effectivemethod of identifying existing technologies in needof further assessment.

NIH and NCHSR are research agencies thatidentify emerging and sometimes new technol-ogies in need of assessment through their priority-setting processes for research grants and contracts.Projects are selected on the basis of technical meritand whether they are addressing important issues.The processes generate information useful to pol-icy decisions, but do not necessarily address theimmediate priorities of operating agencies suchas HCFA.

HCFA reimburses for Medicare and thereforehas obvious incentives for identifying technologiesin need of assessment; nevertheless, it has nomechanisms for the identification of existing tech-nologies in widespread use. For new technologies,the identification is by opportunity. When thequestion of coverage arises for new technologies,HCFA must determine whether it has adequateinformation to make a decision and must set pri-orities for technologies that must be assessed toprovide more information. Also, through its Of-fice of Research and Demonstrations, HCFA setspriorities for assessing technologies that are im-portant to its operations.

NCHC’T was established in 1978 to undertakeand support assessments of medical technology,but did not receive funding in 1982. NCHCT co-ordinated interagency issues, but also set its prior-ities internally and had its own responsibilities foridentifying technologies. Specifically, NCHCTcompiled an annual “emerging technology list” asan early alert system for assessment, but the 1981reauthorization of NCHCT withdrew its authorityto compile the list. (Industry argued that the listthreatens innovation by casting doubt on theeventual marketability of a technology. ) NCHCTalso initiated a plan to develop a joint public-pri-vate model for collecting clinical data on emerg-ing technologies. Finally, the NCHCT Directorchaired the Technology Coordinating Commit-tee of DHHS, which was the department’s pri-mary mechanism for coordination of issues associ-

ated with medical technologies.

Overall, the current identification stage of thecurrent system of technology assessment has seri-ous shortcomings. The degree to which current

98- IL+4 O - 82 - 2


processes identify technologies varies. Emerging sector has a clear responsibility for the task. Newand new drugs and devices are adequately identi- mechanisms are especially needed to identify forfied for assessment prior to their being marketed. the purpose of assessment existing technologiesHowever, emerging and new medical and surgical of all classes, new applications of existing technol-procedures are not adequately identified, because ogies of all classes, and medical and surgical pro-no one in either the private or the Government cedures in all four phases of development.

TESTING: TYPES OF INFORMATION NEEDED ANDMECHANISMS FOR TESTING

As a basis for decisions, a strategy to assessmedical technologies must take into account whatis known, what is not known, what is needed,what can be obtained, and at what cost. Infor-mation will never be perfect, and money and timewill always be limited; thus, evaluation methodsmust be used judiciously and their results mustbe interpreted cautiously, in conjunction withnumerous other measurements, especially withconsideration for society’s moral and ethicalvalues. Three categories of information about amedical technology are needed for policy deci-sions: I) health effects, 2) economic effects, and3) social effects. The methods and procedures fordetermining these effects have strengths and weak-nesses closely paralleling those of the identifica-tion phase.

Health Effects

Health effects are determined during the testingstage of assessment. The basic questions asked are:Does the technology work? and How well doesit work? The former question seeks informationabout efficacy, effectiveness, and performancestandards, and the latter about safety (and risk).The information provided by analyses of healtheffects helps decisionmakers determine whethera drug or device should be allowed on the marketor whether further investment in R&D is war-ranted.

Patient outcome is the desired endpoint meas-ured in efficacy and effectiveness analyses; effi-cacy is tested under ideal clinical conditions,whereas effectiveness is tested under average, ortypical, conditions. Tests for effectiveness dem-onstrate whether efficacy information can be gen-

eralized to the population at large. For new drugsand certain devices, if the technology is in theemerging phase, its efficacy must be establishedin preclinical, biochemical, or animal tests beforeit can be tested among humans. The method thatgives the most valid and most reliable informa-tion about efficacy is the randomized clinical trial(RCT). The strength of the RCT lies in its random-ization process, producing two or more groupsthat are identical except for chance occurrence,which can be estimated statistically. The draw-backs of RCTS are that they can only be used incertain settings, they are sometimes not ethical toconduct, and they do not always provide com-plete information about safety.

Thus, despite the highly valid information theycan produce, RCTS are not always the methodof choice. Other methods can be used as substi-tutes for RCTS or to supplement them. Observa-tional methods are designed to analyze data fromnonrandomized study designs. Several techniquesare used to minimize selection bias. Observationalmethods can be useful in ruling out competing ex-planations for an observed effect and for testinghypotheses in large, diverse populations once atechnology is widely diffused. Prospective cohortstudies, for example, can be used to detect rareadverse reactions to drugs that were unsuspectedprior to marketing. Case-control studies are aninexpensive means of indicating whether the useof a technology results in a small level of risk.

Another, more common type of study is thecase study, typified by a physician reporting hisor her experience with particular technologies andpatients. Case studies are useful in an overall as-sessment strategy in that they can facilitate theidentification of technologies in need of assess-

Ch. l—introduction and Summary ● 1 1

ment. Case studies are important identificationmechanisms of opportunity, as defined earlier.However, the validity of case studies is extreme-ly low because of, among other things, observerbias and the placebo effect. Nevertheless, clini-cians are very often swayed by these case reports,which fill the medical literature and which oftendescribe the successful application of a technol-ogy.

Safety is measured in terms of a risk-to-benefitratio; it is therefore a relative concept, and its es-timation may be a byproduct of testing for ef-ficacy and effectiveness. A low risk maybe unac-ceptable if there is no benefit, but a high risk maybe acceptable if the benefits are also high. RCTstend to give risk information only on a small seg-ment of the population. To generalize to other seg-ments, supplemental information is needed fromsurveys and methods which can make use of reg-istries, and clinical data banks.

For certain technologies, especially devices, es-tablishing the technology’s performance integrityis a prerequisite for efficacy assessment. Perform-ance standards usually pertain to the chemical,physical, and electric properties of devices. Similarstandards are often used in evaluating technolo-gies which have an intermediate rather than adirect effect on the patient’s health outcome, e.g.,diagnostic and often prevention technologies. Insuch cases, the technology is evaluated in termsof its ability to cause one effect that in turn willcause the desired result. For example, an automat-ic blood pressure monitoring device must accu-rately measure and record blood pressure if it isto be used for diagnostic purposes. Coronary ar-tery bypass surgery is a preventive procedure forheart attack in that it increases blood flow to theheart, the expectation being that pain and the like-lihood of a heart attack will be reduced.

No precise formula exists for choosing the bestor most appropriate evaluation method. The stageof development of the technology itself —e.g.,emerging, new, or existing—will partially deter-mine the appropriateness of a method. The pur-pose of the technology —e.g., diagnostic, thera-peutic, or preventive —will limit the range of ap-propriate methods. However, other factors suchas existing knowledge about the risks and benefits

and available resources may influence or overrideotherwise “ideal” choices. The important criterionin selecting analytic methods is not which is theo-retically more sophisticated, but which is practic-ally the most appropriate.

Economic Effects

What does it cost the Nation and the individualto develop and use a medical technology? Whatdoes it cost not to develop or use a specific tech-nology? The answers to these questions supplydecisionmakers in Government and industry withinformation they need for allocating financialresources. All who pay for care—Government,insurance companies, individuals-need to knowwhether the use of the medical technology isworth the cost.

Analytical methods to determine economic ef-fects comprise a spectrum ranging from sophisti-cated computer-based data analyses to best-guessestimates of costs and benefits. The broad termscost-benefit analysis (CBA) and cost-effectivenessanalysis (CEA) refer to two techniques for com-paring the positive and negative consequences ofalternative ways to allocate resources. The princi-pal distinction between the two is that CBA valuesall costs and benefits in monetary terms whereasCEA produces a measure of the cost involved interms of some desirable health-related effects (e.g.,years of life gained).

Measurements of economic effects should con-sider both direct and indirect costs. Direct costsare those associated with direct medical careusage: the cost of the physician, the hospital, themedical supplies. Indirect costs are associated withthe value of time lost in receiving medical careand in being sick. When indirect costs are consid-ered in economic analyses—and often they arenot—they are frequently measured in terms of lostor gained wages.

Economic analysis is complex and must consid-er more than charges for services. For example,cost analyses should develop information on op-portunity cost, marginal valuation, joint produc-

tion considerations, R&D costs, overhead, costsv. prices, and discounting. Just as no one methodis invariably appropriate in the evaluation of


health effects, no one method of economic analy-sis is appropriate. The user of the information willpartially determine the kinds of analyses done.For the patient, the actual cost of services is theimportant information. For policymakers, morecomplex information is required. In the sequenceof the assessment process, information about theeconomic effects may be useless if reliable and ap-propriate information about the health effects isnot available.

Social Effects

Urgent ethical and social questions are beingraised in areas of biomedicine such as experimen-tation with human subjects, genetic engineering,human reproduction, and the possibly inappropri-ate prolongation of life. Who is affected by a med-ical technology? Who is not affected? What valuesof individuals and society are involved in use ofthe technology? What ethical principles are in-volved in testing the technology?

To varying degrees, medical technologies mayaffect the personal and work lives of patients andtheir families; influence the structure of medical,legal, and economic systems; and challenge socie-ty’s most fundamental beliefs. Considerations ofthe social and ethical implications of medical tech-nologies, therefore, must take an important placein the development of policies. Social implicationsare the direct or indirect effects of medical technol-ogy on the concepts, relationships, and institu-tions society considers important. Ethical ques-tions in relation to medical technologies—espe-cially those concerning principles of distributivejustice, respect for individuals, and benevolence—may also have profound social implications.

Unlike health and economic effects, social andethical issues do not lend themselves to quantita-tive measurement and analysis. However, the sys-tematic identification and evaluation of the socialimpacts resulting from the use of medical technol-ogies can be crucial. A related task is to identifythe values that underlie policy alternatives, includ-ing moral and ethical values. Systematically as-sessing values does not necessarily elucidate asingle, clear, conclusive answer about which pol-icy to adopt; but, rather, it clarifies the array ofchoices, the reasons for disagreements, and thecompromises required.

A second aspect of assessing values is to makea reasonable inquiry into the values that permeateand underlie the assessment itself. Value judg-ments enter into every aspect of technology assess-ment; they determine which technologies will beassessed and at what phase of their development,the scope of assessments, the kinds of data thatwill be collected and analyzed, the methods of theassessment, and how the assessment findings willbe used in decisionmaking. It is important to clar-ify, therefore, why an assessment of a particulartechnology was initiated and how it fits into largercultural and political contexts, what affects theperformance of assessment (e.g., the choice of as-sessors and the analytic goals and methods), andwhat values affect the application of the results.

Mechanisms for Testing

The major problem with the testing phase ofthe current assessment system is the lack of a sys-tematic approach for testing identified technolo-gies in all phases of development for all types ofrequired information.

FDA, in its regulatory role, is probably the mostsignificant agency in stimulating technology test-ing. Most FDA regulation requires industry totest, according to approved protocols, new drugsand many medical devices for safety and efficacy.For drugs, Phase I studies determine levels of tol-erance (toxicity), followed by early dose rangingstudies for safety and sometimes efficacy. If safe,the drug can be tested in Phase II studies to dem-onstrate efficacy and relative safety under con-trolled conditions. Phase 111 studies are expandedcontrolled and uncontrolled clinical trials. If thesetrials are successful, the company may file anNDA. FDA then reviews the data and may ap-prove the drug for marketing. Since 1962, FDAhas approved about 1,000 NDAs. For devices,FDA requires that 90 days notice be given aboutany new device industry intends to market. If adevice does not meet safety and performancestandards for its assigned classification, or if ade-quate information is not available for such a deter-mination, FDA may require testing of the device.For drugs and devices, FDA’s assessment activitiesare generally limited to safety and efficacy anddo not involve cost, cost effectiveness, or socialeffects.

Ch. I—Introduction and Summary ● 1 3

Unlike drugs and devices, medical and surgicalprocedures are not regulated, and their testing,if done, is through research whose funding comesprimarily from NIH and from private founda-tions. The costs of the later developmental phaseof procedures tend to be paid by patients (or bythe Government), usually through standard medi-cal insurance policies, even when the procedurehas been clearly designated as experimental. Medi-cal and surgical procedures usually begin as user-generated innovations; for example, a surgeonmay modify an existing technique during surgery.Increasingly, innovations arise in-academic cen-ters, from researchers who know how to presenttheir innovations in a technically acceptable man-ner at professional meetings and in journals. Theseresearchers’ presentations tend to legitimize inno-vations without their receiving a routine, formalexamination for safety and efficacy.

Whereas FDA regulations affect efficacy andsafety, four other regulatory programs are con-cerned with cost issues: section 1122 review, Statecertificate-of-need laws, the National Health Plan-ning and Resources Development Act of 1974, andProfessional Standards Review Organizations(PSROs). Although HCFA, which makes reim-bursement policy, has its own research arm, theOffice of Research and Demonstrations, it has sel-dom conducted technology assessments. NCHCT,an agency legislatively mandated to support com-prehensive assessments of health care technologiesfor all effects (including health, economic, andsocial), was not funded for 1982.

Social assessment activities have been con-ducted by several Government mechanisms. OTAwas established in 1972 as an analytic supportagency to conduct policy research on science andtechnology issues for congressional committees.

OTA’s health-related reports have focused pri-marily on methods available for assessing tech-nologies and issues prompted by their use. TheNational Commission for the Protection of Hu-man Subjects of Biomedical and Behavioral Re-search was established in 1974 to develop ethicalguidelines for conducting research in human sub-jects. The National Commission produced numer-ous reports with recommendations, many ofwhich were adopted by DHHS, * particularly

those governing the protection of human subjects.The Ethics Advisory Board, which was establishedin 1978 at the National Commission’s recommen-dation but was not funded in 1980, was man-dated to review ethically problematic researchprotocols and research involving human projects.The board fielded queries from other DHHS agen-cies such as NIH and the Centers for Disease Con-trol. The President’s Commission for the Studyof Ethical Problems in Medicine and Biomedicaland Behavioral Research succeeded the NationalCommission in 1978. Members of the President’sCommission are appointed representatives fromDHHS, the Department of Defense, the VeteransAdministration, the Central Intelligence Agency,the National Science Foundation, and the WhiteHouse Office of Science and Technology Policy.The President’s Commission conducts studies inmedical practice and biomedical research and ex-amines five subjects for legal and ethical implica-tions: informed consent, privacy, uniform defini-tion of death, genetic issues and unborn humans,and availability of health services. NCHCT’s re-sponsibilities, as mentioned above, included as-sessment of the ethical, legal, and social implica-tions of medical technologies.

*Then the Department of Health, Education, and Welfare

SYNTHESIS: USING INFORMATION AS THE BASIS FOR DECISIONSSynthesis of the information generated during The synthesis activities that pertain to medical

the testing stage of the assessment process is the technology assessment fall into two broad areas:necessary step to providing a convincing and re- 1) synthesis of the results of individual researchsponsible basis for decisions made during all studies; and 2) synthesis of a body of researchphases of a technology’s lifecycle. findings with various concerns such as risk, social,


ethical, or cost factors. The first type of synthesisaddresses questions of safety, efficacy, or effec-tiveness of a given technology; the latter is morepolicy oriented, often seeking to set guidelines orstandards for medical practice or reimbursementpolicy. The value of the latter depends, in largepart, on the adequacy of the former.

Synthesis of Research Findings

The traditional approach to synthesizing re-search information is the literature review, an ar-ticle summarizing the data of those studies areviewer believes to be the most relevant to thetopic under review. Literature reviews are usefuland heavily relied on, but because of their scopeand the delays in the journal publication system,such reviews are rarely timely, especially in re-porting an ineffective or unsafe technology. Fur-thermore, the reviews are subjective and oftenhave no commentary on methodological problemsin individual studies.

More systematic procedures for integrating andinterpreting sets of research evidence do exist andcan be employed. The most simple technique isa simple classification technique, sometimes calledthe “voting method.” This technique involvesselecting a sample of evaluative studies, codingsome aspect of the design, classifying outcomesas favorable, neutral, or unfavorable and con-structing tables of research findings. The methodidentifies methodological strengths andweaknesses among studies and can help determinepatient populations and under what conditionsthey are most likely to benefit from a technology.

Meta-analysis is a technique that assesses themagnitude of treatment impact by quantitativecomparison of actual study results. This methodis particularly useful in assessing treatments forwhich a large number of studies are available andfindings across studies seem to have greatvariability. However, it may have drawbackswith respect to sample selection.

Currently, no single technique is fully adequatefor synthesizing research; however, the applica-tion of formal quantitative procedures is begin-ning to give a better understanding of methodo-logical problems in research itself. Formal pro-cedures can segregate differential outcomes

according to treatment characteristics and metho-dological approaches. Contradictions can then beidentified, analyzed, or further researched. In theperformance of formal quantitative analyses, animportant suggestion is that the significance of theresults should be interpreted and reported in lan-guage that is useful to decisionmakers.

Synthesis of Health, Economic,and Social Effects

How, then, does one bring together and synthe-size all information available about all three cate-gories of the effects of medical technologies—health, economic, and social? Once specific infor-mation has been synthesized through variousmethods in each of these realms, how can a deci-sionmaker balance the values and interpret theminto programmatic actions?

OTA’s report on CEA concluded that perform-ing an analysis of costs and benefits can be veryhelpful to decisionmakers, because the process ofanalysis gives structure to a problem, allows anopen consideration of all relevant effects of a deci-sion, and forces the explicit treatment of key as-sumptions. Formal techniques such as CEA canbe used to aid in the synthesis of information con-cerning the health and economic effects of a tech-nology. OTA found, however, that although CEAcan be useful as a decision-assisting tool, it exhib-its too many methodological and other shortcom-ings for the numerical results to be used as thebasis of policy or program decisions. For exam-ple, although CEA can be used to synthesize infor-mation concerning health and economic effects,it cannot in itself adequately address social andethical issues. These have to be addressed morefully by other means.

The most appropriate approach to any assess-ment is to perform it in an open forum so thatassumptions and underlying values can be chal-lenged; to identify, measure, and, to the extentpossible, value all relevant benefits/effects andcosts; and to present the results of the analysisas an “array” of benefits/effects and costs ratherthan forcing the results into a single aggregatemeasure. By arraying effects in a systematic fash-ion, one can place the appropriate relative empha-sis on given effects whether they are quantifiable

Ch. l—Introduction and Summary ● 1 5

or not. This technique is designed to make moreexplicit the health, economic, and social conse-quences of any decision.

Synthesis of Opinion

Synthesis of information may occasionally pre-sent a clear-cut indication of the next stage ofassessment or phase of technology development.More likely, uncertainty will still predominate fordecisionmakers. The uncertainty may reflect thepresence of random events or may reflect a basiclack of knowledge. The former can be analyzedby various statistical techniques: decision analysis,confidence limits, computer simulation, sensitivityanalysis. However, these techniques cannot actu-ally resolve policy controversies or substitute forinformed judgment.

Policy judgments may require a synthesis ofopinion which can be solicited from groups and

DISSEMINATION OF INFORMATION

What potentially are the direct effects of dis-semination of assessment information? Whoshould have top priority in receiving information?How should the information be disseminated? Thedissemination of assessment information directlyaffects the development and diffusion processesof medical technologies. The consideration ofwhether to disseminate information is thereforeweighty. If a decision is made to disseminate infor-mation because the technology is deemed eitherworthy or unworthy of its next phase of develop-ment, the information must reach, at a minimum,the decisionmakers involved with the technologyin any aspect of its use. That audience may rangefrom directors of R&D in private industry, tohealth professionals, to the general public. Reach-ing the audience in a timely manner requires asystematic approach to information dissemina-tion, especially in view of the pace and quantityof information development and the lack of mech-anisms for the systematic synthesis of informa-tion. In a sense, the information available is atonce too much and too little.

The dissemination phase of medical technologyshould comprise the mechanisms and coordina-

expert input. The most common format of solic-iting group opinions is the unstructured confer-ence which may involve presentations, discus-sions, and debates. Another informal techniqueis the advisory panel approach used by manyGovernment agencies. The four best known for-mal techniques used in medical contexts for re-solving conflicts and uncertainty are: 1) the Delphitechnique, 2) the nominal group process tech-nique, 3) the consensus development conference(NIH), and 4) a computerized knowledge basewhich maintains expert opinion on the state ofthe art of a specific topic (e.g., the HepatitisKnowledge Base of the National Library of Med-icine, NLM). Although these formal techniquesproduce more reliable opinion information thanan unstructured conference does, evidence of ef-fectiveness is contradictory for the Delphi andnominal group processes and sparse for the NIHand NLM processes.

tion of communication activities. Unfortunately,current procedures are highly flawed; there ex-ists no system for disseminating information, onlya variety of traditional mechanisms. Little isknown about the adoptive process or how infor-mation is used once it is received, but it is clearthat medical practice varies greatly from providerto provider and that even when good informa-tion is available, many technologies are used in-appropriately.

Government

The Federal

Activities

Government produces, collects,and disseminates assessment information.NCHSR, for example, disseminates the results ofhealth services research to relevant Governmentagencies, the research community, and other in-terested parties through publications, press re-leases, conferences, and workshops. In 1978, thelegislation authorizing NCHSR was modified torequire that at least $1 million or 5 percent of itsbudget, whichever is less, be used for dissemina-tion activities. In response, NCHSR establisheda User Liaison Program to provide substantive as-


sistance to non-Federal health care leaders con-cerned with critical policy issues and operationalproblems in the organization, administration, reg-ulation, and delivery of health care services atState and local levels.

Monitoring NIH’s dissemination activities is theresponsibility of the Office for Medical Applica-tions of Research (OMAR), established in 1978in the NIH Office of the Director, and assistedby the OMAR Advisory Committee. One impor-tant mechanism for dissemination is the consen-sus development conference. The synthesis ofopinion that is achieved at a consensus conferenceis presented in consensus statements and support-ing materials which are distributed to practicingphysicians, other health professionals, the bio-medical research community, and the public—through a mailing list of over 21,000 names. Also,members of the press are invited to the confer-ences and are encouraged to publish the results.Leading medical journals and medical societieshave published the consensus materials.

In conducting medical technology assessments,information from several subject areas is often re-quired. A common need in most assessments,however, is for information from the field of bio-medicine. NLM is the major Federal library re-source for biomedical literature. It is the predomi-nant creator and disseminator of biomedical bibli-ographic information. NLM’s coverage of thehealth services literature is less comprehensivethan its coverage of the biomedical literature, inpart because relevant health services informationappears in so many diverse documents.2 Another

source of information for medical technology as-sessments is the National Technical InformationService (NTIS). NTIS is the central repository forscientific and technical information generated byfederally funded R&D projects, including thosein DHHS.

Other Mechanisms

Apart from formal Federal agency activities,mechanisms for dissemination include the publicmedia, the mail, advertising, personal contacts,the educational process, libraries, and other typesof information centers. The appropriateness ofany of these mechanisms depends on whether theinformation is to be used in assessing or marketinga medical technology. Print media, radio, and tel-evision are primary channels to the public. In ad-dition to news about medical technologies and is-sues, they increasingly tend to have health col-umns and special in-depth features about healthtechnologies. For more targeted audiences, mail-ings are used for solicited and unsolicited infor-mation dissemination, for example, newslettersfrom drug companies, advertisements from prod-uct distributors, and Federal literature. Advertis-ing of drugs occurs in all media for the public andfor health professionals. A recently developedform of advertising, the video cassette, is suppliedto medical facilities. Personal contacts are an espe-cially credible source of information exchangeamong health professionals. These often occurformally and informally at professional meetings.

‘This topic is explored at greater length in a separate OTA tech-nical memorandum entitled MEDLARSand Heahh Znfbrrnation PoLicy, to be published in fall 1982.

MAJOR CONCLUSIONS OF THE STUDY

In this study of medical technology assessment, sented in figure 1: identification, testing, synthe-OTA has reviewed the evidence and concludes sis, and dissemination.overall that there is no coherent system of assess- Identificationing medical technologies. There is, however, anurgent need for such a system. The following are Emerging Technologiescapsule statements of OTA’s conclusions about OTA concludes that emerging drugs and de-the adequacy of the present system with respect vices are adequately and appropriately identified,to the four stages of technology assessment pre- but that emerging medical and surgical procedures

Ch. l—Introduction and Summary ● 1 7

could be better identified. Overall, however, theidentification of emerging technologies for assess-ment is not a critical weakness of the present as-sessment system.

New Technologies

OTA concludes that new drugs and devices areadequately identified for the purposes of assess-ment, but that new medical and surgical pro-cedures are not. The most pressing need is forsome routine mechanism, e.g., the reimbursementsystem, to identify new procedures before theyare widely adopted. The reimbursement systemmay be the prime candidate, because coverage andpayment decisions are critical points in the diffu-sion of many technologies. The priority-settingsystems of the institutes of NIH and of other Fed-eral research agencies (e.g., NCHSR) are adequateand appropriate for their respective mandates, butthere is not an adequate similar system to fulfillthe needs of operating agencies (e.g., HCFA, plan-ning agencies). Finally, sufficient mechanisms ofopportunity for identifying new technologiescould be developed. Medical specialty societiescould be helpful in this area.

Existing Technologies

OTA concludes that the system for identifyingexisting technologies in need of assessment is in-adequate. The most promising possibility for iden-tifying such technologies may be FDA’s postmar-keting surveillance of marketed products. In thecase of existing as well as new technologies, thepriority-setting procedures of Federal researchagencies may be adequate for those agencies’ re-spective needs; however, these procedures are notadequate for the needs of operating agencies suchas HCFA. And the operating agencies themselvesdo not adequately identify existing technologiesfor assessment. Medical specialty societies couldbe helpful in this area. Finally, NCHCT’S activitiesof identifying nationally important priority tech-nologies for assessment were valuable but are notcurrently funded. Thus, no organization is cur-rently performing this important task.

New Applications of Existing Technologies

OTA concludes that new applications of exist-ing technologies in need of assessment are not ade-quately identified. The most promising approach

would seem to be the use of the reimbursementsystem to link the diagnosis with the use of tech-nology. Medical specialty societies could be help-ful in this area.

Testing

OTA concludes that, in general, drugs and de-vices are adequately tested for safety and efficacyprior to being marketed. Medical and surgical pro-cedures, which often include the use of drugs anddevices within the practice of medicine, are notwell tested for either safety or effectiveness. Noclass of technologies is adequately evaluated foreither cost effectiveness or social and ethical im-plications. Finally, there is no organization whosemission it is to ensure that medical and surgicalprocedures are assessed for safety and efficacy orto evaluate medical technologies for cost effec-tiveness and for social/ethical effects.

Synthesis

OTA concludes that the synthesis phase of thepresent system of technology assessment is un-necessarily weak, within both the private andpublic sectors. Research evidence regarding thesafety, efficacy, and effectiveness from the use ofmedical technologies is seldom examined syste-matically and objectively. Federal agencies andprivate insurers and organizations set policies,guidelines, regulations, and/or make reimburse-ment coverage determinations, many of whichprofoundly affect the adoption and level of useof medical technologies. Yet, their decisions areusually based on informal, subjective, group-gen-erated norms which tend to support the statusquo. Formal, more objective techniques do exist,however, not only for evaluating research evi-dence but also for making decisions and setting

policy. These techniques could be used more oftento aid in better decisionmaking.

Dissemination

OTA concludes that better methods need to befound to communicate information about medicaltechnologies to health practitioners, health re-searchers, and health policymakers.


OTA also concludes that Government-gener-ated research reports, many of which may be im-portant to technology assessment, are not as ac-cessible as they could be. Finally, NLM’s missionand capabilities should be examined to determine

POLICY OPTIONS

The most important policy need is to bringforth a rational, systematic approach from thepresent multiplicity of agencies and activities topromote and coordinate medical technology as-sessment. Such integration could be accomplishedin any of several ways. The options listed belowand discussed at greater length in chapter 8 aredivided into two broad categories: legislative andoversight. OTA finds that there are relatively fewrealistic legislative options necessary for Congressto consider, primarily because there is alreadysubstantial power invested in the Secretary ofHealth and Human Services to develop a coherentsystem of medical technology assessment. Thus,in most of the deficient areas noted within thisreport, congressional oversight may be sufficient.

Legislative Options

1.

2.

3.

4.

5.

Sponsor or grant a charter to a private/publicorganization to undertake medical technologyassessment activities.

Maintain the authority of, and appropriatefunds for, NCHCT.

Change the statutes so that HCFA can selec-tively reimburse for experimental technologiesin return for clinical data.

Increase funding to train researchers in meth-odological and statistical principles.

Increase efforts to train health professionalsin methodological and statistical principles.

Oversight Options

6. Encourage the private sector to take the leadin assessing medical technologies.

7. Examine how Federal research institutes(e.g., NIH), agencies (e.g., NCHSR), and re-

whether more Government reports and nonserialliterature should be included in its data base, andwhether NLM should index articles differently forresearchers interested in technology assessments.

8.

9.

10.

11.

12.

13.

14.

search programs of operating agencies withinDHHS could identify technologies betterwhen setting research agendas; and how thePSRO program and the reimbursement sys-tem could be used to more advantage for iden-tifying technologies for assessment.

Continue to conduct oversight hearings con-cerning the duplication and fragmentation ofhealth-related data collection activities.

Examine the ability of operating agencieswithin DHHS (e.g., HCFA) to generate suf-ficient information for their own decisionsrelated to medical technologies, and examinethe extent to which the Secretary of Healthand Human Services utilizes the department’sother research arms (e.g., NCHSR, NIH) toprocure that information in a timely manner.

Examine the activities, plans, and potentialfor elements of DHHS (e.g., NIH) in utiliz-ing various research methods to determine theappropriate use of medical technologies.

Explore how research evidence could be bet-ter evaluated by Federal health agencies whenrecommending, setting, or implementinghealth policy.

Examine the disposition of federally generatedreports to determine how accessible and usefulthey have been both to private and public re-searchers and policymakers.

Examine whether NLM should include moreGovernment research reports and other non-serial literature in its MEDLARS data bases.

Encourage use of the powers vested in the Sec-retary of Health and Human Services to devel-op a coherent system of medical technologyassessment.

Ch. l—introduction and Summary ● 1 9

ORGANIZATION OF THE REMAINDER OF THE REPORT

Chapters 2, 3, and 4 discuss the types of infor-mation technology assessment seeks to generate,establish, and synthesize: namely, information onhealth, economic, and social/ethical effects. Themethods and mechanisms used to synthesize thatinformation are discussed in chapter 5. Chapter6 includes a description of the drug and deviceindustries, as well as a description of the innova-tion process for drugs, devices, and medical andsurgical procedures. It also presents an analysisof the effects that reimbursement and Federal reg-ulatory policies exert on the innovation process.A critique of current assessment policies and pro-cedures in chapter 7 summarizes the strengths andweaknesses in eachment and presentsChapter 8 presents

Eight appendixes

of the four stages of assess-OTA’s major conclusions.the policy options.

are included to serve as ex-tensive technical data supporting and amplifyingthe issues and conclusions of the report. Appen-dix A and B are a compendium of statistical datasources for medical technology assessment and acompendium of bibliographic data bases for med-ical technology assessment, respectively. Appen-dix C is a paper on the methods used in the evalua-tion of medical technologies. Appendix D de-scribes the innovation process for medical tech-

nologies, which five case studies in appendix E areintended to illustrate. Appendix F presents a pro-posed model for an Institute for Health Care Eval-uation. The method of study and the other vol-umes of this assessment are described in appen-dix G, and acknowledgments appear in appendixH. Appendix I is a glossary of acronyms andterms.

Throughout this study, OTA paid special atten-tion to the innovation process for medical technol-ogies, since a successful strategy of assessmentshould not, at a minimum, unnecessarily interferewith beneficial innovation and, to the extent possi-ble, should encourage useful innovation. OTAbelieves that none of the policy options presentedin this report would unduly restrict the innova-tion of medical technologies.

Three other volumes are being published in con-junction with this report: 1) PostmarketingSurveillance of Prescription Drugs, 2) MEDLARSand Health Information Policy, and 3) MedicalTechnology Under Proposals To Increase Compe-tition in Health Care. These volumes are briefly

described in appendix G. In addition, chapter 1of this report is available as a summary pamphlet.

2Information for Assessment

Knowledge is of two kinds. We know a subject ourselves, or we know wherewe can find information upon it.

—Samuel Johnson

— — . .

Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .,Information Needed for Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Health Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Performance Standards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Economic Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Social Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sources of Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Data Banks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Vital Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Insurance Claims Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Surveys of Patterns of Medical Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Publications and Bibliographic Data Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Libraries and Other Information Resource Organizations . . . . . . . . . . . . . . . . . . . . . . . . . .Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . $ . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Page2323232324242425252626272$30

Information for Assessment

INTRODUCTION

A primary consideration in developing a strat-egy for assessing medical technologies are theneeds of assessors for various types of informa-tion. This chapter is an introduction to the infor-mation needed for assessing health effects, per-formance standards, economic effects, and socialeffects of medical technologies, (Subsequent chap-ters will describe and critique the methods for ob-taining and synthesizing this information). Thesecond section of this chapter also provides anoverview of existing sources of statistical data forassessment, both public and private. The thirdsection describes existing biomedical literaturesources, including bibliographic data bases. Andthe final section discusses libraries, clearinghouses,and similar organizations which can provide in-

formation that is useful for technology assess-ment. To supplement the material in this chapter,a compendium of statistical data sources and acompendium of bibliographic data bases are in-cluded as appendix A and B.

The discussion of the sources of information formedical technology assessment in this chapter isnot an exhaustive one. Because of the complexi-ty and diversity of medical technologies, infor-mation from disciplines such as engineering, socialbehavior, and genetics is often needed to performa technology assessment. The focus in this chapteris on the significant sources of information thatis directly related to health.

INFORMATION NEEDED FOR ASSESSMENT

Health Effects

The information needed for a complete assess-ment of a technology’s health effects falls intothree broad categories: efficacy, effectiveness, andsafety. Efficacy refers to the probability of (usuallyhealth) benefit to individuals in a defined popula-tion from a medical technology applied for a givenmedical problem under ideal conditions of use(266). Effectiveness is similarly defined, exceptthat it refers to the probability of benefits underaverage conditions of use (266). Safety is a judg-ment of the acceptability of risk (i.e., the proba-bility and severity of an adverse effect) associatedwith the use of a technology.

Information regarding the efficacy of a technol-ogy is needed to establish, within some definedestimate of probability, whether the use of a par-ticular medical technology under “ideal” condi-tions can cause changes in patient outcome. Be-cause information concerning the efficacy of medi-cal technologies (under ideal conditions of use)

often cannot be generalized to wide populationsreceiving medical care in diverse settings, informa-tion on the effectiveness of such technologies(under average conditions of use) is also needed.

Most medical technologies have an element ofrisk associated with their use. Thus, any assess-ment policy needs mechanisms to determine theprobable risk, or, conversely, safety, of a tech-nology under various conditions of use, and thento weigh the risk with the expected benefit. Weigh-ing risks with benefits is necessary, because theconcept of risk is a relative one—i. e., a low riskis unacceptable if there is no expected benefit, buta high risk may be quite tolerable if the expectedbenefit is very high.

Performance Standards

Strictly speaking, only an improvement in pa-tient outcome can be considered evidence of atechnology’s efficacy or effectiveness. However,some technologies’ ultimate effect on health may

23


be so far removed from the technology’s use thatonly some intermediate outcome can reasonablybe assessed (270). For these technologies, informa-tion is needed concerning standards of perform-ance.

Such information is especially needed in thecase of medical devices, many of which performsome particular mechanical, electrical, or chemicalfunction ultimately intended to be related to achange in health status.

In an earlier report, OTA classified relevantoutcomes of diagnostic technologies to be: 1) tech-nical capability, 2) diagnostic accuracy, 3) diag-nostic impact, 4) therapeutic impact, and 5) pa-tient outcome (266). Since diagnostic technologiesare directly associated with intermediate outcomesand are often far removed from actual health out-comes, performance standards may be the mostmeaningful, as well as most easily obtained, meas-ures of assessment.

Economic Effects

Good economic data are essential to a systemof medical technology assessment. This is especial-ly true today, when one of the driving forces be-hind technology assessment is concern for the high

SOURCES

All medical

OF DATA

technology assessments requiredata. The chapters that follow discuss a varietyof methods of assessment which can be used tosystematically generate data about specific medi-cal technologies and to produce information thatis useful for setting policy. In addition, as de-scribed below, there are numerous systems whichproduce health-related data routinely.

The health statistics system of the United Statesis largely decentralized. Responsibility and au-thority for health statistical activities are dividedamong Federal, State, local, and independentagencies and organizations.

The compendium of statistical data sources inappendix A lists current public and private sourcesof data on the health of the population, the avail-ability and use of health resources, and health care

costs associated with the adoption and use of tech-nology. For the purpose of assessing medical tech-nologies, information is needed on the direct costsassociated with medical care usage: the cost of thephysician, the hospital, and medical supplies. Alsoimportant to consider are indirect costs: the costsassociated with the value of time lost while seek-ing and receiving medical care and, especially, inbeing sick. Indirect costs are often overlooked inassessments; when included, they are often meas-ured in terms of lost (or gained) wages.

Social Effects

The importance of addressing social and ethicalconcerns in the assessment of medical technologieshas been noted in a previous OTA report (270)and will be discussed at greater length in chapter4. Even though such concerns cannot ordinarilybe valued, they are often essential to the measureof worth of a medical technology. The distribu-tion of costs and benefits, respect for the auton-omy of individuals, and a myriad of other socialand ethical issues result from the introduction, ex-tension, or modification of medical technologies(19). These issues are important to consider inassessments.

expenditures (especially as they relate to the as-sessment of medical technologies). Information forthat compendium was obtained from a diversegroup of individuals, governmental agencies, andnongovernmental organizations through datafiles, published reports, and personal interviews.Since each sponsoring agency or organization col-lects data using its own methods and procedures,the health data described in the compendium varyconsiderably with respect to source, method ofcollection, definitions, and reference period(93,282).

The largest single participant in the U.S. healthstatistical system is the Federal Government. Theonly Federal agency established specifically to col-lect and disseminate data on the health of theAmerican people is the National Center for Health

Ch. 2—Information for Assessment ● 2 5

Statistics (NCHS). Since 1960, NCHS has playeda major role in the development of national healthstatistics policies and programs. The NCHS Divi-sion of Vital Statistics collects natality, mortal-ity, marriage, and divorce statistics from the indi-vidual States and registration areas. In addition,NCHS conducts several general purpose surveysthat provide statistics about the health status ofthe U.S. population. NCHS also has the primaryadministrative responsibility for the CooperativeHealth Statistics System, a joint Federal, State,and local program for the collection of healthdata. ’

An OTA study on Federal health data collec-tion systems in 1979 found that the system for col-lecting, storing, processing, and disseminatinghealth care information had been affected by therapid growth of the Federal role in health care(282). At the time of that study, the Public HealthService alone administered 153 individual dataprojects; the Health Care Financing Administra-tion operated at least 13 large statistical projects;and several other agencies and departments out-side the Department of Health and Human Serv-ices (DHHS)* also conducted major health statis-tical activities. OTA concluded that these numer-ous data bases were uncoordinated and in manycases duplicative (282).

Four types of medical data sources are discussedfurther below: 1) data banks, 2) vital statistics,3) insurance claims records, and 4) surveys of pat-terns of medical practice (197). Not every medicaldata system necessarily fits exclusively into oneor another of these categories, but this breakdownof categories is useful for evaluating generalstrengths and weaknesses of different existing col-lections of data.

Data Banks

A potential source of information about pa-tients, their characteristics, and their responses tothe use of different biomedical technologies dur-

I Additional information concerning NCHS data systems can beobtained from Data Systems of the National Center for Heafth Sta-tistics (93).

● Then the Department of Health, Education, and Welfare(DHEW).

ing their care are medical data banks. * Medicaldata banks, which are often computer based, areusually created by establishing a common termi-nology or vocabulary to describe a patient’s clini-cal history and then entering observations on pa-tients as events occur; sometimes, a data basemanagement system is used to ensure accuracy,security, and easy entry and retrieval of observa-tions. By extension, a medical data bank networkcontains the clinical history of large numbers ofpatients (from multiple centers) described in a uni-form manner (218).

A medical data bank that contains extensive,comprehensive, reliable, longitudinal data on anumber of patients can provide two importantfunctions in assessing medical technologies. First,it can serve as an instrument for collecting thebaseline and followup data on patients who havebeen subjected to a treatment. Second, the patientson whom this data has been collected can func-tion as their own historical control group, whichallows the investigation of the health effects asso-ciated with the treatment (93). Data banks canalso be used to provide physician practice pro-files and to assess the relative values of diagnosticand therapeutic choices.

Medical data bank demonstration programshave been funded by at least two Federal agen-cies: the National Institutes of Health (NIH) andthe National Center for Health Services Research(NCHSR). NCHSR has supported a chronic coro-nary artery disease data bank program at DukeUniversity and a rheumatic disease data bank pro-gram at Stanford.

Vital Statistics

Vital statistics such as those compiled by NCHSare of potential use in analyzing the safety andefficacy of medical technologies. When observa-tional data are used to draw inferences concerning

● Although the terms are sometimes used interchangeably, a med-ical data bank differs from a registry (e. g., that used for cancertumors). In a registry, data typically are abstracted from a specificdocument (e.g., medical record or death certificate) using criteriathat are applied retrospectively, rather than by adhering to a com-mon terminology in the prospective collection of data (56,218). Inpractical terms, registries generally cover discrete political or geo-graphic areas (267). In this chapter, registries are discussed underthe heading “Surveys of Patterns of Medical Practice. ”

98-144 (1 - 82 - 3


the safety and efficacy of medical technologies,however, there is a need to protect against biasin selecting the sample of records from which in-ferences will be made (197).

Vital statistics data can have an important func-tion in the area of record linkage. The NationalDeath Index (NDI) recently put into operation byNCHS is a case in point (267). Historically, be-cause there was no integration of records for thecountry as a whole, no mechanism had existedat the national level to determine whether a per-son has died. The NDI is intended to serve thatpurpose.

The NDI is designed to provide medical andhealth researchers with probable fact of death, thedeath certificate number, and the location of thedeath certificate when supplied with a minimumset of identifiers (generally the person’s name andsocial security number or date of birth). A re-searcher may then contact the registration areawhere the possible match has occurred to obtainthe death certificate or the required information.The NDI will be of immediate use in ongoing long-term studies which include mortality. Beebe (24)has described this index as the most important re-cent advance in making vital statistics accessibleto researchers (24). The National Cancer Insti-tute’s Surveillance, Epidemiology, and End Results(SEER) program plans to use the NDI to deter-mine deaths of all persons in the SEER registries.SEER often loses track of people who move outof SEER areas before they die. Use of NDI shouldreduce the number of people lost to followup bySEER and provide better information aboutsurvival.

Insurance Claims Records

Large data sets have been compiled by third-party payers in both the private and public sec-tors. Blue Cross/Blue Shield, for example, collectsdata on its subscribers—about one out of threeAmericans, or more than 80 million people—aswell as on subscribers and participants in variousother programs that it administers (includingMedicare and Medicaid in some regions) (197).

On the Federal side, passage of the Medicareand Medicaid legislation in 1965 created the needto establish a national yet decentralized adminis-

trative mechanism to pay for services to benefici-aries of these programs and to gather statistics formanaging the programs. Systems needed to paythose bills were designed to provide information(primarily determinations of patient eligibility, thecompleteness of the claim forms, the appropriate-ness of length of stay in hospitalizations, and in-formation on charges) for use by fiscal intermedi-aries in making interim payments. Although noneof the data are collected solely for statistical pur-poses, the resulting information is useful for thatpurpose (198).

Surveys of Patterns of Medical Practice

Disease-specific (or procedure-specific) regis-tries, hospital discharge data systems, and Federalstatistical surveys of selected medical practices area fourth general source of data for evaluatingmedical technologies.

A recent and somewhat prominent example ofa procedure-specific registry is the voluntary regis-try established by the National Heart, Lung, andBlood Institute for physicians using the new tech-nique percutaneous transluminal coronary angio-plasty (PTCA). (The PTCA registry is discussedfurther in app. E.) Because it has achieved fairlyhigh physician participation, this registry permitsthe collection of data for evaluation of PTCA atan early stage of the technology’s diffusion, andat a cost that may be less than that of a random-ized clinical trial (197).

Several hospital discharge abstract systemsemerged in the early 1950’s to provide summaryinformation abstracted from hospital medical rec-ords about patients and their episodes of illnessin short-term general hospitals. Although theytended to be established by independent organiza-tions, many were similar in their origins, systemcharacteristics, available data items, and sponsor-ship (which was generally private, nongovern-mental).

The earliest system, the Professional ActivityStudy, developed in 1953 with support from theW. K. Kellogg Foundation. This system is oper-ated by the Commission on Professional and Hos-pital Activities (CPHA), a nonprofit corporationin Ann Arbor, Mich., sponsored by the Ameri-can College of Physicians, the American College

Ch 2—Infromation for Assessment ● 2 7

of Surgeons, the American Hospital Association,and the Southwestern Michigan Hospital Council.The purpose of the Professional Activity Studyis to link medical and surgical procedure rates within-hospital mortality. The CPHA sample is a largeone (approximately one-fourth of all hospitalizedpatients and one-third of the hospital dischargesin the United States), and although not a randomsample (it is based only on those hospitals thatsubscribe to their service), it has historically beenrepresentative of important hospital characteris-tics.

There are also an additional estimated 18 to 20private, nonprofit hospital discharge abstract sys-

tems throughout the United States that processhospital utilization data for about half the hospi-tals in the country, representing over 20 milliondischarges yearly (198).

The Federal counterpart to these private, non-governmental systems is the National HospitalDischarge Survey initiated in 1964. Administeredby NCHS, this survey collects information on thecharacteristics of patients, lengths of stay, diag-noses, surgical operations, and the patterns of useand care. Only short-stay hospitals with six ormore beds and with an average length of stay forall patients of less than 30 days are included inthe sample (267).

PUBLICATIONS AND BIBLIOGRAPHIC DATA BASES

Information concerning medical technologies isoften found in primary publications such as jour-nals, books, Government reports, technical publi-cations, and patents. Because of a dramatic risein the number of publications in the field of medi-cal technologyto the primarydifficult. As ae.g., catalogs,stracts—which

assessment (13), however, accessliterature is often confusing andresult, secondary publications—indexes, bibliographies, and ab-facilitate access to primary litera-

ture sources are increasingly important in the in-formation transfer cycle. Secondary publicationscan also provide readers with superficial informa-tion about subject matter. (A bibliography oncomputed tomography, for example, can allowa reader to crudely approximate the state of theart solely by scanning the listed titles. )

Secondary publications are now increasinglyavailable in the form of bibliographic data basesthat can be read by a computer. The informationcontained in many of the bibliographic data bases(i.e., references to literature) also can be obtained“on-line.” (A person at a computer terminal cancarry on a dialog with a computer, directing itto locate, retrieve, and then display the informa-tion at the terminal or print the information onpaper. ) The growth of machine-readable biblio-graphic data bases in recent years has been extra-ordinary. In the United States alone, the numberof data bases increased from 301 in 1976 to 528

in 1979, a 75-percent increase (394). Accompany-ing the growth in number of data bases has beena corresponding growth in use. The number ofrequests for information searches for individualdata bases (on-line) grew from 700,000 in 1974to an estimated 4 million in 1979 (395).

Medical technology assessment often may re-quire information from many subject areas. De-pending on the medical technology, the type ofassessment, and a myriad of other factors, infor-mation may be sought in such diverse disciplinesas biomedicine, law, finance, economics, and soci-ology. Such information is often obtained fromcomputer-readable data bases. A common needin medical technology assessments is for informa-tion from the field of biomedicine. Informationon biomedicine is found in numerous data bases.Worldwide, over 90 computerized data bases con-tain information on medicine alone (394). Al-though there is some overlap in the contents ofmany of the biomedical data bases, no one database exactly duplicates another; each is unique inmany aspects (e.g., contents, arrangement, andindexing philosophy). A descriptive list of bio-medical-related bibliographic data bases of signifi-cance that are useful for medical technology as-sessment, along with the creator and vendor, ap-pears in appendix B.

The major on-line service organizations, bothpublic and private, that provide access to biomed-


ical data bases in the United States are the Nation-al Library of Medicine (NLM), which is a part ofNIH, Bibliographic Retrieval Service, DIALOGInformation Service, Inc., and System Develop-ment Corp.

NLM provides access domestically and inter-nationally, both to data bases in the biomedi-cal field that are created and maintained solelyby NLM and to data bases that it sponsors orproduces in collaboration with other Govern-ment entities and private organizations, bymeans of its computerized bibliographic retrievaland technical processing system MEDLARS(Medical Literature Analysis and RetrievalSystem). 2 Its major biomedical data base isMEDLINE (MEDLARS On-line), which contains600,000 references to biomedical journal articlespublished in the current and 2 preceding years.Tapes of MEDLINE and other MEDLARS’ databases, including TOXLINE* and HEALTH,** can

also be leased from the National Technical Infor-mation Service (NTIS) of the U.S. Department ofCommerce.

Bibliographic Retrieval Service, DIALOG Infor-mation Service, and System Development Corp.are commercial firms that do not produce databases, but enter into licensing agreements withthe data base producers which permit thesefirms to mount producers’ data tapes on their owncomputer facilities, adapt the tapes to their ownsoftware (i. e., computer instructions), and sellon-line access to subscribers. These commercialvendors typically sell access to a broader groupof data bases than just biomedical bibliographicones. The commercial vendors sell access toMEDLINE and other MEDLARS data bases, aswell as to health-related data bases such asEXCERPTA MEDICA and BIOSIS PREVIEWSwhich are produced in the private sector. Theyalso sell access to non-health-related data bases.

‘This system is the topic of an OTA technical memorandum en-titled MEDLARS and Health Information Policy (276).

“Toxicology Information On-line. This is a collection of about600,000 references from the last 6 years on published human andanimal toxicity studies, effects of environmental chemicals and pol-lutants, and adverse drug reactions.

● *Health Planning and Administration. This data base containsabout 200,000 references to literature on health planning, organiza-tion, financing, management, manpower, and related subjects.

LIBRARIES AND OTHER INFORMATION RESOURCE ORGANIZATIONS

The increase in the number and diversity of in-formation products and services has been accom-panied by an increase in the diversity of organiza-tions that provide access to them.

Among the more important of these are healthscience libraries, which have for many years ac-quired, organized, and provided literature on bio-medically related areas. As of 1979, over 2,700public and private health science libraries wereidentified in the United States (74). The librarieswere sponsored by medical schools, professionaland vocational schools, business and industrialorganizations, research organizations, societiesand foundations, hospitals, area health educationcenters, health maintenance organizations, andhealth planning organizations (74).

In addition to having access to journals, books,and other print and nonprint materials, manyhealth science libraries have access to biomedical

and other computerized data bases. For example,about 1,500 health science libraries have terminalsconnecting them directly to NLM’s MEDLARSsystem for computerized searches. * In addition,many other health science libraries provide in-direct access to MEDLARS by referring requeststo facilities with terminals. Indeed, informal andformal networks and the use of telephones, com-puters, and interlibrary loans have broadened ac-cess to information resource organizations con-siderably.

Although there are other types of informationresource organizations which have provided, andcan provide, information that is relevant to med-

● The number and types of institutions which obtain access to themedical and other literature through the major commercial retrievalservices (i.e., Bibliographic Retrieval Service, DIALOG Informa-tion Service, and System Development Corp. ) are not known,because the information is proprietary.

Ch. 2—Information for Assessment ● 2 9

ical technology assessments, it is difficult to getcomprehensive information about them—andeven their typology is elusive and fluctuating.

For example, the first conclusion of a 1980 Na-tional Science Foundation (NSF) study and surveyis (138):

, . . that it is difficult to obtain comprehensiveinformation on Federal S&T [science and technol-ogy] information centers. Even the National Re-ferral Center at the Library of Congress is notprovided accurate, timely, and complete informa-tion on federally supported information centers.

NSF restricted its definition of an information cen-ter in the 1980 survey to an organization whoseprimary function was:

. . . to store, retrieve and distribute scientificand technical information. Both textual and nu-meric data information of a primary (e.g., infor-mation analysis) nature were included. Organiza-tions such as agency libraries, which were in-volved in serving parent organizations, were ingeneral excluded.

NSF identified 55 “science and technology infor-mation centers” in DHHS (then DHEW), includingNLM.

A more recent study prepared for DHHS identi-fied and categorized 157 “human resource infor-mation organizations” (6). Forty-one had a healthfocus. Of the 157 organizations, 98 were fundedby Federal agencies, 43 by private organizations,and the others by academic institutions or Stateand local governments, The majority of federal-ly sponsored organizations were funded byDHHS, but others were supported by the Depart-ments of Education, Transportation, Housing andUrban Development, Energy, Labor, Justice,Commerce, Agriculture, and the CommunityServices Administration. Of the 157 organiza-tions, 72 were classified as “clearinghouses,” 76were classified as “other types of informationresource organizations, ” and 9 could not be clas-sified because of incomplete information. (Theother types of information resource organizationsincluded special libraries, document depositories,information analysis centers, information refer-ral centers, resource centers and networks, andtechnical assistance centers. )

Clearinghouses, which were the focus of thestudy, were distinguished on the basis of particu-lar activities and functions. Through a variety ofuser services, these organizations perform threeimportant tasks: the collection, the analysis, andthe dissemination of information. They identify,select, acquire, process, and sort documents andother materials, and provide “locator tools” (e.g.,indexes) to this collection. They also synthesizeand digest this information to guide users to thespecific data in the collection that best serve theirneeds. A wide range of clearinghouse services,from bulletins and announcements to bibliogra-phies and handbooks, can lead users to the infor-mation they seek. Some clearinghouses tend tocollect unpublished Government reports, projects,descriptions, speeches, and other types of “fugi-tive” literature that is hard to get elsewhere.

For example, the National Health Planning In-formation Center, a clearinghouse in the Bureauof Health Planning of the Health Resources Ad-ministration, was created to provide informationfor the analysis of issues and problems related tohealth planning and resources development. Thiscenter acquires, screens, and stores informationon published journal articles, books, and otherdocuments about health planning and resourcesdevelopment. Besides published reports, thecenter seeks unpublished reports, conference andproceedings papers, bibliographies, publicationlists, and appropriate audiovisuals and micro-films. Its collection currently includes some 20,000documents on a wide range of general subjectspertaining to health planning (e. g., health caretechnology and equipment impact, health careutilization). To facilitate the dissemination of in-formation to health planners, the center issuesselected publications in three series: Health Plan-ning Methods and Technology, Health PlanningInformation, and the Health Planning Bibliogra-phies (92,123,124). Finally, the center is now col-laborating with NLM to include the center’s database in MEDLARS’ HEALTH data base (276).

A second information clearinghouse is projectShare, which was created by DHHS to providea central, systematic source of information for im-proving the management of human services. Tar-

—

30 . Strategies for Medical Technology Assessment

geted primarily at State and local officials, Proj-ect Share acquires, announces, and makes avail-able documents relevant to the planning, manage-ment, and delivery of human services. Types ofinformation collected and disseminated includepublished and unpublished papers, theses, re-search reports, bibliographies, technical reports,operating manuals, and conference proceedings.Besides providing a source of documentary andreference services, the clearinghouse analyzes andsynthesizes reports and other documents describ-ing human services program activities, conductsoriginal research, and publishes state-of-the-artliterature and state-of-knowledge reports (121).

The central repository for scientific and techni-cal information generated by federally funded

CONCLUSION

The information needs for the assessment ofmedical technologies are both broad and deep, re-quiring the involvement of diverse disciplines. Asthe discussion in this chapter indicates, there ex-ist numerous resources that are useful for assess-ing medical technologies. An earlier OTA studynoted that a primary weakness of health-related

research and demonstration projects is NTIS. TheNTIS collection exceeds 1.2 million titles. Mostare drawn from the Departments of Energy andDefense, and approximately 10 percent have comefrom DHHS–5,700 reports from DHHS in 1979.In addition, NTIS has working relationships forthe computerized processing of documents withat least three entities within DHHS: the NationalCancer Institute, Project Share Clearinghouse,and the Bureau of Health Planning within theHealth Resources Administration. NTIS is catego-rized as a clearinghouse, but the size of its collec-tion and its function as the permanent repositoryof Federal technical information documents set itapart from all other clearinghouses except NLM(84,85).

data sources is the lack of coordination betweenthem (282). This problem persists.

The next three chapters consider systematicmethods for gathering and synthesizing informa-tion concerning the health, economic, and socialeffects of medical technologies.

3.Evaluating Health and

Economic Effects

Ignorance never settles a question.

— Disraeli

Contents

Page

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33Evaluating Health Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Experimental Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Randomized Clinical Trials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Observational Studies.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Cohort Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Case-Control Design... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38Evaluating Economic Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Opportunity Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Marginal Valuation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Joint Production Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40R&D Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40Overhead Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40Cost v. Prices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40Discounting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Evaluating Health and Economic Effects

INTRODUCTION

Evaluating health and economic effects of med-ical technologies is central to any assessment; in-deed, some would argue that evaluations of healthand economic effects are the essence of an assess-ment.

The first section of this chapter discusses healtheffects. The main technical decision to be madewhen testing for the health effects of a technologyis which study design is most appropriate. Thissection describes the study designs available and

EVALUATING HEALTH EFFECTS

Despite recent attention to the economic andsocial impacts of medical technology, the mostcritical aspect of the use of medical technologiesremains their effect on health. An evaluation ofhealth effects may examine efficacy (or effec-tiveness), safety, or both. Efficacy* is the healthbenefit as measured under controlled conditions(such as those existing in a randomized clinicaltrial). Effectiveness is the benefit of technologyunder average conditions of use. Efficacy or ef-fectiveness generally measure the intended effectsof the use of a technology. Safety is a judgmentof the acceptability of the risk** involved in usinga technology.

There are many similarities between efficacyand safety —e. g., ‘both are relative concepts andthus are discussed in terms of probabilities. Veryimportantly, however, their measurement may re-quire different study methods. They differ in sev-eral key factors. In assessing efficacy, a study isusually oriented to a limited number of specific

● OTA defines “efficacy” as “the probability of benefit to in-dividuals in a defined population from a medical technology ap-plied for a given medical problem under ideal conditions of use”(266).

* *Risk is a measure of the probability and severity of harm tothe health of individuals in a defined population associated withuse of a medical technology applied for a given medical problemunder specified conditions of use (266).

compares the designs presented in terms of theirvalidity. Additional material on methods used toevaluate health effects is presented in appendix C.

The second section of this chapter concerns eco-nomic effects. It is primarily drawn from portionsof a previous OTA report entitled The Implica-tions of Cost-Effectiveness Analysis of MedicalTechnology (270). This section provides the readerwith a brief discussion of the issues involved whenevaluating economic effects.

benefits. The measurement of safety, however,usually involves a study design that is able to iden-tify a broad range of risks; such risks are oftenunknown or unexpected, they may occur far inthe future, and they may affect only a smallpercentage of individuals. These factors imply thatefficacy and safety are not simply the plus andminus columns of a single measure. Each requiresseparate attention, although judgments of the im-portance of either a benefit or a risk should onlybe made in relation to the other.

There are methodologic principles that guidethe design, conduct, and interpretation of any par-ticular investigation. Specific methods for eval-uating health effects of technologies are describedbelow. Each method has its strengths, weaknesses,and limitations for detecting favorable or unfav-orable outcomes associated with a technology.

Of particular concern in research design andanalysis is the validity of the findings, whichvaries with the study design. Validity refers to theextent to which a situation (as observed or eval-uated by other criteria) is reflective of the “truesituation. ” Four components of validity have beendescribed (68). Internal validity refers to whetherthe observed effects of a medical technology,under the conditions of the study, are attributableto the technology and not to some other factors.

33


Statistical conclusion validity, which is a subsetof internal validity, refers to the appropriatenessof statistical tests and their ability (or power) todetermine whether observed effects could be ex-plained by chance fluctuations and to detect truedifferences in performance of the technologyunder study. External validity refers to thegeneralizability of the observed effects to otherpatient populations, settings, or conditions. Con-struct validity refers to the adequacy of the theorythat an investigator has about what makes thetechnology effective.

The appropriateness of a particular study designis dependent on the purpose of the study, themethods available, the effects to be measured, andthe technology’s pattern of use. The choice ofmethod is also influenced by other factors suchas ethical concerns, limits on the n-umber of par-ticipants available for study, the need for timelyresults, and available budget.

The discussion that follows is focused on selectstudy designs which are commonly used to assessthe health outcomes of a technology.

Experimental Studies

Experimental studies are characterized by theintentional application of a technology to a studypopulation, and subsequent observation of effects.These studies must be carried out prospectively.They are frequently used prior to the dissemina-tion of a technology, but can also be employedafter the technology has diffused.

Randomized Clinical Trials

Randomized clinical trials (RCTs) are con-sidered the most definitive experimental methodfor evaluating the efficacy or health benefits ofa technology (60,148,187). An essential elementof an RCT is randomization. Patients in an RCTare randomly assigned to one of at least twogroups: one or more study groups, in which sub-jects are exposed to the experimental treatments,and a comparison group, in which the subjectsare exposed to the control condition. The controlcondition can be either no treatment, the standardtreatment (for comparison with a new treatment),or a variation (e.g., a different dosage) of the ex-perimental treatment. The basic question to be

answered in an RCT is: Are the effects observedin the experimental group also observed in thecomparison group? If the answer is essentially“no,” the effects observed in the experimentalgroup can be attributed, within the limits of prob-ability, to the treatment technology.

RCTs are a family of designs that vary in sizeand complexity. The number of treatment con-ditions (e.g., dosage levels) can vary, as can thesize of population tested and the statistical power*of the study. Small RCTs may be performed earlyin the development of a technology to demon-strate or test the efficacy of the technology’s in-novative elements. Large-scale, multicenter trialscan be conducted at a later stage in the develop-ment of a technology to establish its efficacy andsafety across a large population and in diverse set-tings (266), as well as to increase the statisticalpower that results from a larger sample size. Amajor goal of the multicenter trial is to improveexternal validity in regard to larger populations.

Sometimes a favorable or unfavorable outcomeis observed (i.e., a participant gets better or worse)because the participant believes that the treatmentwill work or believes the treatment is harmful.This “placebo” effect,** psychologically relatedbut nonetheless real, results in a change in the par-ticipant’s condition. Further, the effect may be in-fluenced by the investigator’s expectations. To re-duce potential bias from the placebo effect, treat-ment can be offered under conditions where theparticipant (“blinding”) or both the participantand the health care provider (“double blinding”)are not aware whether the participant is given theexperimental or the control treatment. Anotherlayer of “blinding” is added when the personanalyzing the data is not told which group is theexperimental and which is the comparison. Thatperson may be a statistician, but frequently is amedical specialist, and also may be the provider.

The principal advantage of RCTs is that theyhave high internal validity, i.e., they permitrelatively unambiguous conclusions as to whether

● The “power” of a study is the probability of its detecting an ef-fect (of technoloW being tested) when one actually exists. The greaterthe power, the less likely one is to incorrectly reject an effective tech-nology.

● *Although the placebo effect is discussed here under RCTS, itis not peculiar to such studies.

Ch. 3—Eva/uating Hea/th and Economic Effects ● 3 5

the observed effects of a treatment under the con-ditions of the study are due to the technology orsome other factor(s). Randomization protectsagainst potential selection bias in assignment ofsubjects to experimental and comparison groups.Within the limits of sampling error, the only dif-ference between the groups is that the experimen-tal group is given the treatment under study andthe comparison group is not. Therefore, dif-ferences in outcome can be attributed to the dif-ferences in treatment, with a known probabilityof error due to chance.

Although well-designed RCTs are generallyhigh in internal validity, they do not necessaryresolve the problem of external validity (68). Ex-ternal validity is usually established only whenlarge heterogeneous samples of participants aretested under a variety of circumstances, typical-ly across a number of studies, or through largemulticenter RCTs with carefully selected popu-lations.

A disadvantage of RCTs is that they can be dif-ficult to carry out in settings such as hospitalclinics and physicians’ offices and can be especiallydifficult for technologies that are already widelydiffused and perceived as being effective (253,401). In such situations, administrators and clini-cians may be reluctant to make the changes in pol-icies and procedures needed to conduct an RCT.Preexisting conclusions on the treatment beingevaluated are a major obstacle to conductingRCTs (159). Such conclusions may subvert therandomization process. For example, the assess-ment of high-oxygen environments as a cause ofblindness in premature infants was impeded bywell-intentioned nurses (346). In one study nursesraised the oxygen level for the experimental groupof infants in the belief that the low-oxygen envi-ronments were harmful. In another study, it wasnecessary to implement the treatment only par-tially, until evidence of the harmful effects of ox-ygen were more apparent.

RCTs are generally considered more complexand expensive to conduct than other types ofstudies. The decision to initiate an RCT shouldbe based on strong evidence that the hypothesisunder consideration merits the possible expenseand effort of conducting such a study.

Finally, RCTs maybe of limited utility in study-ing safety. As indicated above, safety is a measureof risk, and risks may occur after a considerabletime, may occur infrequently, and may be unex-pected. These types of effects maybe difficult toplan for and measure by an experimental study,thus necessitating the consideration of other formsof assessment.

Observational Studies

Observational studies may be valuable in gen-erating or testing hypotheses about the health ef-fects of a technology once the technology is widelydiffused. They also may be considered in situa-tions where experimental studies are inappropriateor impossible to conduct. The common elementin all observational study designs is that the in-vestigator does not control the application of thetechnology under study. The division of a popula-tion group into “cases” and “controls” or “ex-posed” and “unexposed” occurs through mecha-nisms unrelated to carrying out a study, such asthe treatment preference of a physician (e.g., inthe care of a stroke victim) or self-determination(e.g., in the choice of a method of contraception).Although the internal validity of observationalstudy designs generally does not match that of ex-perimental study designs, observational studiesmay allow evaluators to rule out competing ex-planations for the observed effects.

Because the investigator does not employ thedeliberate or intentional modification of condi-tions between the study groups, steps must betaken to try to eliminate any potential bias inselecting the study groups. The investigator musttry to control for bias, which may result whengroups differ with respect to “confounding var-iables” (age, sex, health status, or any othercharacteristic which may account for observedoutcomes). However, in nonrandomized studiesthe extent of selection bias cannot be known, andthus the effectiveness of the steps taken to min-imize bias also cannot be known with certainty.

Cohort Design

Cohort studies begin with a “naturally occur-ring” population, or a sample thereof, chosen by


the investigator as defined by: 1) some criterionor combination of criteria such as specific age,location, time period, etc., and 2) exposure ornonexposure to a technology. The population isfollowed over time to observe the differences inhealth status between the exposed and unexposedgroups. In a “prospective cohort study,” thepopulation is identified at the time of exposureand health status is assessed at a future time. Ifthe population is identified after the exposure hasoccurred and the health status of the individualsis assessed at the present or a future time, it istermed a “retrospective cohort study. ”

A 1978 study by Roos and colleagues (319) em-ploys a retrospective cohort study in assessing theeffectiveness of tonsillectomy (with or withoutadenoidectomy) in preventing subsequent epi-sodes of respiratory illness. This study illustratesmany of the features that can be built into cohortdesigns to minimize the effects of confoundingvariables in an attempt to improve internalvalidity.

Roos and colleagues used medical claims andpatient registration data provided by theManitoba Health Services Commission to iden-tify the population from which the cohorts weredrawn. Two operated groups were created: oneconsisting of all patients operated on for tonsillec-tomy only during January 1973; the other con-sisting bf all patients operated on for tonsillectomyonly or tonsillectomy plus adenoidectomy for allof 1973. In addition, two comparison (nonoper-ated) groups were formed: children under the ageof 14 whose records indicated evidence of tonsillarillness but no tonsillectomy operation during a3-year period (1972-74); the other consisting ofnonoperated siblings of operated patients.

Analysis of the data indicated that, on the aver-age, the surgical procedures averted about oneepisode of respiratory illness per child over the2 years following surgery. The greatest benefit ac-crued to the patients who had experienced thegreatest number of episodes of respiratory illnessin the year preceding surgery.

Had the investigators not taken measures tocontrol for confounding variables, the observedresults might have been explained by factors otherthan the surgical procedures. Specifically, matura-

tion (changes in health with age) was not believedto affect the results since the groups were similarin age and sex. The effects of “history,” temporalevents such as new health practices, was mini-mized by the use of concurrent controls. “Localhistory,” the effect of family preference or physi-cian factors, such as a predisposition toward sur-gery, was also minimized by including the siblingcontrol group. These design features presumablyminimized threats to the internal validity of thestudy.

Postmarketing surveillance, the mechanismused to detect unsuspected adverse drug reactionsafter a drug is marketed, generally employs theprospective cohort design. * Typically, a user pop-ulation of a particular drug is entered into a reg-istry and followed over time for various “healthevents. ” Rates of such events are compared withrates in a nonuser population. Thus, unusual med-ical events may be associated with use of the drug.These studies, because they use relatively largepopulations, may detect associations betweendrug use and unusual adverse reactions which aregenerally not detected in small population studiessuch as those used in premarketing assessment ofthe drug.

Historical Controls.—Innovations in medicineoften diffuse so rapidly and completely that newtechnologies or new treatment variations may be-come standard in a fairly short time. It may beimpossible to assess the long-term outcome of thetechnology in a conventional prospective cohortstudy, for lack of a group of patients not giventhe new treatment. In these instances, if research-ers are to conduct a study, they may use a variantof the cohort design which employs historical con-trol groups, i.e., patients treated prior to the in-novation. The use of historical controls, however,adds a serious limitation to the cohort study de-sign: change, other than the change in the treat-ment being assessed, is constantly occurring inhealth care, and such change may affect the in-ternal and construct validity of the study. Yetdespite their limitations, historical control studiescan be useful, particularly if the temporal gap be-tween control and treated groups is small, since

*For a detailed discussion of postmarketing surveillance, see OTA’Sreport entitled Postmarketing Surveillance of Prescription Drugs(281).

Ch. 3—Evaluating Health and Economic Effects ● 3 7

the likelihood of some validity problems is re-duced. Great care, however, should be exercisedin their use and interpretation.

One of the first studies assessing the efficacyof coronary care units (CCUs) (314) relied on his-torical controls. The first 200 patients with acutemyocardial infarction (heart attack) admitted tothe CCU at Royal Perth Hospital formed one co-hort, and the last 200 patients treated for acutemyocardial infarction prior to the opening of theCCU formed the comparison group. Althoughmortality rates by severity of infarction weresomewhat better for patients treated in the CCUthan for patients in the historical comparisongroup, the difference was not statistically signifi-cant, Because the patients were all treated at thesame hospital (though at different times), the twogroups were similar in many respects: the basepopulation was similar, the hospital staff wasbasically the same, and hospital records, on whichthe study relied, were similarly kept. The validi-ty of the study (i. e., did the CCU produce the ef-fect), however, was compromised by the introduc-tion at about the same time as the CCU of a num-ber of other therapeutic measures (e.g., lidocaineand atropine to treat and prevent arrhythmias andthe use of transvenous pacemakers to treat con-duction blocks). This study of CCU efficacy wasnot definitive, and the value of CCUs, themselvesnot strictly defined entities, is still an open ques-tion. However, the study did raise enough ques-tions to spawn further investigations.

Studies of the use of high-dose methotrexatechemotherapy for treating osteosarcoma, a formof bone cancer, * illustrate a case where the useof historical controls so compromised the studythat erroneous results were obtained. Followingthe development of chemotherapy in the early1970’s, researchers began to experiment with waysto improve its apparent effectiveness. One ap-proach was to treat patients with drugs beforetheir cancer had spread. Studies using historicalcontrols indicated that nearly half the patientstreated in 1970 lived 2 years without a recurrenceof the disease, compared to only 20 percent of agroup of patients treated in 1960. However, thechange in therapy from 1960 to 1970 was accom-

*G. B. Kolata, “Dilemma in Cancer Treatment, ” Science 209:792,1980.

panied by other changes in diagnosis, treatment,and patients. For example, the patient mix un-doubtedly changed over the 10-year span so thatpatients with the worst prognoses (i.e., metastaticcancer) no longer constituted the majority of thosetreated, rendering the cohorts noncomparable.One can have little confidence in the results ofa study which seems to show the chemotherapyefficacious, when the confounding effects of theother secular changes that occurred between 1960and 1970 could account for the effects of the treat-ment in analyzing the study data. In particular,the Mayo Clinic found that patients not treatedwith chemotherapy in the later time period alsohad higher survival rates.

In summary, cohort studies using historical con-trols serve a limited but sometimes helpful pur-pose. They may allow for an inexpensive prelim-inary inquiry as to the value of a technology,capitalizing on existing data. However, they sel-dom, if ever, provide definitive information onwhich to make decisions about the value of thetechnology.

Case-Control Design

Case-control studies compare a group of peo-ple with a disease (or other outcome event), cases,to another group without the disease, controls,and then determine whether they differ in theirprevious exposure to a presumed causal agent(e.g., a drug). These studies are retrospective innature, the exposure having occurred prior to theidentification of cases and controls.

Substantial biases are possible in case-controlstudies. The most serious result from the selec-tion of an inappropriate comparison (control)group. Because it is not possible to achieve com-plete comparability between the comparisongroup and the case group, controversies about theinterpretation of case-control studies generally

revolve around the question of whether or not thecontrols are an appropriate representation of thepopulation that gave rise to the cases. Other prob-lems also exist: the retrospective nature of themethod implies no control over the treatment,forces reliance on individuals accurately recall-ing past events, and forces reliance on recordsthat were kept for reasons other than those of car-rying out a study.


The studies of a possible association of estrogentherapy with endometrial cancer illustrate theproblems encountered in using case-control de-signs. The major dispute among researchers (191,196) concerns the appropriateness of the controlgroup. The traditional approach, to compare pa-tients with endometrial cancer to control patientswith other genitourinary cancers, has found a con-sistently high association between the use ofestrogen and endometrial cancer. Critics of thisapproach note that because estrogen use may pro-voke uterine bleeding, and because a woman withbleeding is very likely to seek medical attention,there may be a higher percentage of women care-fully examined and tested in the group takingestrogens than in the population of women nottaking estrogens. This would lead to a higher rateof detection of endometrial cancer in the estrogengroup than in the nonestrogen group.

Horwitz and Feinstain (191) contend that be-cause of this increased surveillance, cancers aredetected in the estrogen group that otherwisewould not come to clinical attention during thelifetime of the women, and that if the nonestrogengroup were tested as carefully, more cancerswould be detected in that group. To counteractthis potential selection bias, these investigatorsrecommended selecting controls from amongwomen surgically treated for noncancerous uter-ine diseases. The use of such a population to createthe control group should adjust for the bias re-sulting from increased surveillance and diagnosis.Horwitz and Feinstein showed that when thisselection procedure was employed, the likelihoodof estrogen being linked to cancer was significant-ly lower than under previous study approaches.As Cole (61) has stated, however, patients under-going the same diagnostic procedures as the casescan be “an inappropriate control group, ” since thesame causal agent may be responsible for theirillnesses.

These studies have not resolved the issue, how-ever, and proponents of traditional control selec-tion procedures claim that there is little detectionbias in their method, since most cases of en-dometrial cancer are eventually diagnosed (196).These critics maintain that the controls used byHorwitz and Feinstein are biased, because theydo not give an appropriate picture of estrogen use

in the underlying population. However, recentevidence from autopsy studies has shown thatmany cases of endometrial cancer indeed are un-suspected during life and are first detected, if atall, at autopsy (192).

Is there more or less bias in Horwitz’s and Fein-stein’s control group selection than in the tradi-tional approach? The two approaches might beviewed as providing a range of estimates for therelationship being examined. Because of the in-ternal validity problems associated with thismethod, the use of different control groups toestimate the range of relative risk estimates mightbe considered.

In summary, case-control studies are relative-ly inexpensive, can be carried out in a relativelyshort time and usually employ smaller samplesizes than other study designs. The case-controldesign lends itself to ascertaining the associationsbetween known rare events or outcomes and sus-pected causal agents when the events occur onlyyears after the exposure. For example, this designmight be used to investigate the relationship be-tween a commonly used drug and a rare adverseeffect. Case-control studies can be used to explorea hypothesis without disrupting medical practice.However, case-control studies are not useful fordiscovering previously unsuspected effects or dis-covering adverse effects of rarely used drugs.

Summary

Observational study designs used to assess theoutcome of a medical technology are those inwhich the investigator does not control the ap-plication of the technology to the study popula-tion and applied in essentially the same manneras observational designs would be applied to ex-amine other risk factors for disease. These designsare most applicable for detecting or ruling out spe-cified but unforeseen adverse consequences of atechnology after the technology has been diffused.Experimental designs, those in which the investi-gator controls the application of the technologyaccording to specific criteria, are in theory andoften in practice more useful, especially in deter-mining efficacy.

The degree of validity, particularly internal andexternal validity, of the findings varies with the


study design chosen. Observational studies’ (e.g.,case-control and cohort) lack the high degree ofinternal validity found in the design of choice forexperimental studies, the RCT. That is, it is usual-ly more difficult in observational as opposed toexperimental studies to determine whether the ob-served differences can be attributed to the tech-nology under study. Because observational studiescan more accurately reflect the conditions of useof medical technologies in the population, theymay, in some cases, have a higher degree of ex-ternal validity than experimental studies.

The study design ultimately selected dependson several factors, including the developmental

EVALUATING ECONOMIC EFFECTS

The economic effects of medical technologyhave been assessed through a variety of methods,most notably cost-benefit analysis, efficiencystudies, cost-effectiveness analysis (CEA), cost-impact (total costs associated with a technology),or private sector-oriented techniques such asreturn-on-investment analysis.

Currently, the most visible and potentially themost useful of these techniques is CEA. CEA isnot simply an economic technique; it is a blendof economics and clinical information. As such,it will be described in chapter 5 with synthesis.

No matter which form of analysis is chosen,certain methodologic considerations need to betaken into account. These considerations wereidentified and examined in previous OTA reports(270,271), and the discussion presented here isbased on that earlier work.

Opportunity Cost

The principal concept when evaluating the eco-nomic effects of a medical technology (or any ac-tivity) is opportunity cost. The opportunity costof a resource is its value in its next best use. Thus,the true cost of a resource is not necessarily itsmarket price tag. Rather, it is what one must giveup elsewhere in order to use that resource.

An illustration should help to clarify the dif-ference between a market price tag and a re-

stage of the technology, the purpose of the study,ethical considerations, the population available,and budget constraints. Seldom is assessment aone-time event. Associations of cause and effectcan rarely be established through a single study.In theory, judicious decisions in study designselection should be based on a review of previousand ongoing studies so that each new study be-comes a building block toward the total assess-ment, leading to sound policy decisions. In prac-tice, sometimes they are not.

source’s true opportunity cost. From the perspec-tive of a hospital accountant, volunteers’ time isfree; it is not found on the hospital’s wage bill andthe accountant would ignore it. But is volunteerlabor not a true cost of running the hospital? Vol-unteers definitely contribute to the output of thehospital. And from a social perspective, if thevolunteers’ labor would have been donated else-where had the individuals not worked at the hos-pital, such labor clearly has value. In essence, theopportunity of using the labor productively inother activities has been foregone. From a socialperspective, therefore, the volunteers’ time shouldbe included in an assessment of costs. Althoughdetermining an appropriate dollar value maybedifficult, the social value of volunteer time shouldnot be ignored in an analysis.

Furthermore, as stated in chapter 2, both directcosts—resources purchased directly—and indirectcosts—the value of the lost “production” timeseeking care or being sick—should be included inan analysis.

Marginal Valuation

The worth of a technology should be assessedat, what economists term, “the margin. ” That is,an analysis should seek to compare the added, ormarginal, cost of producing the next unit of ben-efit.

—


In an evaluation of computed tomography (CT)scanning, the issue is no longer whether the tech-nology itself is cost effective, but, rather, whetherthe various applications of the technology are costeffective. Should CT be used for confirming sus-pected brain disease/trauma, or for ruling outbrain disease/trauma when persistent headachesare presented? In what instances are body scansindicated—or cost effective?

In general, the relevant inputs or costs whichmust be considered in the case of a medical tech-nology will be tied to whether the technology isalready in place or whether it has yet to beadopted/purchased.

Joint Production Considerations

Many technologies have multiple applications,and the technological process being studied isseldom applied in isolation. These two considera-tions can have enormous effects on cost calcula-tions.

For instance, since a single blood test can beand is often used as a source of information fornumerous diseases and bodily functions, analyz-ing the cost of drawing blood for only one pur-pose is inadequate if the total cost is used; it eitheroverstates the associated costs, understates the po-tential benefits, or both. Likewise, a CEA of a Papsmear program should be done in recognition ofthe fact that many other health evaluations arenot only possible but are ordinarily performedduring the examination, whether formally or in-formally. That is, a woman who is given a Paptest may be screened for other pelvic disorders,high blood pressure, fever, skin rashes, weightproblems, and many other conditions. All of theseprocedures carry certain potential benefits and allof them should be assigned some of the cost (or,conversely, less cost should be assigned to the Paptest); or the analysis should be evaluating the com-plete examination rather than just the Pap test.

Including the effects of joint production addsgreatly to the problems of measurement and val-uation, but these difficulties in no way diminishthe conceptual importance of fully consideringthese effects in a complete CEA. Sometimes, forinstance, a very small incremental (or marginal),

increase in cost to an existing production processcan have large benefits spread over multiple ap-plications. However, some large cost increasesmay produce fewer benefits than existing produc-tion processes when their contributions to all theapplications are taken into account.

R&D Costs

R&D costs may pose a problem when evaluat-ing a technology’s worth. In general, where R&Dis an integral part of the immediate program inquestion (e.g., when analyzing the costs and ben-efits of a new technology in a medical researchcenter), the R&D resources should be includedalong with the program’s operating inputs. Whenthe R&D has preceded the program being evalu-ated—that is, its existence is independent of theimmediate policy decision—R&D resourcesshould be excluded from consideration.

Overhead Costs

Determining how to allocate overhead costs isparticularly difficult. If the use of the technologyat issue is truly marginal to the overall enterprise,one might be tempted to ignore overhead, to lookonly at the marginal resource needs associatedwith the program. However, if the existence ofsome of the overhead depends on the program inquestion, clearly it must be identified and in-cluded. The general principle of seeking the mar-ginal inputs still holds, but often in practice onemay have to attribute to the program a share ofoverhead proportional to the program’s share ofthe total enterprise.

Costs v. Prices

Uncritical use of market prices can lead to largegaps between cost estimates and true costs. Illus-trative of this problem is the use of hospital chargedata to reflect the costs of hospital care. A com-mon practice, this form of “pricing” ignores theknown idiosyncrasies of hospital accounting inwhich hospitals charge well above true marginalcosts for certain services and use the profits to sub-sidize other services for which charges do notcover marginal costs. If the deviations from mar-ginal costs were small, one might reconcile accept-


ance of imperfect hospital data as a readily avail-able source of information providing a qualitativevalid picture. However, studies of the discrepan-cies between true costs and charges show dramaticdifferences. For example, hospital pharmacycharges can vary from 10 to 1,000 percent of thetrue costs of drugs depending on the frequencyof their use, their level of cost, purpose, etc.

Discounting

Costs and benefits seldom occur at the samepoint in time. Through the application of amethod termed discounting, however, they canbe treated as if they all occurred in the present.

The rationale for discounting future costs andbenefits stems from the fact that resources can beproductively invested for future gains, as well asfrom the observation that people expect to berewarded for postponing gratification. For in-stance, in order to induce individuals to save, in-

CONCLUSION

Choosing a research method for assessing healtheffects depends on various factors. In general, oneshould opt for the study design that producesresults. But constraints such as economic andsocial/ethical factors limit one’s choice. There isa role in medical technology assessment for eachof the methods discussed in this chapter. The im-portant point to remember is that each methodhas its inherent strengths and weaknesses, and onemust always exercise caution in accepting theresults of a study without carefully taking noteof the study’s limitations.

terest must be paid, even in the absence of infla-tion. The rate of interest determines the futurevalue of the amount invested. Thus, for exam-ple, $100 invested at 5-percent interest this yearwill become $105 next year. Discounting is thereverse process: $1O5 next year has a “presentvalue” of $100 when the discount rate is 5 percent.

Although there is general agreement amongeconomists and policymakers that discounting

future moneys is conceptually correct, there is noconsensus concerning what discount rate shouldbe used, and there is still some confusion as tothe proper method of valuing future nonmonetarybenefits/effectiveness. However, when benefitsare long delayed, almost any discount rate willreduce benefits substantially (to near zero in ex-treme cases), making them less important to theoutcome of the analysis (270). Thus, this phenom-enon results in making the rate used and the uncer-tainty of future events less important than theyotherwise would be.

Evaluating economic effects requires careful at-tention to the principles outlined in the latter por-tion of this chapter. An important point to alwayskeep in mind is that costs are usually not whatthey appear to be, especially in health care.

The evaluation of the social and ethical implica-tions of medical technologies is discussed in thenext chapter. The following chapter reviews meth-ods for synthesizing results from research studies,CEA, and group decisionmaking techniques.

98-144 9 - 82 - 4

4Social Values in

Technology Assessment

The very success of science has ended its pleasant isolation.—Robert Sinsheimer

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Value Premises of Medical Technology Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Social Implications of Medical Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

End-Stage Renal Disease Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Maternal Serum Alpha-Fetoprotein . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Artificial Heart. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .General Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Policy-Related Activities for Considering Social Effects of Technologies . . . . . . . . . . . . . .Office of Technology Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .National Commission for the Protection of Human Subjects of Biomedical and

Behavioral Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Ethics Advisory Board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .President’s Commission for the Study of Ethical Problems in Medicine and

Biomedical and Behavioral Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .National Center for Health Care Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Page454547484848495151

5152

515253

4 ■

Social Values in Technology Assessment

INTRODUCTION

Any decision to develop or use a medical tech-nology, or not to do so, inevitably rests on valuejudgments, though such values may not be ex-plicitly acknowledged, One goal of technologyassessment, therefore, is (63):

. . . the identification of [social] implicationspossibly overlooked by decisionmakers who, bytheir own values, would not want them over-looked or implications that decisionmakers can-not afford to ignore, even if they so desire, be-cause many other people, by their own values,may find them important.

This chapter explores the role of values in med-ical technology assessment. The first section ex-amines the value premises of the assessment proc-ess itself. Implicit value judgments permeate everystage of a technology assessment, and it is impor-tant to recognize these when conducting or ex-amining a specific assessment. The second sectionof this chapter considers the role of value analysisin assessing the social and ethical implications ofpolicy decisions regarding the development anduse of medical technologies. The third section re-views the efforts of some past and present Federalbodies to consider these implications.

VALUE PREMISES OF MEDICAL TECHNOLOGY ASSESSMENT

Technology assessment and other types of pol-icy analysis can never be totally objective orvalue-free. Even at the most fundamental level,technology assessment is based on the value as-sumptions that: 1) it is better for society tosystematically analyze the far-reaching conse-quences of technological change and developmentfor its effect on economic, social, and ethicalvalues; and 2) it is better for society, in terms ofmaximizing benefits, minimizing risk, and pro-moting efficiency, to have the information thattechnology assessments can provide. That thesethings are “better” represents a fundamentalassumption, an unspoken value judgment.

Indeed, although they are rarely made explicit,value judgments enter into every phase and aspectof technology assessment. Such judgments deter-mine: 1) which technologies will be assessed andat what stage of their development, 2) what thescope of assessments will be, 3) what kinds of datawill be collected and how these data will be ana-lyzed, 4) what methods will be used in an assess-ment, and 5) how the results of an assessment willbe presented and interpreted.

The first thing to consider, perhaps, is thegenesis of the assessment process. Assessments ofmedical technologies are requested by decision-makers concerned with questions relating to theirsafety and efficacy, reimbursement, and appro-priateness. The factors and perceived needs thatlead to a specific request should be clearly under-stood by those conducting or examining an assess-ment. The constraints inherent in the requestshould also be understood. What types of issuesis the assessment not expected—or not per-mitted—to consider? Finally, the assessmentshould be put into a larger perspective. Why isthe technology a concern to society? Where doesthe assessment topic fit into the current culturaland political setting?

Also important to consider are the values of theassessors and of the analytic methods, goals, andobjectives they choose to employ. Here the issuesare most numerous. First, there is a need to deter-mine the boundaries and scope of the assessment.What values should the assessment include? Ifvalue judgments are organized into a hierar-chy/continuum of abstractions, running from

45


generalities to specifics, a technology assessmentbegins at some point in this hierarchy/continuum,with the implicit assumption being that only thevalue implications below that point will be con-sidered. Values above that point (i. e., those con-sidered more abstract or general) are prior as-sumptions that are accepted, laying a foundationfor the analysis that follows. In other words, anagreed upon set of decisions is in some sense“final,” at least from the perspective of the analysisat hand, so that the analysis is only concernedwith specific policies or refinements (214).

In some sense, establishing a hierarchy of valuejudgments incorporates the values of the in-dividuals performing the assessment. Never-theless, the values of the assessors warrant specialattention on their own. It is unreasonable topresume that one can begin to appreciate, in anyworthwhile fashion, the psyches of those conduct-ing an assessment. The concern, however, shouldnot so much be what the assessors’ individualbiases are as with ensuring that their values donot overly bias the outcome of the assessment.Thus, it is important to build into the assessmentmeasures to correct for possible value distortions.Bioethicists have been particularly successful inthis regard, bringing a broader set of values—more representative, perhaps, of those held by thepublic—to the fore in technology assessment. *The key is to look for values that have been con-sciously or unconsciously omitted from the anal-ysis. Attempts to broaden the values representedin an assessment can also be made by perform-ing an assessment under public scrutiny. In con-ducting its assessments, for example, OTA usesmultidisciplinary advisory panels that include in-terested parties.

The “tools” of technology assessment-e. g., themethods of collecting and evaluating data—mustbe applied with great caution and with the broad-

‘Bioethics is a discipline that brings analytic rigor to consideringvalues camouflaged or implicit in medical, biomedical, and healthcare issues. In conducting assessments, one may wish to involve bio-ethicists and others with expertise and experience in moral or ethicalprinciples to identify and analyze relevant moral principles in avail-able policy options. Though experts in value analysis may be quiteproficient in arraying ethical and social implications, this does notimply that their expertise qualifies them to select which of those prin-ciples society should pursue most vigorously (377).

est possible understanding of the kinds of distor-tions they can create. One danger in technologyassessment is that problems may be reduced toterms that mistake their underlying structure andignore their total character. Indeed, the problemof using too narrowly defined objectives is of con-cern in all policy analysis. Convenience for theanalyst often leads to inaccurate definitions ofproblems.

The measurement and analysis of data are tasksinvolving more than technical procedures, andcarry implicit value systems and orientations(140). In assembling data and information rele-vant to an assessment subject, value judgmentsare incorporated in the choices of what to lookfor, the manner in which data are measured, andthe manner in which information is presented.

For example, in measuring improvements inhealth status, changes are frequently expressed inlevels of resource inputs. It is not clear, however,that inputs such as more doctors, more hospitalbeds, more computed tomography (CT) scannersdirectly translate into improved health status. Fur-thermore, even if one assumes that more of a re-source input is “better,” one still cannot measurethe improvement in health status that results fromthe addition of each resource unit. If decision-makers are attempting to determine which re-source is needed more, they must know the levelsof effectiveness involved in the addition of doc-tors, beds, or CT scanners. Otherwise, there isno basis for choosing among programs emphasiz-ing one of these resources. In addition, measur-ing health status in this fashion carries the implica-tion that the development of any new programsmust be biased in favor of such measurable goals,or the new programs cannot be compared againstolder programs (140).

The methods available to assess medical tech-nologies also have normative underpinnings. If,upon examination, the underlying normative as-sumptions are found to be unsatisfactory, the con-clusion of the analysis must be rejected. OTA’sreport on cost-effectiveness analysis (270) dis-cussed in detail the inappropriateness of ag-gregating costs and benefits in economic analyses.Thus, for example, the technique of cost-benefitanalysis is of concern, because it requires aggrega-

Ch. 4—Social Values in Technology Assessment ● 4 7

tion which tends to ignore the distributional ef-fects of the costs and benefits that the techniqueattempts to measure. Encouraging the conduct ofrandomized clinical trials implicitly signals a will-ingness to subject a relatively small number of in-dividuals to varying degrees of risk, hoping thatgreater benefits will accrue to society as a whole(19). The danger here is that one small group (e.g.,the urban poor) may bear a disproportionateshare of the risks of experimentation, raisingserious questions of equity and autonomy.

Thus, throughout an assessment there is a needfor constant inquiry into the nature of the ques-tions being asked. Despite deficiencies in measure-ment techniques and the difficulty of translatingsocial principles and values into practical terms,technology assessment can contribute to betterjudgments regarding appropriate responses totechnological change and development whensocial values are explicitly considered.

SOCIAL IMPLICATIONS OF MEDICAL TECHNOLOGIES

To varying degrees, medical technologies maydirectly or indirectly influence the quality of thelives of individual patients and their families; thestructure of medical, legal, political, and economicsystems; and the fundamental values on whichthese social systems rest, including society’s senseof ethics and morality. * These “social implica-tions” cannot easily be quantified, but they canbe identified and rigorously analyzed.

An important task in medical technology as-sessment is to identify the conflicting social valuesthat underlie policy alternatives. This assessmenttask is sometimes referred to as “value analysis.”* *In the broadest sense, the task of value analysisin examining social implications is to bring intofocus the compromises that are made with socie-ty’s preexisting goals, values, and institutionswhen choices are made between policy alter-natives. Often, value analysis may simply pro-vide a more conceptually clear understanding ofpolicy problems by describing the complex in-teraction of interests within the confines ofestablished social—economic, medical, legal, andcultural—values.

● Ethics comprises “the principles of morality, of right and wrongconduct, of virtue and vice, and of good and evil as they relate toconduct” (304), “Ethics” and “morality” are sometimes used inter-changeably.

● *Value analysis and the social implications of medical technologyare most often discussed by bioethicists. Much of the bioethicsliterature focuses on the ethical implications of medical technologies,as ethical issues are often the most profound and difficult to recon-cile. However, not all of the social implications of medical technol-ogies are ethical ones.

Some work has been done in the area of sug-gesting techniques for assessing social implica-tions. In 1976, OTA developed an illustrative listof questions that could be asked regarding a med-ical technology (269):

●

●

●

●

What are the implications of the technologyfor the patient?What are the implications for the patient’sfamily?What are the implications for society?What are the implications for the medicalcare system?What are the implications for the legal andpolitical systems?What are the implications for the economicsystem?

Wolf has suggested that certain economic ana-lytic techniques (e.g., cost-benefit analysis) couldbe modified and applied as value analysis (397).

Jensen and Butler have suggested that valueanalysis specifically in the area of ethical implica-tions should be structured by the following threetasks (205):

1. articulation of relevant moral principles;2. elucidation of proposed policy options in

light of the identified principles; and3. rank ordering of policy options for choice.

The first task is to identify the moral and ethicalprinciples around which the policy issue turns,and to set these principles into the center of thediscussion in as definite form as possible. In set-ting public policy to guide the conduct of biomed-

—


ical research on fetuses, for example, two ap-parently conflicting concerns overwhelm thedebate: respect for the autonomy of individualsv. the knowledge (and hence the benefits) socie-ty gains through such research efforts. The secondtask is to examine how policy options interactwith the various identified ethical principles andtheses. This task involves identifying and isolatingthe subtleties of ethical questions. Finally, thethird task is to rank the policy options to showhow each policy would look and what its prob-able outcome might be if one moral principle wereranked over another. This ranking is similar toeconomists’ arraying of tradeoffs in costs andbenefits when comparing alternative policies(270,383).

Public policies in the United States are not di-rected toward a single set of objectives. Differentpolicies reflect different social goals, which mayoften appear to be in conflict. In determiningwhether to further the development or use of amedical technology, questions of safety, efficacy,and cost effectiveness are centrally important, butin some cases—especially when the technologybrings two or more of society’s values and goalsinto conflict—these may be outweighed by abroader set of costs and consequences. Such con-flicts are illustrated by the technologies discussedbelow.

End-Stage Renal Disease Program

This year, the Federal Government will spendover $1.2 billion to provide dialysis treatment andkidney transplants to some 50,000 patients suf-fering from end-stage renal disease (ESRD) (seecase study in app. E). Without such treatment,these patients would die from renal failure. Al-though no one denies that the Federal ESRD pro-gram provides medically necessary services topeople in dire need, the program is surroundedby agonizing questions for policymakers facedwith decisions about allocating medical resources.Because of their enormous costs, disease-specificprograms (like that for ESRD) cannot be public-ly funded for all patients whose medical needs areequally pressing or more ambiguous (48).

The ESRD program was enacted as a humani-tarian response to the vivid impact of dialysis and

transplants and the plight of needy patients. Inthat it does not offer guidance for selecting amongequally needy groups of patients suffering fromother diseases, the enactment of this program doesnot reflect a guiding ethical principle. The absenceof analytic foresight in this instance makes theESRD program appear to be a political accident.Because choices among programs competing forscarce resources inevitably do and should rest onvalue judgments, more coherent public policiesmight evolve if attendant analyses represent acareful working through of the ethical underpin-nings of policy alternatives.

Maternal Serum Alpha-Fetoprotein

Maternal serum alpha-fetoprotein (MSAFP)(see case study in app. E) is the first in a seriesof diagnostic tests used to screen and diagnose twotypes of fetal neural tube defects: anencephaly (ab-sent or undeveloped brain) and open spina bifida(failure of the spine and overlying skin to fullyclose over the spinal cord). Initially, attention wasfocused on MSAFP because of profound social im-plications inherent in its diffusion and use. Thetest is given to expectant mothers to provide themwith information about the fetus, the value as-sumption being that it is better for them to knowin advance if their children are to be born withneural tube defects. Since some mothers given in-formation that such defects are present might beexpected to terminate their pregnancies voluntari-ly, MSAFP was thrust into the ethical argumentover abortion.

The test has also raised questions of distributivejustice, particularly with regard to entitlementprograms. For example, the Health Care Financ-ing Administration was concerned with the ethicalimplications of reimbursable MSAFP tests forMedicaid recipients. Women receiving Medicaidwere not entitled to abortions. Thus, the dilem-ma arose: If Medicaid agrees to reimburse physi-cians for providing MSAFP tests, what happensto women with test results indicating fetal defectswhen abortions are not reimbursable?

Artificial Heart

Another medical technology that illustrates theimportance of social implications is the artificial

Ch. 4—Social Values in Technology Assessment ● 4 9— ——- — — —-— . — — — ——

heart. In 1972, the National Heart and Lung In-stitute’ convened a panel of physicians, ethicists,lawyers, and social scientists to identify andevaluate the “economic, ethical, legal, medical,psychiatric, and social implications of a totallyimplantable artificial heart. ” The institute wasconcerned about the broader implications of suchan innovation, implications which the institute’sphysicians and administrators recognized were“beyond the limits of their own expertise” (204).

The panel primarily focused on two sets ofquestions, both related to the ethical problem ofdistributive justice. First were questions concern-ing access to the device and the selection of pa-tients to receive what would most likely be ascarce, expensive medical resource—problemsendemic to modern medicine. Second were ques-tions applicable only to an artificial heart poweredby nuclear energy. (The National Heart and LungInstitute had been developing three power systemsfor its device: an electric motor powered by a bio-logical fuel cell, a motor powered by rechargeablebatteries, and a nuclear engine fueled with plu-tonium. ) The panel noted that patients faced withimminent death from heart disease might be will-ing to accept the attendant risks of prolonged ex-posure to radiation from the nuclear device, butit expressed greater concern for persons exposedto “slight though significant risks” through con-tact with recipients. A majority of the panel’smembers doubted that a decision to make a nu-clear-powered artificial heart widely availablewould be “ethically justified when measures to im-prove the health and extend the life of specific in-dividuals pose a risk to the health and lives of thepopulation generally, including unborn futuregenerations” (204),

General Comments

Not every new medical technology warrants afull-scale technology assessment with an examina-tion of social implications. Some technologies

probably do not even warrant a formal assess-ment. In the area of values and social implications,however, the lack of sound, effective criteria fordetermining which technologies to assess is de-cidedly evident.

*Now the National Heart, Lung, and Blood Institute.

Despite the abundance of offerings from bio-ethics on microallocation issues, * the methods forassessing values and classifying social effects havethus far received little attention in the literature.Macroallocation issues** are well described in theliterature, but more apt to be ignored in an anal-ysis. The panel that examined the developmentof the artificial heart, for example, focused itsdiscussion on the microdistribution issue of whichindividuals would receive this scarce, expensivetechnology. The panel was criticized for failingto consider the artificial heart as an experimentaldevice raising profound questions of patients’abilities to meet informed consent criteria (46).However, this concern misses an even larger socialquestion: Is the development of technologies suchas the artificial heart, which benefit only a few,a proper way to spend social resources? What thenare the implications, the social costs and ben-efits—and how are these distributed—to society?Developing and maintaining an effective concernfor the social implications of medical technology

will be extremely difficult unless further work isdevoted to important questions such as these.

Systematic consideration of relevant social andethical values will not necessarily lead to con-clusive answers about which policies decision-makers should adopt. Nevertheless, choices andcompromises need to be identified so that deci-sionmakers can see which ethical principles willbe sacrificed or compromised by specific policyoptions. Value analysis cannot determine whatthe policymakers’ values should be, but it canbring into focus the impact of choices on estab-lished goals and institutions (373). By describingthe complex interaction of interests within the con-fines of the economic, legal, social, and culturalvalues, and technical facts prevailing at the timeand anticipated in the future, value analysis canprovide a more focused, conceptually clear un-derstanding of policy problems. Value analysiscan show decisionmakers where they disagree andwhy they disagree, as well as identify the longrunsocial implications of their decisions.

‘Microallocation issues are concerned with singfe specialized eco-nomic units (e.g., individual, hospital, household).

“Macroallocation issues are related to larger or multiple economicunits which make up the economy (e.g., government, business,health care).

50 . Strategies for Medical Technology Assessment— . —

POLICY= RELATED ACTIVITIES FOR CONSIDERINGSOCIAL EFFECTS OF TECHNOLOGIES

Congress has explicitly recognized both thevalue of technology assessment and the impor-tance of considering the social implications oftechnological change and development. This rec-ognition is manifest, at least in part, in legisla-tion establishing the Office of Technology Assess-ment (OTA), the National Commission for theProtection of Human Subjects of Biomedical andBehavioral Research (National Commission), thePresident’s Commission for the Study of EthicalProblems in Medicine and Biomedical and Be-havioral Research (President’s Commission), andthe National Center for Health Care Technology(NCHCT). The activities and efforts of thesebodies are reviewed briefly below. Also reviewedare the activities of the Ethics Advisory Board inthe Department of Health, Education, and Wel-fare (DHEW). *

Office of Technology Assessment

OTA was established in 1972 as an analyticsupport agency of Congress to conduct policy re-search on science and technology issues for con-gressional committees. OTA clarifies the range ofpolicy options available on a given set of issues,and assesses the potential physical, biological,economic, legal, and social impacts that mightresult from adopting each option. OTA has con-ducted assessments in wide-ranging areas of con-gressional interest, including energy, internationalsecurity and commerce, materials, food and re-newable resources, biological applications, com-munication and information technologies, oceansand environment, space, transportation, innova-tion, and health. Although OTA reports haveonly rarely been directly translated into policy orlegislation, they do serve to provide comprehen-sive background information on complex issuesrelated to scientific and technological develop-ments.

As exemplified by this report, OTA’S health re-ports have primarily focused on “generic” issuesin the use and assessment of medical technology.

*Now the Department of Health and Human Services (DHHS).

OTA has studied specific technologies (e.g., CTscanners) as illustrative issues in technology as-sessment and policy.

National Commission for theProtection of Human Subjects ofBiomedical and Behavioral Research

The National Commission was established in1974 to develop ethical guidelines for conductingresearch on human subjects, and to recommendapplications of these guidelines for research con-ducted or supported by DHEW (now DHHS). Inestablishing the National Commission, Congressalso requested recommendations regarding theprotection of human subjects in research con-ducted outside DHEW’S purview.

The National Commission’s work, which endedin 1978, was prodigious. Reports and recom-mended guidelines were generated covering re-search on human fetuses, children, prisoners, andthe institutionalized mentally infirm, and on theappropriate utilization of psychosurgery. The Na-tional Commission proposed guidelines for pro-tecting patients in DHEW-funded health carecenters. It also recommended the establishmentof institutional review boards in research centersas a means of ensuring that biomedical and be-havioral research efforts were conducted in anethically acceptable fashion. Many of the NationalCommission’s proposals were recommended, re-vised, and adopted by DHEW, particularly thosegoverning the protection of human subjects.

The National Commission completed two re-ports that have been important resource docu-ments to later studies of human experimentationand biomedical technology. One, The BelmontReport (88), reviewed and clarified the ethicalunderpinnings of research conducted with humansubjects, removing much of the conceptual con-fusion and semantic misunderstanding that hadconfounded previous attempts at rational policyfor human research. Ethical analyses were devel-oped for the distinction between research andpractice, the lack of distinction between thera-

—

Ch. 4—Social Va/ues in Technology Assessment ● 5 1

peutic and nontherapeutic research, the notion ofrisk, and the purpose and use of informed consent,

The National Commission also conducted aspecial study of the social, ethical, and legal im-plications of advances in biomedical and be-havioral research and medical technology (89).The study addressed the implications of computerapplications to medicine, life-extending technol-ogies, genetic screening, and reproductive engi-neering. The report offered no definitive recom-mendations and was less easily translated intopolicies than some of the commission’s otherworks (349).

Ethics Advisory Board

One of the National Commission’s recommen-dations to the Secretary of Health, Education, andWelfare resulted in the establishment of the EthicsAdvisory Board in 1978. The board was given abroad mandate to review ethically questionableresearch protocols and research involving humansubjects. Its most formidable efforts focused onin vitro fertilization and embryo transfer (the Na-tional Commission determined that under certainspecified conditions, such research could be con-ducted in an ethically acceptable manner). Theboard also examined ethical questions raised byuse of fetoscopy for diagnosing sickle cell anemiaand hemoglobinopathies.

Because it was established in the Office of theSecretary, the Ethics Advisory Board often fieldedqueries from other DHEW agencies. Conductingresearch with human subjects raised particularproblems for agencies in handling inquiries gen-erated under the requirements of the Freedom ofInformation Act. The Centers for Disease Con-trol (CDC) and the National Institutes of Health(NIH) were especially concerned about releasingspecific types of research and epidemiologic data,because the data were supplied voluntarily byhealth care providers and institutions. CDC wasafraid that its sources would dry up if the datawere released. NIH wanted to avoid disclosing in-complete or preliminary findings from its ongo-ing clinical trials and observational studies. TheEthics Advisory Board focused attention on theneed for those agencies to withhold informationunder the provisions of the Freedom of Informa-

tion Act. Funding for the board was eliminatedin 1980.

President’s Commission for theStudy of Ethical Problems in Medicineand Biomedical and Behavioral Research

The President’s Commission was established asthe successor to the National Commission in 1978.Its enabling legislation directs the Secretaries ofHealth and Human Services and Defense, the Ad-ministrator of the Veterans Administration, andDirectors of the Central Intelligence Agency, Na-tional Science Foundation, and White House Of-fice of Science and Technology Policy to appointrepresentatives as liaisons with the President’sCommission. Thus, the president’s Commission’ssphere of influence is much broader than was thatof the Ethics Advisory Board.

The President’s Commission is mandated to

conduct studies in the broad areas of medical prac-tice and biomedical research. The President’sCommission is also to examine five specific sub-jects for their legal and ethical implications andtheir importance to public policy:

1.

2.

3.

4.

5.

the requirements of informed consent forparticipation as human research subjects andfor receiving medical treatment;the procedures designed to assure the privacyof human subjects, the confidentiality of in-dividually identifiable patient records, andappropriate access for patients to informa-tion contained in medical records;the issue and advisability of developing auniform definition of death;voluntary screening, counseling, informa-tion, and education programs concernedwith genetic diseases, and the fundamentalequality of all human beings, born and un-born; andthe differences in availability of and accessto health services as determined by such var-iables as income and residence.

, its first year, much of the President’s Com-lnmission’s work focused on the protection ofhuman subjects. It examined proposed DHEWregulations (which had been based largely on theguidelines of the National Commission), specific


problems inherent in social science research, andthe operations of institutional review boards. ThePresident’s Commission paid particular attentionto an issue originally raised by the Ethics AdvisoryBoard: compensation for subjects injured in bio-medical and behavioral experimentation. This isthe only activity of the former Ethics AdvisoryBoard carried on by the President’s Commission;the President’s Commission does not review ques-tionable protocols for DHHS-funded research.

The first formal report of the President’s Com-mission was released in July 1981 (296). A land-mark examination of the medical, legal, andethical issues surrounding the determination ofdeath, the report is expected to serve as a modelin formulating a uniform statute defining death.

In keeping with the legislative requirement thatit respond to specific Presidential requests, thePresident’s Commission has been studying issuesrelated to genetic engineering. At the urging ofreligious leaders, the President’s Commission wasasked to address the social and ethical implica-tions of the new technology. A draft report, focus-ing on questions of safety, technical capabilities,and specific issues in therapeutic and diagnosticuse, was in preparation at the time of this writing.

The President’s Commission has spent most ofthe past year looking at the distribution of healthcare resources, particularly as it differs amongpopulation groups. To date, the discussion hasincluded barriers in access to care, disparities anddifferentials in the utilization of health services,an attempt to identify relevant ethical principlesaimed at defining equity of access, the nature ofspecial health care facilities, cost considerations,and freedom of choice for both patients and pro-viders.

National Center for Health CareTechnology

NCHCT was established in 1978 to undertakeand support assessments of medical technologies,

including questions of their safety, effectiveness,and cost effectiveness and their economic, ethical,legal, and social implications. Funding forNCHCT was not provided for fiscal year 1982.Congress had envisioned two primary missionsfor NCHCT: 1) stimulating increased scrutiny ofnew and existing health care technologies to en-sure that the questions listed above were thor-oughly explored; and 2) encouraging the dissem-ination of new technologies proven safe, effective,and cost effective (277).

NCHCT had no regulatory authority. Its pur-pose was to provide current evaluations of healthcare technologies to individuals and agencies withregulatory and decisionmaking responsibilities.NCHCT funded and conducted two types of as-sessments—’’focused” and “full’ ’-but did no ac-tual testing of technologies.

“Focused” assessments examined the scientificand medical aspects of a technology to evaluatethe technology’s safety and efficacy. Such assess-ments were prompted by coverage questionsraised by HCFA. NCHCT gathered and evaluateddata, then made a recommendation to HCFA asto whether the technology should be covered byMedicare. By 1981, NCHCT had completed morethan 60 focused assessments.

“Full” assessments examined the technology’ssafety, effectiveness, cost effectiveness, and socialimplications of a technology. Full assessmentswere integrated analyses conducted by NCHCTin cooperation with other interested Federal agen-cies, representatives of appropriate private sec-tor organizations, and individuals from a broadrange of relevant disciplines. When possible, par-ticipants attempted to reach agreement and pro-vide recommendations for appropriate utilizationof the technology. NCHCT identified technologiesfor full assessments with the assistance of an ad-visory council. Such assessments were conductedfor coronary artery bypass surgery, cesarean de-livery, and dental radiology.

Ch. 4—Social Values in Technology Assessment ● 5 3— — . — — — . . . — ——— ——— —..— .

CONCLUSION

Value analysis has a dual role in medical tech-nology assessment. The first is to consider the ef-fects on society’s cultural, ethical, and politicalvalues that may result from the introduction,modification, or extension of medical technol-ogies. Such effects cannot easily be measured andbalanced, yet can profoundly affect determina-tions of a technology’s worth. Like any form ofpolicy analysis, technology assessment is foundedon value premises. The second role of value anal-ysis is to ensure that these value premises are madeexplicit and do not unduly influence the outcomeof the assessment.

There is no established set of methods or tech-niques for conducting value analyses, nor is there

a coherent, agreed upon set of principles an anal-ysis should incorporate. Government efforts to

promote value analysis probably do not requirecoherent sets of methods or principles. In con-sidering the social implications of medical tech-nology, Government promotes more comprehen-sive policy analysis; and in making the valuepremises of assessments explicit, it furthers ac-countability for its decisions.

Methods for synthesizing information about thehealth effects of medical technologies and for com-bining this information with information aboutsuch technologies’ economic and social effects arediscussed in the next chapter. Also discussed aremethods for dealing with uncertainty.

5.Synthesis, Cost-EffectivenessAnalysis, and Decisionmaking

Truth is rarely pure, and never simple.

—Oscar Wilde

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Synthesizing Research Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The Literature Review: Unstructured Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Structured Synthesis Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Classification or Voting Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Meta-Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Other Synthesis Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Advantages of Structured Synthesis Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Cost-Effectiveness Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Group Decision Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Delphi Technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Nominal Group Technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Consensus Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Knowledge Base Development. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Advantages and Disadvantages of Formal Group Process Methods . . . . . . . . . . . . . . . .

Decisionmaking Under Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

List of Tables

Table No.l. NIH Consensus Development Meetings, September 1977 Through October 1982 . . . .

Page5757575858585960606262626363656667

Page

64

5.Synthesis, Cost-EffectivenessAnalysis, and Decisionmaking

INTRODUCTION

This chapter is concerned with synthesizing re-search information and economic data in a sys-tematic fashion so that results are useful for pol-icy. This is not an easy task. There is an over-whelming amount of information on medical tech-nologies (basic research, applied research, cost/outcome data), and policy analysts are faced withthe problem of making sense of this information.This chapter describes and critiques methods thatare intended to assist policy analysts in weighinginformation from diverse sources. Much of thediscussion is based on material presented in ap-pendix C and previous OTA reports (270,271).

The first section discusses methods for synthe-sizing research results from multiple studies whichare generally concerned with safety and efficacy.Next, economic concepts are included in a presen-tation of the principles of cost-effectiveness analy-sis (CEA). This discussion is followed by a presen-tation of methods for soliciting group opinionsand a brief discussion of quantitative decision-making techniques.

SYNTHESIZING RESEARCH RESULTS

Individual research studies, in themselves, donot constitute a technology assessment. An assess-ment must consider the evidence from a set ofstudies, evidence concerning social effects as wellas safety and efficacy. Typically, the first step inconducting a medical technology assessment is areview and synthesis of available research evi-dence. Although issues pertaining to synthesishave been neglected in the health care literature(266,398), consideration of these issues is essen-tial if the range of information available aboutparticular medical technologies is to be utilized.Problems encountered in using traditional synthe-sis procedures such as the literature review are de-scribed below. Also discussed are various formalprocedures to systematically synthesize researchresults.

The Literature Review:Unstructured Synthesis

The traditional approach to synthesizing re-search is the literature review. Typically, a review-er selects a set of studies believed to be most rele-

vant and summarizes the evidence. There are anumber of problems in relying on literature re-views. First, such reviews tend to be subjective.Second, methodological problems in individualstudies are often ignored, distorted, or obscured.Finally, there is the problem of timeliness: the re-views must be available prior to the dissemina-tion of ineffective or unsafe treatments.

A number of biases may affect reviews of theresearch literature. The understandable enthusi-asm which investigators have for any new treat-ment that potentially improves care can lead toerrors. This problem may be somewhat dimin-ished when reviews are prepared by groups (e. g.,National Institutes of Health (NIH) panels) ratherthan by individual physicians or researchers,

Aspects of the problem of investigator bias areillustrated by the controversy over electronic fetalmonitoring (EFM). Different perspectives appar-ently cause reviewers of the literature to attendto different aspects of the evidence (see app. C).The paucity of randomized clinical trials (RCTs)generally available (4oo) presents another impor-

57

58 “ Strategies for Medical Technology Assessment

tant obstacle to synthesis, because well-controlledresearch methods are probably the best mecha-nism to guard against investigator bias. Whensuch methods are employed, a reviewer can usemethodological arguments to discuss disagree-ments between studies.

However, even when well-controlled trials areavailable, they may not fully answer questionsabout a technology. Tonsillectomy is a case inpoint. There exists a substantial literature on thesafety and efficacy of tonsillectomies, includingclinical trials and other types of research. Accord-ing to Cochrane (6o), none of the three differentclinical trials on tonsillectomies conducted in Eng-land during the 1960’s resolved the policy contro-versy over the appropriate use of tonsillectomy.The available RCTs, he noted, had two method-ological problems: 1) the treatment was comparedwith no or inadequate treatment (instead of analternate treatment); and 2) the patients’ parentswere not blind to the conditions of the experi-ments, so those whose children were on the wait-ing list may have exaggerated their children’ssymptoms.

Structured Synthesis Procedures

Because of the problems inherent in literaturereviews, efforts have been made to develop moresystematic procedures to integrate and interpretsets of research evidence. These range from ele-mentary classification procedures to sophisticatedstatistical techniques. A description of such tech-niques is presented below. Additional material ispresented in appendix C.

Classification or Voting Method

A simple structured synthesis technique in-volves organizing a body of literature accordingto a prespecified set of criteria and is actually aclassification procedure (226) Sometimes calledthe “voting method, ” this synthesis technique in-volves selecting a particular sample of evaluativestudies of a technology, coding some aspect of thedesign and/or conceptual framework, classifyingobserved outcomes as to whether they are favor-able, neutral, or unfavorable (i.e., “taking avote”), and then constructing tables of researchfindings. This method is frequently used to dem-

onstrate the differences obtained by various meth-odological approaches used to assess the sametechnology.

The value of the voting method lies in the pre-cise identification of the type of studies to be sam-pled, and the coding of substantive and methodo-logical aspects of the studies according to clearlydefined procedures. More widespread use of thisclassification technique could probably aid in de-termining the specific patient populations and/orconditions that can be effectively treated by amedical technology (i.e., to establish external va-lidity). This technique helps to avoid the problemsof more traditional literature reviews, notedabove, which selectively describe research evi-dence and which neglect consideration of method-ological strengths and weaknesses.

Krol (216), however, cites three problems withthe method: sample size, effect size, and Simp-son’s paradox. The first problem is that large stud-ies are likely to produce statistically more signifi-cant results than studies with small numbers ofsubjects. Thus, for example, a study finding of“no results” or “no difference” between treatmentand control conditions may be correlated withsmall sample size. The second problem is that thevoting method does not take effect size into ac-count, i.e., small, marginal effects are not distin-guished from large effects. Simpson’s paradox isa more subtle statistical point. It is possible, undercertain conditions, to reach conclusions by aggre-gating data from all studies that are different fromthe conclusions reached by counting each studyseparately. A fourth problem with the votingmethod is that it may oversimplify the results ofstudies and cause reviewers to overlook subtle in-teractions among variables (226).

Meta-Analysis

A rigorous statistical approach to research syn-thesis is a sophisticated quantitative synthesistechnique called meta-analysis (166,165). Thistechnique uses the actual results of studies and per-mits the determination across a set of studies ofthe magnitude of treatment impact. Meta-analysesare useful in assessing treatments for which a largenumber of studies are available and findingsacross studies seem to have great variability.

Ch. 5—Synthesis, Cost-Effectiveness Analysis, and Decisionmaking ● 59

As used by Glass (165), such analyses requirethat both comparison and treatment groups beavailable and that the original research reportscontain appropriate statistical information suchas the group mean and standard deviation. Effectsizes (ES) are calculated by determining the differ-ence between the mean of the treatment group (T)and the mean of the comparison group (C), di-vided by the standard deviation of the compari-son group (SD). Thus,

ES = T – CSD

This procedure converts the average effect of eachoutcome measure into a common scale (i. e.,standard deviations) that can be compared to re-sults of other studies. If a treatment has no effect,then the effect size would be zero; if the treatmentis effective (i. e., better than the current alterna-tive), then the effect size is positive; and if thetreatment is inefficacious, the effect score is neg-ative. If some assumptions are made about theskewness of experimental and control groupscores within each study and the distribution ofeffect sizes across a large number of studies (i.e.,that they are normally distributed), effect sizes canbe converted into percentile ranks and inferencescan be made about the overall effects of a medicaltechnology.

One recent example of the use of meta-analysisis Smith, Glass, and Miller’s (353,354) review ofthe outcome literature on psychotherapy. Theseinvestigators searched the published literature andincluded within their analysis all available con-trol group studies of the effectiveness of any formof psychotherapy. * For each study, the investiga-tors coded an extensive number of variables, in-cluding methodological criteria such as the natureof patient assignment to condition (e.g., randomv. matching), experimental mortality, and otherthreats to internal validity. Effect size scores werecalculated for each principal outcome. A code wasalso developed for validity of the outcome meas-ures.

‘Drug studies were analyzed separately, and studies that did notinvolve the use of professional therapists (operationally defined aspsychologists, psychiatrists, and social workers) were eliminatedfrom the analysis.

Smith, Glass, and Miller’s (354) findings indi-cated that, on average, the difference betweenscores of the groups receiving psychotherapy andthe control groups was 0.85 standard deviationunits. Assuming the normal distribution of effectsize scores, this average standard score can betranslated to indicate that a typical person whoreceives psychotherapy is better off than 80 per-cent of the persons who do not. The investigatorsalso performed a number of analyses to determinewhether the methodology of the outcomes studyaffected results and whether different therapies (orother factors) were differentially efficacious.

The work of Smith and colleagues (353) hasbeen criticized, because it “lumps together” a largenumber of what some consider incomparabletreatments and outcomes (e.g., 137). It should benoted, however, that the strength of the effect sizetechnique is that it provides a common metric thatpermits analysis of these differences (methodologi-cal and substantive). Smith, et al. ’s (354), classifi-cation variables for each study were fairly com-prehensive and yielded a systematic comparisonof studies on the basis of their conceptual andmethodological designs. What is problematic,however, is that the findings are heavily depend-ent on a number of decisions that are not alwaysmade explicit. These include criteria for selectingliterature and criteria for selecting variables. It isnot possible to ascertain biases resulting from theinvestigators’ sampling decision and whether onlycertain types of studies, therapies, or variables areassessed using control group designs (273).

Other Synthesis Procedures

A number of other methods exist for statistical-ly combining the results of independent studies(see 69,292,324). The effect size method describedabove actually incorporates several procedures.The most important of these is the comparisonof treatments to detect interactions between thecharacteristics of a study and outcome (i.e., ex-ternal validity). AS noted in the discussion of theclassification method, some of these procedurescan be employed when effect scores are not COm -

puted.

Additional statistical methods can be used tocombine probability values from various studiesand to adjust outcome scores according to the rele-

—— —


vance of the data. Rosenthal (324) describes anumber of procedures for combining probabilities.These range from adding observed probabilitylevels across different studies to adding weightedstandardized scores. They also include the testingof mean probability values. Use of such proce-dures indicates whether significant effects are ob-tained across a set of studies. The problem in usingprobability values is that the number of subjectsper study influences the statistical power to detectwhether significant overall differences are present.

An interesting method for synthesizing the re-sults of experiments done with human and animalspecies according to the relevance of the data hasbeen proposed by Du Mouchel and Harris (131).This method involves the sophisticated applica-tion of a statistical theorem (Bayes’ theorem) toprovide a quantitative prediction of data rele-vance. Du Mouchel and Harris use the procedureto provide estimated carcinogenic risk of varioussubstances derived from the results of a series ofepidemiological studies.

Advantages of Structured Synthesis Procedures

A number of advantages result from the use offormal procedures for data synthesis (292). Onebenefit is that formal syntheses help to identifycontradictions in the literature by systematicallyorganizing studies according to specified classifica-tion factors. Thus, differential outcomes can besegregated according to treatment chacteristicsand/or methodological approaches.

The use of effect size scores offers a second ben-efit. Such scores provide insight as to the worthof a treatment and provide a benchmark for laterresearch. Thus, for example, an analysis of 23controlled studies of patient education programsby Posavac (294) found a 0.75 average effect size.According to Posavac, this should provide a

COST-EFFECTIVENESS ANALYSIS

CEA can be thought of as an aid to synthesisof both the health effects and the economic ef-fects of a medical technology. CEA was itself thetopic of a recent OTA assessment (270,271,272,273,274). OTA’s findings from that study are sum-marized here.

standard against which new patient educationprograms can be assessed. A finding that the ef-fect size of a new program is only 0.20 (providingthat similar dependent measures are employed),would probably indicate that the program was notparticularly effective, at least for the problem orpopulation for whom it had been designed.

Another advantage of quantitative synthesismethods is that they serve to control for some sta-tistical conclusion validity problems (e.g., power)that some commentators have reported as severein the medical literature (e.g., 141,172,337). Thewidespread use of meta-analysis and other quanti-tative approaches to research synthesis would like-ly improve statistical reporting practices by call-ing attention to investigators’ use of data. Further-more, most quantitative analyses of multiple stud-ies would compensate for errors in analyses suchas the use of multiple-independent inferential testswithout appropriate error rate control or incor-rect inferences because of a lack of power. Al-though errors in data collection would not be cor-rected by quantitative synthesis methods, the sys-tematic analysis of multiple studies should renderthe effect of such errors less consequential. Theattention to systematic considerations of the“weight” of evidence across research studiesshould have a generally salutary effect.

These synthesis procedures seem most appro-priate for evaluating more mature medical tech-nologies about which there has accumulated aconsiderable body of research. Often however,they may be applicable to less developed technol-ogies. In some cases, where only meager evidenceis available from a small set of studies, the effec-tiveness of a medical technology maybe suggestedby a review of specific components from someother portion of the literature.

The value of CEA lies more in the process ofperforming the analysis than in any numericalresults. There are a number of reasons for this,among the most important of which are CEA’Sinability to adequately address ethical issues andthe uncertainty of many of the key variables re-

Ch. 5—Synthesis, Cost-Effectiveness Analysis, and Decisionmaking ● 61— -.

quired for the analysis. Addressing uncertaintyis a topic addressed later in this chapter.

A second major finding from the OTA studywas that there is no one “correct” way to do ananalysis. Not only does each analysis differ interms of which benefits and which costs must beconsidered, but each analysis differs in terms ofhow the benefits and costs are valued. A drivingforce behind these variations are the social/ethicalconcerns mentioned earlier.

OTA suggested that the most appropriate ap-proach to any assessment is to perform it in anopen forum so that assumptions and underlyingvalues can be challenged; to identify, measure,and, to the extent possible, value all relevant ben-efits/effects and costs; and to present the resultsof the analysis as an “array” of benefits/effectsand costs rather than forcing them into some ag-gregate single measure. By arraying effects in asystematic fashion, one can place the appropriaterelative emphasis on given effects whether theyare quantifiable or not. This technique is designedto make more explicit the health, economic, andsocial consequences of any decision.

In suggesting the “array” method, OTA recog-nized that CEA is a decision-assisting technique,rather than a decisionmaking one. In some in-stances, however, a cost per aggregated effect maybe possible, appropriate, and quite acceptable. Acase in point is OTA’s own cost-effectivenessstudy on pneumococcal vaccine, which calculateda ratio of $1,000 per quality-adjusted year of lifesaved for the elderly (282). In that case, the studywas performed under public scrutiny, and theanalysis and assumptions were subjected to exten-sive outside review.

Finally, although OTA concluded that therewas no single “correct” methodology for conduct-ing CEA, it did find general agreement on 10 prin-ciples of analysis. Those principles, including ashort explanation of each, are reproduced below.

1. Define Problem.—The problem should beclearly and explicitly defined and the rela-tionship to health outcome or status shouldbe stated.

2.

3.

4.

5.

6.

7.

8.

9.

10.

State Objectives. —The objectives of thetechnology being assessed should be ex-plicitly stated, and the analysis should ad-dress the degree to which the objectives are(expected to be) met.Identify Alternatives. -–Alternative means(technologies) to accomplish the objectivesshould be identified and subjected to anal-ysis. When slightly different outcomes areinvolved, the effect this difference will haveon the analysis should be examined.Analyze Benefits/Effects. —All foreseeable

benefits/effects (positive and negative out-comes) should be identified, and when pos-sible, should be measured. Also, when pos-sible, and if agreement can be reached, itmay be helpful to value all benefits in com-mon terms in order to make comparisonseasier.Analyze Costs. —All expected costs shouldbe identified, and when possible, should bemeasured and valued in dollars.Differentiate Perspective of Analysis. —When private or program benefits and costsdiffer from social benefits and costs (andif a private or program perspective is ap-propriate for the analysis), the differencesshould be identified.Perform Discounting. —All future costsand benefits should be discounted to theirpresent value.Analyze Uncertainties, —Sensitivity anal-ysis should be conducted. Key variablesshould be analyzed to determine the impor-tance of their uncertainty to the results ofthe analysis. A range of possible values foreach variable should be examined for effectson results.Address Ethical Issues. —Ethical issuesshould be identified, discussed, and placedin appropriate perspective relative to therest of the analysis and the objectives of thetechnology.Discuss Results. —The results of the analysisshould be discussed in terms of validity,sensitivity to changes in assumptions, andimplications for policy or decisionmaking.

62 ● Strategies for Medical Technology Assessment—. —

GROUP DECISION METHODSAlthough the application of formal quantitative

procedures for the integration of data from indi-vidual studies and the use of quantitative decision-assisting techniques should improve the processof technology assessment, such procedures can-not, by themselves, resolve policy controversies.These procedures cannot go beyond the dataavailable on a particular problem, nor can theysubstitute for informed judgment or include soci-etal values (see ch. 4).

A frequently used method for soliciting groupopinions is the unstructured conference at whicha given topic (or topics) is discussed. There aremany conference formats, ranging from formalto informal presentations with or without ques-tions and answers. Sometimes prepared critiquesor presentations are employed; other times a de-bate format is used.

Another informal group opinion technique isthe advisory panel approach such as that used bymany Government agencies (including OTA).Sometimes, the panel is required to endorse a giv-en study. In such cases, studies are often modi-fied until most members are satisfied with the ma-jor points. Often a single member or small numberof members of the panel can exercise de facto vetoauthority over certain elements of the study; othertimes a minority is included in the final report.

Four formal methods for resolving conflictsacross research studies and for developing assess-ments of particular technologies are described be-low: 1) the Delphi technique; 2) the nominal grouptechnique; 3) a new group opinion process, re-ferred to as consensus development, sponsoredby NIH; and 4) a computerized knowledge basebeing developed by the National Library of Medi-cine (NLM). The Delphi and nominal group tech-niques are based on behavioral science principles(163), the goal being to aid groups composed ofindividuals with different information and per-spectives to develop group judgments that besttake account of the positions of the individualmembers. The discussion below illustrates boththe potential and limitations of these methods insynthesizing technology assessment information.

Delphi Technique

Delphi is probably the oldest structured modelfor involving groups in decisionmaking processesand has been used widely in health care (78). TheDelphi technique uses a series of questionnaires(or individual interviews), each followed by anon-ymous feedback summarizing all the participants’responses. Although Delphi was originally devel-oped by the Rand Corp. to synthesize expert opin-ions on national defense problems, it has sincebeen extended to medical problems (232,246,250,318,336).

A unique feature of the Delphi technique is thatpersons selected to participate in the process gen-erally have no direct contact with one another.Instead, participants are provided with a sum-mary of the questionnaire responses, usually bymail. Personality or status variables thus have lit-tle chance to influence participants’ opinions, asthey might in face-to-face meetings. By usinganonymous feedback, each participant has anequal chance of influencing other participants(41). The technique also provides a frameworkwithin which to approach a problem in a focusedmanner. Finally, and perhaps most importantly,the Delphi technique provides a limited time framein which to achieve consensus (41,135). There area fixed number of iterations, usually three, in thequestionnaire-feedback process. (For a discussionof a modified Delphi technique, see ref. 293. )

Nominal Group Technique

Although there are several variations of thistechnique, typically, all participants are seated ata common table and asked to write their viewson each of a number of issues posed by the leaderof the meeting. Delbecq and colleagues (82) callthis the “nominal” group technique, because theindividuals at the table (at the outset) are a groupin name only. Each person’s view is recorded ona separate card, and talking is prohibited. The (si-lent) presence of others while writing the cardsis supposed to stimulate participants to performbetter. The cards are collected, and their contents

Ch. 5—Synthesis, Cost-Effectiveness Analysis, and Decisionmaking • 63

(but not the authors’ names) are listed for all tosee. Subsequent discussions focus on the ideas thathave been proposed, not on who did the propos-ing.

Consensus Development

In response to congressional pressure to assistin the transfer of technology, NIH initiated its con-sensus development program in 1977. Its goal isto bring together various concerned parties—physicians, consumers, bioethicists, etc.—to seekconsensus on the safety, efficacy, and appropriateconditions for use of various medical technologies.Judgments about the technology under considera-tion are intended to be based on the availablescientific evidence. At conferences jointly spon-sored with the National Center for Health CareTechnology (NCHCT), information about thetechnology’s social, ethical, economic, and legalimpacts was included, as well. * The consensusdevelopment process is designed to produce awritten document, called a “consensus statement, ”that can be accepted by clinicians, researchers,and the public. The statement is supposed to iden-tify both what is known and not known aboutthe technology (287). (A list of NIH consensusdevelopment meetings from September 1977through October 1982 appears in table 1.)

The consensus development conferences are co-ordinated by NIH’s Office for Medical Applica-tions of Research (OMAR). The topics are selectedby the relevant institutes, but OMAR helps tomake the final decision in cooperation with thebureaus, institutes, and divisions of NIH aboutthe suitability of the topic, panel composition, andthe proposed format for a consensus exercise. Ad-versary groups and task forces have been almostentirely abandoned. Moreover, the questions thathave been posed to the conferences have been ad-dressed strictly to those issues on which there isenough factual evidence to reach agreement. Un-like the techniques discussed above, consensusconferences have no particular theoretical basesfor their format.

A panel of experts is selected by NIH to hearpresentations by the leading medical researchers

● NCHCT, which was established in 1978, was not funded in 1982.

addressing a prespecified set of questions aboutthe technology. These presentations, usually sum-marizing the latest research findings, are madeover a 2-day period during which both panelistsand audience members discuss the research find-ings. On the evening of the second day, the panelis requested to draft a statement responding to thequestions. Usually, the panel deliberates throughmuch of the night, often writing four or moredrafts of its consensus statement. In some rarecases, minority reports are developed to indicatedisagreement with the majority recommendations.The next morning, the consensus statement is readto the audience for their comments and criticisms.The conference concludes with a press conference.After the panel disperses, it sets about the finaltask of revising the statement. The statementsfrom consensus development conferences arewidely disseminated to thousands of organizationsand individuals by NIH and by publication ar-rangements with leading medical journals such asthe New England Journal of Medicine.

Knowledge Base Development

In response to the often overwhelming numberof articles and other information concerning a par-ticular topic, NLM’s Lister Hill National Centerfor Biomedical Communications developed aunique system for soliciting expert opinion. NLM’ssystem, termed the “Hepatitis Knowledge Base, ”is intended to function as a continually updated,computerized body of knowledge concerning viralhepatitis (129). Although this system’s topic is adisease rather than a technology, its approach tosoliciting and maintaining expert opinion couldalso be applied to the latter.

Information for the Hepatitis Knowledge Basewas originally assembled by a small body of ex-perts who reviewed and combined the critical in-formation from an identified set of 40 importantreview articles on the subject. Any inconsistency

in the information was addressed by these experts;they either reached a consensus or noted that therewas an unresolvable conflict.

The resulting base of “knowledge” was com-

puterized and made immediately accessible toeach expert within this group. Subsequent infor-mation which appears in the literature is reviewed

— -—..


Table I.—NIH Consensus Development Meetings,September 1977 Through October 1982

Sponsors Title Dates held

NCINCI

NIDRNCINIANINCDS

NIAID

NCINIGMSNIAMDDInteragency Committeeon New Therapies forPain and Discomfort(Organizer)NICHDNHLBI

NHLBI

NCI

NCINEINIANIAID

DRS

NIDRNHLBININCDSNCINCI, NIA, NICHDa

NIAMDD a

NICHD a

NCI

NHLBl a

NINCDS

NINCDSNIAIDNIAIDNIAID

- - --Breasl Cancer screeningEducational Needs of Physicians and thePublic Regarding Asbestos ExposureDental Implants Benefit and RiskMass Screening for Colo-Rectal CancerTreatable Brain Diseases in the ElderlyIndications of Tonsillectomy andAdenoidectomy: Phase IAvailability of Insect Sting Kits toNon physiciansMass Screening for Lung CancerSupportive Therapy in Burn CareSurgical Treatment of Morbid ObesityPain, Discomfort, and Humanitarian Care

Antenatal DiagnosisTransfusion Therapy in PregnantSickle Ceil Disease PatientsImproving Clinical and Consumer Use ofBlood Pressure Measuring DevicesThe Treatment of Primary BreastCancer: Management of Local DiseaseSteroid Receptors in Breast CancerIntraocular Lens Implantationa

Estrogen Use and Post-Menopausal WomenAmantadine: Does It Have a Role in thePrevention and Treatment of Influenza?The Use of Microprocessor-Based“intelligent” Machines in Patient CareRemoval of Third MolarsThrombolytic Therapy in ThrombosisFebrile SeizuresAdjuvant Chemotherapy of Breast CancerCervical Cancer Screening: The Pap SmearEndoscopy in Upper GI BleedingChildbirth by Cesarean DeliveryCEA and Immunodiagnoses

Coronary Artery Bypass Surgery:Scientific and Clinical AspectsThe Diagnosis and Treatment ofReye’s SyndromeCT Scanning of the BrainDefined Diets and Childhood HyperactivityTotal Hip Joint Replacementlmmunology—The Bee Sting

Sept. 14-15, 1977May 22, 1978

June 13-14, 1978June 26-28, 1978July 10-11, 1978July 20, 1978

Sept. 14, 1978

Sept. 18-20, 1978Nov. 10-11, 1978Dec. 4-5, 1978Feb. 16, 1979

Mar. 5-7, 1979Apr. 23-24, 1979

Apr. 26-27, 1979

June 5, 1979

June 27-29, 1979Sept. 10-11, 1979Sept. 13-14, 1979Oct. 15-16, 1979

Oct. 17-19, 1979

Nov. 28-30, 1979Apr. 10-11, 1980May 19-21, 1980July 14-16, 1980July 23-25, 1980Aug. 20-22, 1980Sept. 22-23, 1980Sept. 29-Oct. 1, 1980Dec. 3-5, 1980

Mar. 2-4, 1981

Nov. 4-6, 1981Jan. 13-15, 1982Mar. 1-3, 1982Oct. 6-8. 1982b

aJointly ~pon~ored with the National Center for Health Care Technology.

bPlanned date.

SOURCE: National Institutes of Health, Office for Medical Applications of Research, 1982.

individually by the panel, consensus is sought, To date, the Hepatitis Knowledge Base systemand the knowledge base is updated. The panel can for reaching consensus on a timely, complex bio-caucus through a technique known as “computer medical topic is a research effort only. It has notconferencing” without being together geo- been evaluated for either clinical utility or costgraphically or temporally. As time permits, each effectiveness.member can access the terminal o enter his orher comments and to review the comments of An important outgrowth of the Hepatitisothers. Knowledge Base research effort is the develop-


ment of the Knowledge Base Research Program.This experimental program, which currently in-cludes viral hepatitis, peptic ulcer, and humangenetics, may contribute to more effective accessto and use of available biomedical informationin solving the daily problems of diagnosis, prog-nosis, and treatment of illness. The program isexploring ways by which medical computer scien-tists and medical subject matter experts can selectand organize relevant and accurate informationfrom the biomedical literature.

Advantages and Disadvantages ofFormal Group Process Methods

The methods discussed above are the betterknown and developed methods designed to facil-itate group decision processes and make themmore efficient. These methods are also intendedto produce more reliable and valid informationthan an unstructured conference. Evidence of ef-fectiveness, however, is somewhat contradictoryfor the Delphi and nominal group techniques, andsparse for the newer methods such as NIH’s con-sensus development conferences and NLM’sknowledge base.

The relative strengths and weaknesses of boththe Delphi and nominal group process methodshave been summarized by Delbecq, et al. (82).They believe that these methods are superior tosimple, unstructured group interaction providingfor a higher level of independent thinking bothin terms of quantity and quality, especially withrespect to specificity. Delphi seems to be particu-larly relevant for generating predictive informa-tion (78) when data are poor and for resolvinghighly controversial issues likely to be distortedwhen participants interact with one another.

Nevertheless, in comparison to formal decisionanalysis, the Delphi technique has been criticizedas being little better than the “seat-of-the-pants”methods currently employed by policymakers andas being a method which bases “knowledge” onan informal set of opinions (332). Others (10)maintain that it is subject to the same total erroras most predictions. The process is also time- andgroup-dependent, because the results are basedon information available to a specific group ofexperts at a specific point in time. As a conse-

quence, the process should be repeated as datachange with time. It also appears less well suitedas a process for resolving minimally controver-sial issues (318) or for synthesizing the state ofthe art in a given field (163).

A recent study compared Delphi with the nom-inal group process technique (368). Physicianswere randomly assigned to one of three Delphior nominal group technique panels to developprocedures for handling four hypothetical emer-gency medical services cases. In order to deter-mine the reliability of the decisions, panelists werecontacted individually 6 months later and askedto cast an anonymous vote on the proceduresoriginally discussed. The degree of consensusachieved was the same for both techniques. Themost striking finding, however, concerned the reli-ability of decisions over time. There were “veryextensive” changes in the nominal group techniquevote 6 months later, suggesting that it is “a lessthan reliable technique for reaching a consensus. ”Although the physicians reported that they likedthe nominal group technique much more thanDelphi, group norms and pressures were devel-oped using the nominal group technique that pro-duced unstable or false consensus.

Virtually no critiques of NIH consensus confer-ences have been published to date. However, amajor study is currently being funded by NIH toevaluate its process. A recent Institute of Medicine(IOM) publication notes that consensus confer-ence results were reported to be particularly usefulin health planning, quality assurance, and settingreimbursement (197). Although no current evi-dence is available regarding the usefulness and im-pact of the conference results, NIH is planning tofund an ambitious impact study. Wagner sug-gested that the success of any consensus formatis dependent on the following considerations(197):

1.

2.

3.

the composition of the evaluation panel, es-pecially participation by epidemiologists andbiostatisticians;the amount, quality, and comprehensivenessof available data;the duration of the process (sufficient timefor participants to synthesize information iscritical; several meetings held during a longer

.


period are preferable to an intensive, one-time meeting); and

4. the resources for support.

Although highly structured, the NIH consen-sus format, is not designed, as the Delphi andnominal group techniques are, to limit group dy-namic problems such as potential dominance byselected individuals or groups within the panel.From a methodological perspective, two aspectsof the NIH consensus development process are ofconcern: its sensitivity to the limitations of theresearch evidence and the extent to which it con-siders a comprehensive and systematic review ofthe research literature. However, NIH is awarethat its search for consensus resolution must be

confined within the limits of the expertise andevidence assembled. In order to broaden the ex-pertise and evidence to the maximum extent prac-ticable, NIH strives for diverse, open, and bal-anced representation and participation at its con-sensus conferences. It also performs an exhaustivereview of the research literature to compile rele-vant background evidence. A more complete dis-cussion of NIH consensus meetings is presentedin appendix C.

Relatively less is known about the value of theknowledge base approach. Nevertheless, the con-cept seems extremely interesting and potentiallyquite helpful to researchers and clinicians alike.One major concern is the cost of such a system.

DECISIONMAKING UNDER UNCERTAINTY

Elements of uncertainty abound throughout theprocess of assessment. Over the years, varioustechniques have been developed to assist in mak-ing rational decisions under such conditions. Thediscussion here is intended to introduce the readerto the more useful techniques for evaluating un-certainty within the field of medical technologyassessment. It is not an exhaustive description ofsuch methods, nor is it intended to provide thereader with a complete understanding of any onetechnique.

It is important to distinguish between two typesof uncertainty: that which reflects the presenceof random events and that which reflects a basiclack of knowledge. Random events occur accord-ing to a known probability distribution. For exam-ple, the flipping of a coin will result in headsroughly half of the time and tails the other half.Thus, although the result of a single flip of a coinremains uncertain, the probability distribution ofheads and tails is known. By contrast, when un-certainty reflects a lack of knowledge, one notonly does not know the probabilities of variousoutcomes’ occurrence, one may not even knowwhich types of outcomes can occur. Thus, admin-istering a brand-new chemotherapeutic agent toa terminal cancer patient may have any of severaltherapeutic and toxic effects; conceivably, the lat-ter might include side effects never previously rec-ognized in cancer chemotherapy.

Uncertainty due to random events can be ade-quately handled by a variety of techniques. Acommon approach is statistical confidence limits.Another common method is decision analysis. *Possible courses of action (outcomes, etc.) are dia-gramed on a “decision tree. ” Branches in the dia-gram are associated with known or imputed prob-abilities and “payoffs. ” As a result, an outcomevalue (positive and negative) is associated witheach pathway. Thus, analysts can trace plausi-ble paths to determine the probability and ex-pected value of each final outcome. (For a discus-sion of decision analysis in clinical decisionmak-ing, see ref. 386. )

In addition to decision analysis, a variety ofcomputer simulation techniques allow analysts tomodel real-world phenomena and estimate theirconsequences over hypothetical periods of time.By manipulating all such models until outcomesmirror empirical findings, analysts may be ableto acquire valuable insight into real-world proc-esses. The potential usefulness of modeling tech-niques is great, but analysts and policymakersshould always retain an awareness of the influenceof underlying assumptions. Technical sophistica-tion can mask tenuous assumptions, particularlyfor those individuals who lack familiarity with theanalytical approaches.

*Decision analysis is also helpful for uncertainty due to lack ofknowledge.


One general approach to handling uncertaintyis called sensitivity analysis, which is a conceptu-ally simple but powerful tool with which to ad-dress both random and nonrandom events. Actu-ally a series of techniques, sensitivity analysis cantest whether variations in assumptions affect thequalitative conclusions of an analysis. It is partic-ularly useful in CEA. For instance, if an analystassumes a discount rate of 2 percent and concludesthat the program in question is desirable (i.e., itsbenefits exceed its costs), he or she can try dis-count rates of O to 4 percent to determine whetherthe program’s basic desirability is a function of,or is sensitive to, the discount rate. Thus, sensitiv-ity analysis can shed light on the importance ofcertain assumptions, especially as to whether ananalysis is meaningfulcertainty.

CONCLUSION

despite the presence of un-

The techniques for synthesizing research, CEA,soliciting group opinions, and decisionmaking allhave the common goal of making sense out of abody of information.

In technology assessment, the objective of syn-thesizing the information gained from multipleresearch studies is to understand the cause andeffect relationships from the use of a technology.As chapter 3 explained, research studies, no mat-ter what type of design, never provide perfect in-formation and often provide only insights intorelationships. Most of the synthesis techniquesdescribed in this chapter were developed to pro-vide analyst/researchers with systematic meansto make those relationships clear. These techni-ques are primarily concerned with safety, efficacy,and effectiveness.

CEA can be regarded as adding at least onemore dimension to the synthesizing techniquesdescribed above. Traditionally, CEA has beenused to balance the safety /efficacy/effectivenessof a technology with its economic effects. How-ever, a recent OTA report, The Implications ofCost-Effectiveness Analysis of Medical Technol-ogy (270), described how social values can alsobe incorporated into the analytical framework.

Sensitivity analysis can produce four importantresults. First, it can demonstrate that a conclu-sion of a study substantially depends on a partic-ular assumption, thereby suggesting that the over-all analysis cannot be viewed as “definitive. ” Sec-ond, it can demonstrate that a conclusion is “ro-bust” with respect to a particular assumption (i.e.,that violation of the assumption does not signifi-cantly affect the conclusion) and, hence, that thetenuousness of the assumption is not a source ofconcern. Third, it can establish a minimum ormaximum value which a variable must have fora program to appear worthwhile. Finally, sensi-tivity analysis can identify issues (uncertainties)deserving of research attention.

The 10 principles of analysis, described by OTA,are intended to make CEA more policy-relevantthan most applications of the technique are. But,as OTA concluded, CEA is a decision-assistin g

rather than a decisionmaking tool.

A systematic approach to decisionmaking is tosolicit group opinions, This approach can be usedto synthesize diverse individual and group values

with clinical research findings and economic ef-fects. Several of the group process techniquesdescribed in this chapter were developed and re-fined by the social science community to maximizethe benefits and minimize the liabilities of groupdynamics. Evidence indicates that these formaltechniques produce more and better informationthan unstructured conference-type sessions. Todate, however, there have been few broad applica-tions of these techniques.

One purpose of assessing medical technologies,is to produce information needed by policy mak-ers. Policies that affect the development, diffu-sion, and use of medical technologies—in par-ticular, drugs, medical devices, and medical andsurgical procedures—are described in the nextchapter.

6Factors Affecting the

Development, Diffusion, andUse of Medical Technologies

Technology is not an historical institution like science and capitalism, butis rather the essence of man in an active and uncritical state.

—H. T. Wilson

Contents

Page

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Innovation Process for Medical Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sources of Medical Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Drugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Medical Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Medical and Surgical Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Influences on the Diffusion of Innovations.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Programs and Policies Related to the Development, Diffusion, and Use of

Medical Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Research Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Federal Research Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Private Sector Research Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Regulatory Responsibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Federal Regulation of Drugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Federal Regulation of Medical Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Other Regulatory Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Reimbursement Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Variations of Reimbursement Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Influence of Reimbursement on the Development, Diffusion, and Use of

Medical Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Coordination Efforts and Dissemination of Information . . . . . . . . . . . . . . . . . . . . . . . . . .

Federal Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Other Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... ....O+.

List of Tables

71717171727373

74747577787879808181

8284848586

Table No. Page

2. Number and Amount of NIH Support for Clinical Trials Active in Fiscal Year1979 by Institute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ........ 75

3. Amount of NIH Support for Clinical Trials Active in Fiscal Year 1979 by Institutefor Type of Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

●

6Factors Affecting the

Development, Diffusion, andUse of Medical Technologies

INTRODUCTION

Information on medical technologies is neededby various private and public parties at everystage in the development and diffusion of medicaltechnologies. All markets need good informationto function effectively. However, the market formedical technologies has some unique character-istics that affect the type of information neededand the timing of that information.

Because of the many social values related tomedical care, there is a degree of public respon-sibility for ensuring that medical technologies aresafe, efficacious, and, in some cases, cost effec-tive. Various Federal agencies provide researchfunds, regulate market entry, and decide whichmedical technologies will be reimbursable forFederal beneficiaries. Technology assessment in-formation is often needed in order to make re-sponsible decisions that protect the public safetyand the public purse, yet do not unduly impedethe innovation process.

This chapter is intended to set the stage for acritique of the current “system” for assessingmedical technologies. The first section summarizesselected aspects of the innovation and diffusionprocess for drugs, devices, and surgical and med-ical procedures. The second section explores re-search, regulatory, and reimbursement policiesthat affect the development, diffusion, and use ofmedical technologies.

Additional material concerning the drug anddevice industries and the Federal policies that af-fect the innovation process is presented in ap-pendix D. Appendix E contains five case studiesof medical technologies which are intended to il-lustrate some of the points made in this chapter,especially regarding the way in which technologiesare developed and adopted and the effects whichFederal policies have on the process.

INNOVATION PROCESS FOR MEDICAL TECHNOLOGIES

In some respects, the innovation process formedical technologies parallels that for other tech-nologies. Although there are many variations, abasic model of the process can be outlined. Aninnovation is conceptualized by recognizing bothtechnical feasibility and potential demand. If adecision is made to pursue the innovative idea,problem-solving activity follows, drawing fromavailable information and from further researchand development (R&D) activities. If a solutionto the problem is found, it may be the one origin-ally sought, or a solution to a modification of theoriginal problem.

Sources of Medical TechnologiesDrugs

The U.S. pharmaceutical industry is composedof about 600 firms. These include a small numberof large firms which produce most of the inno-vative drugs and a much larger number of rel-atively small firms which produce and marketmainly generic and some patented drugs. The in-dustry is characterized by high and rising develop-ment costs for new products, and there has beena strong shift toward greater concentration of newproducts in the largest firms.

71


Since the late 1950’s, the number of new chem-ical entities and number of firms producing a newchemical entity has declined (174). Innovative out-puts have been concentrated in the 20 largest ofthe 600 drug firms, and most of this concentra-tion is among the top four to eight innovators.From 1957-61 to 1967-71, the four largest firmsin the industry nearly doubled their share of in-novative output. During the same period, how-ever, their share of total prescription drug salesremained fairly constant.

These observations suggest that most of thelarge drug firms are dependent on a few drugs formuch of their income. Apparently, after thepatents for new products expire, generics erodesome of the market captured by the innovatingdrug firm, and the large innovative firms regaintheir share of total sales through the introductionof new drugs.

The Food and Drug Administration’s (FDA’s)regulatory responsibilities regarding new drugs arediscussed in the next major section of this chapter.

Medical Devices

The U.S. medical devices industry has experi-enced substantial growth since World War II. In-dustry sales in 1977 were $8.1 billion—five timesthe amount in 1958 (corrected for inflation).Growth has been predominantly in the numberof firms rather in their size. The U.S. medicaldevices industry is composed of several thousandfirms—many specialized small firms which to-gether have a small share of the market and a fewlarge firms with a high market share. There arehigh entry and exit rates in the industry, mostlyamong small firms (8).

Dominance by large companies suggests thepresence of economies of scale, while the per-sistence of many small companies suggests thateconomies of scale do not apply to specializedareas. Possibly, however, the large firms reallyrepresent the industry; i.e., rather than represent-ing the differentiation of the industry into smalland large functions, the large number of smallfirms may represent a high-birth, high-mortality,and high-turnover sector of the industry (122).Arthur Young & Co.’s (8) survey of the industry,

for example, did not differentiate between bank-ruptcy and acquisition in its observation of high-turnover rates for small firms. However d’Arbel-off (79) comments that high-turnover rates mayreflect a high-risk, high-profit atmosphere forsmall firms.

In general, small firms fill a special niche in themarket, and their growth into larger firms ishindered by conditions such as advertising re-quirements, links with distribution channels, andthe need for new capital expenditures (355). Thus,the industrial pattern is that of limited internalgrowth, with acquisition or establishment of ad-ditional companies being the primary method ofexpansion. Small plants are opened to manufac-ture new products following invention and devel-opment, while large plants are opened by largecompanies to take advantage of lower operatingcosts. These large companies tend to be extreme-ly diversified as a whole, yet there is little prod-uct diversification within their medical devicesplants (8).

Recently, the distribution of medical devices hasshifted from small regional and local suppliers tomajor national dealers. National dealers are oftensubsidiaries of large manufacturers or are ac-quirers of small manufacturing firms. The advan-tage of larger firms is that they are better posi-tioned to provide special buyer education throughtheir larger, better trained staff (355). The inabilityof potential manufacturers to gain access to thesenetworks is an additional barrier to growth of thesmall firms entering the medical devices field andprobably accentuates their acquisition by largermanufacturers.

The U.S. medical devices industry is somewhatinsulated from price competition by the high levelof third-party reimbursement, and price competi-tion is not as significant a force in mitigating priceincreases as it is in other industries. Nevertheless,there is a high degree of product differentiation,and the industry appears to be competitive at var-ious levels even though the market for the mostpart is price insensitive (8).

FDA’s regulatory responsibilities regardingmedical devices are discussed in the next majorsection of this chapter.

Ch. 6—Factors Affecting the Development, Diffusion, and Use of Medical Technologies . 73

Medical and Surgical Procedures

The invention, development, and diffusion ofmedical and surgical procedures can to somedegree be described by the model of the innova-tion process developed for products and theirmanufacturing processes. For the most part, how-ever, medical and surgical procedures are devel-oped within the practice of medicine. Further-more, the initial diffusion of medical and surgicalprocedures is relatively uninhibited by Federal reg-ulation.

New procedures usually involve some drugand/or device. However, a focus on proceduresseparate from the technologies which are used inthem is necessary, because physicians, as users,are both generators (technology-push) and pur-chasers (demand-pull) of these innovations. Thus,it is important to understand how physicians per-form these dual roles. But there are no standarddeterminants of when or how procedures becomemedically acceptable (197) and few criteria forwhen they become obsolete.

Influences on the Diffusionof Innovations

The medical literature on communication aboutand adoption of innovations is weighted towardstudies of single diagnostic or therapeutic medicaltechnologies. There is a large literature on howphysicians learn about and adopt new drugs anda growing literature on specific devices or tech-niques, but very little literature on the com-munication about or the adoption of complexmedical procedures that may not involve drugsor hardware (e. g., psychotherapy). In practice,however, the crucial distinction is between com-munication which informs the physician aboutnovel technologies and that which influencesphysicians to act (405). Even though the most im-portant source of new knowledge about improve-ments in medical technologies is the professionalliterature, physicians cite professional colleaguesmore often as sources they turn to when decidingto use a new procedure for the first time (145,233,234).

The importance of informal communicationboth in the process of scientific discovery and inthe diffusion of technological innovation seems

to be a feature in all fields of technological dis-covery and diffusion (213). Moreover, it may bethat there is a prestige hierarchy, where those atthe top are “trend setters” (49). If this is so,widespread adoption of an innovation could beenhanced by convincing influential organizationsto adopt it first, then letting prestige-seeking or-ganizations imitate them.

Physicians of greater prestige do tend to hearabout innovations sooner than others (62), andthey are also mentioned by their fellow profes-sionals as influential sources of information onthe medical practice of others. However, theadoption process when the adopting unit is anorganization (e.g., hospital) is substantially dif-ferent from the process when the adopting unitis an individual (e.g., physician in solo practice)(178,405), and these processes differ by the levelof complexity of the organization. Outside forcessuch as third-party reimbursement or regulatorypractices may also affect how quickly the in-dividuals in the medical community learn aboutor adopt a technology.

These theoretical and empirical findings pointto a kind of general scenario. Medical and surgicalprocedures usually begin as user- (e.g., physician)generated innovations. In medicine, an innovativeprocedure may be in the form of the adoption ofan existing drug for a new purpose or changingthe mixture of drugs and their dosages to adaptthem to a different medical problem. In surgery,it may be in the form of a modification of an ex-isting technique (usually accompanied by modi-fications of the devices being used) for applica-tion to a new use. In treatment areas that do notdepend on drugs or devices (e.g., psychotherapy)or in which drugs and devices are used but arenot crucial to the innovation (e. g., primary care),it may be an innovative interpretation of the ex-isting knowledge (e.g., the multiple schools of psy-chotherapy which have sprung up or the new spe-cialty of “family practice”).

Increasingly, innovations in procedures arise inacademic or academic-associated centers, wherephysical and professional resources are readilyavailable; a research, innovation-seeking at-mosphere is encouraged; and contacts with othersin the field extend not only nationally, but alsoglobally. Innovators in such settings know how


to present the innovations in a manner that willbe technically acceptable, and they also have theprestige that gives them access to professionalmeetings and journals to publicize their results.Their presentations and publications not only dif-fuse the innovation to a wider audience, but, moreimportantly, begin to legitimize it. Depending onthe claimed innovation’s nature, usually definedin terms of how it might revolutionize or at leastsubstantially affect the related area of medical orsurgical practice, other academic centers will beginto pursue the innovation as well.

At this point, several Federal agencies may enterthe picture. The National Institutes of Health(NIH) may provide support for the innovator andresearchers in other health centers in the form ofrandomized clinical trials (RCTs), most likely con-ducted in some of the clinical research centersfunded by NIH. A new use for a drug, inventionof a new device, or modification of an existingdevice requires FDA approval. Investigationalnew drug or device uses approved by FDA forlimited testing are increasingly given to the samecenters which NIH supports as clinical researchcenters (or at least to the health institutions inwhich these designated centers are located).

Sooner or later, the Health Care Financing Ad-ministration (HCFA) may receive a request forreimbursement of the procedure and will givegreat weight to NIH clinical trials for evidence ofsafety and efficacy. Meanwhile, however, FDAmust make a determination of safety and efficacyfor market clearance of the drug or device under

review. FDA will often have to make its decisionlong before NIH reaches a decision and terminatesfunding for the clinical trials. The reason is thatFDA must act in a timely manner and reach itsconclusion on minimal evidence, while NIH hasno similar regulatory responsibilities and is moreinterested in the cumulative evidence. FDA’s deci-sion, moreover, especially in the case of devices,may rest on the narrow question of the efficacyand safety of the device in a particular setting,not of the entire procedure in general use. Butrelease of the device to the general market, oncepremarket approval is given, tends to speed upthe diffusion of the procedure which NIH may bestudying. This may place more pressure on HCFAto reimburse for the procedure.

Although there are no explicit data on whichto estimate the developmental costs of medical in-novation, they are without a doubt very large.Commonly, the costs of the developmental phaseof early clinical application have been paid by pa-tients, usually through standard medical insurancepolicies. Even for procedures that have been clear-ly designated as experimental, reimbursement hasoften been provided. Thus, for example, whentotal hip replacement was first introduced into thiscountry in 1971, it was still an experimental pro-cedure; reimbursement for the procedure wasnevertheless provided.

PROGRAMS AND POLICIES RELATED TO THE DEVELOPMENT,DIFFUSION, AND USE OF MEDICAL TECHNOLOGIES

The public and other organizational policies described in previous OTA reports (266,270,274,and programs related to the development, diffu- 281).sion, and use of medical technologies can bebroadly classified under four headings; 1) research Research Activitiesactivities, 2) regulatory responsibilities, 3) reim-bursement policies, and 4) coordination of assess- R&D activities are an integral part of the in-ment activities and dissemination of information. novation process. Funding for the basic researchSome of these policies and programs have been which advances medical care comes primarily

Ch. 6—Factors Affecting the Development, Diffusion, and Use of Medical Technologies ● 7 5

from NIH, with smaller but important amountsfrom other Government agencies, industry, andprivate foundations (222). The central role thatbasic research plays in the process of medical in-novation (64) is the justification for the substan-tial public and private moneys invested.

The development and diffusion phases of med-ical innovation are also central to the innovationprocess, but for these phases there is relatively lit-tle formal public funding. NIH, whose primaryfocus is research, appears to have no systematicor comprehensive policy of support for technol-ogy development. Although NIH grants and con-tracts have been given to support technology de-velopment in a number of areas (e.g., the artificialheart program, cancer screening, cancer chemo-therapy, and, in recent years, hemodialysis), fig-ures to document the size of NIH investment indevelopment are not available. The amount in-vested in development probably constitutes a rel-atively small part of the current $3.8 billion NIHbudget.

In the discussion that follows, the emphasis ison public and private research related to medicaltechnology assessment. Information derived fromthis research should be useful in setting policiesaffecting medical technologies.

Federal Research Activities

Medical care research is of two general types:1) biomedical research, and 2) health servicesresearch. More than a dozen Federal agencies areinvolved in conducting biomedical research, butthe 11 institutes of NIH receive approximately 70percent of Federal funds (228). The primary spon-sor of health services research is the NationalCenter for Health Services Research (NCHSR).HCFA sponsors additional health services re-search, but tends to focus it on the needs ofMedicare and its other operating programs, Afourth Federal agency, the National Center forHealth Care Technology (NCHCT), was estab-lished in 1978 to support evaluations of health caretechnologies, but did not receive funding for fiscalyear 1982.

NIH and each of its institutes divide theirresources among extramural grants, contract re-search, and intramural projects initiated by scien-

tists within NIH. The Federal agencies that sup-port biomedical and health services research relyon a peer review system to judge proposed proj-ects (270). The peer review system of NIH, forexample, consists primarily of non-Federal scien-tists and lay advisors from across the Nationgrouped into 130 peer review groups, advisorycommittees, councils, and panels (125). They pro-vide NIH with opinions on the scientific and tech-nical merits of grant applications and contractproposals and on program initiatives and policyissues (270).

NIH both conducts its own testing and encour-ages and funds medical research and testing ac-tivities in academic centers and research institu-tions. It has funded studies of drugs, devices, andmedical and surgical procedures, though it gen-erally does not synthesize the evidence garneredfrom these efforts (266). NIH is the largest singlesource of funds for the support of RCTs in thecountry. Such trials, as described in chapter 3,are a key method for obtaining information aboutthe safety and efficacy of certain medical technol-ogies. In 1975, NIH provided approximately $110million for clinical trials, a figure representing 5percent of its total budget (266). By 1979, sup-port for clinical trials had increased to over $135million. Tables 2 and 3 summarize NIH’s supportfor clinical trials during fiscal year 1979.

NCHSR conducts and sponsors a wide varietyof health services research. This agency has threeprincipal responsibilities: 1) to develop informa-tion that might be used by various decisionmakersin the public and private sectors; 2) to serve asthe focal point for coordination of health services

Table 2.—Number and Amount of NIH Support forClinical Trials Active in Fiscal Year 1979 by Institute

Institute Number of trials Amount of support

NIH . . . . . . . . . . . . 986 $136,160,1 16’NEI . . . . . . . . . . 26 8,605,609NHLBI . . . . . . . 20 56,523,501NIAID . . . . . . . . 120 6,496,938NIAMDD. . . . . . 67 8,240,133NICHD . . . . . . . 32 4,183,244NIDR . . . . . . . . 26 1,778,699NINCDS . . . . . . 40 2,660,949NIGMS . . . . . . . 1 225,750NCI . . . . . . . . . . 654 47,445,293’

‘One trial dld not report amount of support

SOURCE NIH Inventory of Cllnlcal Trials, 1979


Table 3.—Amount of NIH Support for Clinical Trials Active in Fiscal Year 1979 by Institute for Type of Support

Extramural support

Grant and Intramural Total amountInstitute Grant Contract a Contract Total Supportb of support

NIH . . . . . . . . . . . . . . . . . . . . $47,304,588C $75,738,768 $1,954,960 $124,998,316 $11,161,800 $136,160,1 16’NEI . . . . . . . . . . . . . . . . . . 3,141,547 5,378,262 – 8,519,809 85,800 8,605,609NHLBI . . . . . . . . . . . . . . . 4,006,736 50,933,477 159,788 55,100,001 1,423,500 56,523,501NIAID . . . . . . . . . . . . . . . . 2,435,341 3,827,597 – 6,262,938 234,000 6,496,938NIAMDD . . . . . . . . . . . . . . 1,927,658 5,226,975 – 7,154,633 1,085,500 8,240,133NICHD . . . . . . . . . . . . . . . 3,074,448 556,296 — 3,630,744 552,500 4,183,244NIDR . . . . . . . . . . . . . . . . . 221,977 557,672 – 779,649 999,050 1,778,699NINCDS . . . . . . . . . . . . . . 1,786,449 439,000 — 2,225,449 435,500 2,660,949NIGMS . . . . . . . . . . . . . . . 225,750 — — 225,750 225,750NCI . . . . . . . . . . . . . . . . . .

—30,484,682C 8,819,489 1,795,172 41,099,353 6,345,950 47,445,293 C

acOntraCt includes interagency agreements without intramural suPPo~.

~lntramural support includes intramural support in combination with interagency agreementsbOne trial did not report amount of support.

SOURCE: NIH Inventory of Clinical Trials, 1979

research within the Public Health Service (PHS);and 3) to ensure that results from its research,evaluation, and demonstration activities are dis-seminated rapidly and in a form which is usable(290).

NCHCT, though not funded for fiscal year1982, was required as part of its 1978 legislativemandate to undertake and support comprehen-sive assessments of health care technologies, in-cluding analyses of their safety, efficacy, andeconomic, social, and ethical implications.NCHCT had its own extramural program forawarding grants and contracts for assessments,research, demonstration, and evaluations in thefield of health care technology (90,112).

In addition to responding to Medicare coveragequestions (see discussion below), NCHCT iden-tified priority technologies for “focused” or “full”assessments. NCHCT selected technologies forthese assessments through the advice of the Sec-retary of Health and Human Services, its NationalCouncil, and others. Full assessments were com-prehensive, integrated analyses of a technology’ssafety, efficacy, and effectiveness, and any social,ethical, or economic implications. Such assess-ments usually involved commissioning an over-view paper, establishing a Federal planning group,establishing a full planning group, and conven-ing a conference. Conferences were held on suchtopics as coronary artery bypass graft surgery,dental radiography, cesarean section, and elec-tronic fetal monitoring (110,111).

HCFA's research objectives and priorities aredefined by the information needs of HCFA oper-ating programs. HCFA’s research and demonstra-tion mission is to improve the operating effec-tiveness of the Medicare and Medicaid programs.The agency’s Office of Research and Demonstra-tions is currently conducting over 200 intramuraland extramural projects on reimbursement issues,coverage eligibility, and management alternativesto present Federal programs, as well as on the im-pact of HCFA programs on health care costs, pro-gram expenditures, access to services, health careproviders, and the health care industry.

One focus of HCFA’s Office of Research andDemonstrations is on data acquisition and datamanagement systems. Over the past 3 years,grants have been awarded to develop “integrateddata demonstration” systems in a number ofStates, including Iowa, Maine, Massachusetts,Minnesota, Missouri, New York, South Carolina,and Vermont. * Each grantee proposes to developa central data base—often conceived of as a“clearinghouse” or “data broker’’—in whichnumerous types of billing and discharge abstractdata, and sometimes other types of data, will becollected and linked. Although the eventual goalis statewide implementation of a system that col-lects data on all patients, most grantees propose

● Recent decisions by HCFA have resulted in continued fundingof the demonstrations in South Carolina, Maine, Vermont, andMissouri. Demonstrations in Massachusetts and Iowa were up forrenewal later in fiscal year 1981.


initial implementation in only a few pilot or testhospitals (32,105,106,107). The Office of Researchand Demonstrations has several publications fordisseminating research and demonstration find-ings, which are also available through the Na-tional Technical Information Service.

Research and assessment issues are often initial-ly identified by HCFA in the form of reimburse-ment coverage questions: Is the test, procedure,or treatment regimen provided to a specific pa-tient “reasonable and necessary?” (305). Coveragequestions originate when a bill is submitted to theMedicare fiscal intermediary, whose medical di-rector is to determine whether an “unusual” med-ical event has occurred. In order to make a deter-mination, the medical director checks through a“buddy system,” contacting physicians in the localcommunity or recognized experts in the medicalfield involved. If he or she decides that the in-tervention was appropriate, the bill will be paid,and this limited review will be the only “technol-ogy assessment” the procedure will undergo. Ifsome question remains regarding the interven-tion’s appropriateness, however, the bill will beforwarded to the HCFA regional office, and thatoffice will investigate in much the same manneras the intermediary’s medical director. If the in-tervention is accepted in the region’s area, the billwill likely be paid. (Thus, the various regionaloffices may reach different conclusions on similarcoverage questions. ) In the event that there is stilluncertainty, however, the regional office will passthe question along to HCFA’s central office.

Until early 1980, HCFA’s procedures for mak-ing coverage decisions were highly informal. Thestaff of the Office of Coverage Policy, often withassistance from the Health Standards and Quali-ty Bureau, would review the issue, consult expertsin the field with whom they were acquainted, andcome to a decision. Although a formal agreementbetween HCFA and PHS had existed since around1966 (407), a somewhat more formal process in-volving a panel of physicians within HCFA andfrom NCHCT was established in early 1980.When HCFA decided that a procedure involveda question of national importance, a request fora technology assessment was sent to NCHCT(305). Usually, such a request asked NCHCT todetermine the safety and efficacy of a particular

technology and to recommend whether HCFAshould reimburse. Inquiries from HCFA coveredthe full spectrum of medical practice, ranging fromthe appropriateness of continued coverage forhighly questionable or obsolete to medical tech-nologies questions of reimbursement for new orinvestigative medical technologies. (Reimburse-ment policies have profound effects on the adop-tion and use of medical technologies and arediscussed in the section on reimbursement below.)

Private Sector Research Activities

Increasingly, the private sector is involved inevaluating medical technologies, especially todetermine their safety and efficacy. Manufacturersof drugs and devices initiate research and are re-quired by FDA to conduct tests for premarket ap-proval of their products. Large private clinics haveoften led the way in finding effective and efficientapplications. For example, the Cleveland andMayo Clinics were particularly active in the earlyevaluative efforts of the computed tomography(CT) scanner. The Cleveland Clinic has tradi-tionally supported strong research and assessmentprograms in cardiovascular diseases, including anartificial heart development program. The MayoClinic, long recognized for its contributions tobiomedical research, supports a methodological-ly sophisticated cadre of assessors and recentlyestablished a health care studies unit to examineproblems in hospital utilization and delivery ofmedical services in rural areas. The health carestudies unit began with $12 million in NIH grantsin 1975. By 1980, its total budget approached $41million, much of it private foundation money.Most of the unit’s research can be classified asnonrandomized in design, often relying on carefulrecordkeeping (11,37,347).

Other research activities have been undertakenby professional associations. The American Acad-emy of Pediatrics has developed recommendationsconcerning immunization practices. The Amer-ican Public Health Association periodically com-piles a list of effective preventive and therapeuticprocedures for infectious diseases (266). TheAmerican Hospital Association and the AmericanCollege of Radiology have been involved in sim-ilar activities (306).


There is also ECRI (formerly the EmergencyCare Research Institute), a nonprofit organizationprimarily involved in comparative product eval-uations of diagnostic and therapeutic devices andhospital equipment and supplies. ECRI providesa type of “Consumer Report” service for hospitaladministrators, which gives ratings to comparablemedical technologies based on performance, safe-ty, ease of use, and cost effectiveness. Its emphasison the larger economic, social, and ethical issuessurrounding health care technologies has recent-ly been expanded. Further, ECRI maintains a com-puterized health devices data base on over 6,000categories of devices and hospital equipment(251).

Some health insurance companies and nonprof-it organizations also provide funds for researchand technology assessment activities. For exam-ple, Blue Cross of Massachusetts has funded theclinical cost of an RCT comparing CT scanning,radionuclides, and ultrasound for the diagnosisof adrenal tumors, pancreatic diseases, and meta-static tumors of the liver (38). The studies werecarried out at the Peter Bent Brigham Hospital inBoston and at the Johns Hopkins Hospital in Balti-more, and the cost of analysis was paid by theAmerican Cancer Society. Blue Shield of Califor-nia, along with the Multiple Sclerosis Foundation,is exploring the feasibility of similar collaborationsin a clinical trial of plasmapheresis as a treatmentfor multiple sclerosis (38).

Approximately 15 years ago, medical insurancecarriers became concerned about reimbursing newtherapies that were still in the experimental phase.In 1966, therefore, Blue Shield of California es-tablished its Medical Policy Committee (primarilycomposed of physicians but also including mem-bers of the public and representatives of the Cal-ifornia Podiatry Association) to assess the scopeand limits of Blue Shield’s standard medical in-surance policies with respect to new diagnosticand therapeutic procedures (39).

In 1975, in addition to evaluating new pro-cedures, services, and technologies, CaliforniaBlue Shield’s Medical Policy Committee began toidentify obsolete procedures (155). This functionwas subsequently promoted by the Blue Cross andBlue Shield Associations under the Medical Ne-cessity Project.

One outgrowth of that project has been theClinical Efficacy Assessment Project (CEAP),undertaken by the American College of Physicianswith funding from the Hartford Foundation.CEAP’s specific objectives are to: 1) identify andevaluate technologies that are only partially ef-fective, 2) disseminate evaluative information ofpotential use to health care providers and third-party payers, 3) evaluate technologies that aremore efficacious in one setting than in another,4) obtain such measures as cost benefit/cost ef-fectiveness and marginal utility, and 5) discoverhow physicians decide about technology use.CEAP’s mandate at this time does not include theinvestigation of new or emerging technologies. Forthe foreseeable future, therefore, CEAP will re-view tests and procedures that are in current use(4).

Another important initiative of California BlueShield’s Medical Policy Committee, undertakenin the interests of cost containment, was the Am-bulatory Surgery Project. In 1976-77, it identifiedmore than 700 surgical and diagnostic proceduresthat could normally be performed in an ambula-tory setting without admission to a hospital (38).

Regulatory Responsibilities

To regulate effectively, the Federal Governmentmust obtain adequate information in a timelymanner. One major regulator, FDA, requiresmanufacturers of drugs and some medical devicesto submit information about their products whichis gathered according to approved research pro-tocols. Other regulatory mechanisms, includinglocal health planning agencies, are in need of in-formation but have no particular means to ob-tain it.

The major Federal health regulatory activitiesare described briefly below. The reader will notethat one primary effect that regulation tends tohave on medical technologies is to constrain theirdevelopment, diffusion, and in some instances,their use.

Federal Regulation of Drugs

FDA becomes officially involved in the devel-opment process for a new drug when the drug’s“sponsor” (e. g., manufacturer) files a “notice of


claimed investigational exemption for a new drug”(IND) for FDA’s permission to test the drug inhumans. If FDA approves the sponsor’s IND, thesponsor may proceed with clinical testing. Thereare three phases in the clinical investigation of anew drug, and each phase must have been pre-ceded by specified animal tests. (Test requirementsfor contraceptives are more stringent than the re-quirements set forth below for other drugs. )

Phase I studies are investigations of a newdrug’s clinical pharmacology to determine levelsof tolerance (toxicity), followed by early dose-ranging studies for safety (and, in some cases,efficacy) in selected patients. The total numberof both healthy volunteers and patients, whichvaries with the drug, ranges from 20 to SO. If thedrug is found to be safe, the manufacturer canproceed to the next phase of testing. Phase Istudies must be preceded by 2- to 4-week studiesin two animal species.

Phase II studies, designed to demonstrate effec-tiveness and relative safety of a new drug, are car-ried out on 100 to 200 patients under controlledconditions. If the drug’s therapeutic value isdemonstrated and there are no serious toxic ef-fects, the manufacturer can proceed to the nextphase of testing. Phase 11 studies must be precededby 90-day studies in two animal species.

Phase III studies are expanded controlled anduncontrolled clinical trials, involving 500 to 3,OOOpatients in usual medical care settings (clinics,private practice, hospitals). After completing clin-ical testing under IND, the sponsor of the drugmay submit to FDA a “new drug application”(NDA). An NDA is a request for FDA’s permis-sion to market the drug. At least two well-con-trolled clinical trials, accompanied by completecase records for each patient, are usually requiredfor FDA’s approval of an NDA. Chronic animaltoxicity studies (l-year dog, M-month mouse, and2-year rat studies) must be completed by the timeof the NDA submission. If FDA finds the effec-tiveness and toxicity data acceptable, it approvesthe NDA. Since 1962, FDA has reviewed over13,500 applications for INDs and has approvedabout 1,000 NDAs (154).

Once a drug is on the U.S. market, FDA haslittle control over its use or evaluation (274). Proc-

esses for collecting information on the safety (rareadverse reactions, long-term effects) and on theindications for use of drugs on the market are verylimited and for the most part voluntary. In re-cent years, there has been increasing discussionin the United States about relying more on post-marketing controls on drugs and relaxing the pre-marketing controls somewhat. *

Federal Regulation of Medical Devices

The 1976 Medical Device Amendments to theFood, Drug, and Cosmetic Act greatly expandedFDA’s role in regulating the safety and efficacyof medical devices. Prior to the amendments, FDAhad classified devices such as soft contact lenses,pregnancy test kits, intrauterine devices, nylonsutures, and hemostats as “drugs” (359). In 1969,the U.S. Supreme Court ruled that this move wasjustified since Congress intended the public to beprotected from unsafe and ineffective devices(299). The Medical Device Amendments of 1976established a three-tiered system of controls: ClassI, General Controls; Class II, Performance Stand-ards; and Class III, Premarket Approval. Eachdevice is required to be classified on the basis ofthe level of regulation needed to ensure its safetyand efficacy.

Class I devices are low-risk devices that are notused to support or sustain human health, andthese are subject primarily to the Food, Drug, andCosmetic Act’s basic prohibition against mis-branding and adulteration. Although Class I con-trols apply to accuracy in labeling and the sanita-tion and physical integrity of low-risk medical de-vices, all devices must meet these minimum stand-ards. FDA also has the power to ban any device,regardless of classification, which presents a sub-stantial deception or an unreasonable and sub-stantial risk of illness or injury that is not correct-able by labeling.

Class II devices are those for which general con-trols alone are judged insufficient and about whichsufficient information exists or could be developedto establish performance standards. FDA is au-thorized under the 1976 amendments to develop

and establish performance standards.

● This topic is explored at greater length in OTA’S reportPostmarketing Surveillance of Prescription Drugs (281).


Class.III devices are those devices that are life-sustaining, life-supporting, implanted, or presenta potential unreasonable risk of illness or injury,and for which general controls or performancestandards may not provide reasonable assuranceof the device’s safety and efficacy or for whichperformance standards cannot be developed.Class 111 controls are comparable to the premarketapproval process for drugs. * Any device whichwas classified as a “drug” before 1976 is auto-matically assigned to Class III unless reclassified.Any device developed after 1976 which is notjudged by FDA to be “substantially equivalent”to a preamendment device in Class I or Class 11will also be assigned to Class III and require a pre-market approval application. In the first 4 yearsafter implementation of the 1976 amendments,about 98 percent of the listed devices in the 10,540premarket notifications received were declared“substantially equivalent” to a preamendmentClass I or Class II device (270).

The 1976 amendments require any distributorof a device intended to be marketed for the firsttime to file a notice with FDA at least 90 days inadvance to permit the agency to decide whetherthe device needs premarket approval to assuresafety and efficacy. FDA permits earlier distribu-tion if it concludes and notifies the distributor thatpremarket approval is not required. If the 90 dayspass without comment from FDA, marketing canbegin. In 1981, FDA estimated that 2,300 premar-ket notifications would be reviewed.

Industry often uses FDA approval to advantagefor its marketing strategy. All results of clinicalinvestigations will ultimately be included in apackage insert, product data sheet, or physician’sbrochure, which are FDA-approved generators ofpromotional claims (300).

Other Regulatory Activities

Concerns about premature diffusion of themore expensive devices and other capital in-vestments led to the enactment of three overlap-ping Federal programs: 1) section 1122 review,

● The 1976 amendments also allow FDA to permit developing andmarketing approval of a Class 111 device under a “product develop-ment protocol, ” where FDA and manufacturer agree in advance ona plan for the development, testing, and release of the device. Thisapproach has not been implemented.

2) the 1974 National Health Planning and Re-sources Development Act, and 3) State certificate-of-need (CON) laws. Concerns about the utiliza-tion of health care services by beneficiaries of theMedicare and Medicaid programs led to the de-velopment of Professional Standards Review Or-ganizations (PSROs).

The first State CON law was enacted by NewYork in 1964. This law, subsequently followed bysimilar laws in other individual States, empoweredState planning agencies to deny reimbursementto hospitals for large capital expenditures unlessthe planning agency found a “need” for the serv-ice to be provided. In 1972, section 1122 of theSocial Security Act similarly authorized the Med-icare and Medicaid programs to withhold fundingfor depreciation, interest, and return on equitycapital for certain investments found not neces-sary by a health planning agency. State CON lawsand section 1122 review, in effect, constitute afranchising process for potential adopters of ex-pensive medical technologies.

Section 1122 applies to investments of morethan some specified amount (initially $100,000)and covers changes in beds and services that areprovided by certain health care facilities, such asambulatory surgical facilities. Private physicians’offices are explicitly exempted. In 1977, 37 Stateshad contracted with the Department of Health andHuman Services (DHHS)* to conduct section 1122reviews.

The National Health Planning and ResourcesDevelopment Act of 1974 required States to passCON laws by 1983 as a condition of future Federalfunding under the Public Health Services Act, theCommunity Mental Health Centers Act, and theAlcohol Abuse and Alcoholism Act. The originalact applied to the same facilities covered by sec-tion 1122 review. However, 1979 amendments tothe act exempted health maintenance organiza-tions (HMOs) from having to secure a CON forinpatient investments.

PSROs, enacted into law in 1972, are areawidegroupings of practicing physicians responsible forreviewing services provided and paid for by Med-icare and Medicaid, The purpose of their review

*Then the Department of Health, Education, and Welfare.


is to help assure that these services are: 1) medical-ly necessary, 2) of a quality that meets locallydetermined professional standards, and 3) pro-vided at the most economical level consistent withquality of care. However, the primary operationalmission of PSROs has been to constrain excessiveutilization of health care services which is fueledby the reimbursement incentives discussed in thenext section of this chapter.

Other regulatory-type mechanisms that havebeen instituted because of the high demand gen-erated by third-party payment include State hos-pital rate-setting programs, increasing Medicaredeductibles, setting low reimbursement levels forthe Medicaid population, decreasing Medicaidbenefits, and raising Medicaid eligibility require-ments. All of these affect the diffusion and levelof use of medical technologies.

Reimbursement Policies

Reimbursement policies have profound effectson the adoption and use of medical technologies.Reimbursement also influences the innovationprocess, especially for medical and surgical pro-cedures. With the increasing costs of medical carecontinuing to cause concern, reimbursement pol-icy is becoming even more important. Informedcoverage decisions may require even more de-tailed information concerning medical technol-ogies than regulatory decisions. Whereas reg-ulatory decisions tend to be more of a “go,” “nogo” nature, reimbursement decisions are, or atleast could be, more related to appropriate useof technologies, a much finer distinction.

The growth in third-party coverage of medicalcare is seen as a major cause of the excessive adop-tion and use of many medical technologies (142,33 I). It is important to note, however, that fac-tors other than reimbursement policy contributeto the overall tendency to adopt and use medicaltechnologies at excessive levels. Such factors in-clude competition among hospitals to achievequality and prestige to attract patients and physi-cians, public demand for sophisticated technolo-gies, increasing specialization within medicine,physicians’ desires to do as much as possible fortheir patients, uncertainties related to what con-stitutes appropriate use, and the defensive over-

utilization of medical tests and procedures becauseof the threat of malpractice suits.

Variations of Reimbursement Mechanisms

There are two basic forms of payment mecha-nisms in the U.S. medical care delivery system:cost-based and charge-based (305). Governmentprograms, primarily the Medicare and Medicaidprograms, were developed to “buy into” what wasthen perceived as a market pricing system. Thestatutes enacted in 1965 established the principlethat the Government purchaser would pay institu-tional providers the costs of services to patients.Physicians were to be paid their “usual, cus-tomary, and reasonable” fees. The original as-sumption was that the Government was buyingat the margin and would not affect the averagecosts of the system. Subsequently, however, itcame to be recognized that Government purchasesof medical services were sufficiently large to af-fect purchase price and costs. Thus, the 1972amendments to the Social Security Act placedlimits on the amount that would be paid by Med-icare to both institutional providers and physi-cians. Rather than being related to efficiency,these cost limits reflected rates of increases incharges over time.

In the “private” sector of the medical care mar-ket, there are two widely used mechanisms to setreimbursement levels. One, the cost-based BlueCross/Blue Shield reimbursement system is inmany ways similar to the Medicare program. Hos-pitals are reimbursed the “reasonable” cost of pro-viding care to patients, and physicians are paid“reasonable” fees. The second mechanism is pay-ment for billed charges and is used by some BlueCross/Blue Shield plans and in all contracts es-tablished between patients and other insurers topay the bills generated by the patient. Under thisapproach, all or some of the charges of hospitalsand other medical providers are paid through in-surers, unless there are copayments and deduct-ibles which are paid by the patient. Patients notcovered by Government or other insurers are re-sponsible for their own bills. Billed charges aremore like a market mechanism, except that de-mand is not directly affected by the income orwealth of the patient.

. —


A third payment mechanism, not very wide-spread, is cavitation, whereby a fixed amount ispaid for each patient per time period, regardlessof the health services provided. The cavitationmethod generally involves the integration offinancing and the delivery of services, thus plac-ing the provider of medical care at financial risk.

Influence of Reimbursement on the Development,Diffusion, and Use of Medical Technologies

When coverage for new and experimental med-ical and surgical procedures has been offered fromthe outset, a high level of reimbursement has beenjustified on the basis of the special skills and largeamounts of professional time required, and per-haps on the basis of increased risk. When suchprocedures have become routine, requiring lesstime and skill and posing lesser risks, however,professional fees have usually increased ratherthan fallen (316).

Several examples have been provided by BlueShield of California (39). Phakoemulsification ofthe crystalline lens, introduced as an alternativeto lens extraction for cataract, is—once learned—shorter and no more complex than standard lensextraction, yet surgeons initially charged 25 to 30percent more for the new procedure than for theolder, more costly procedure. In this instance,California Blue Shield’s Medical Policy Commit-tee disallowed the increase. Another example isthe flexible fiberoptic endoscope. Although thisnew instrument is easier to use than the standardrigid instrument, physicians introducing the newprocedure charged 25 percent more. A third ex-ample is arthroscopic menisectomy for torn kneecartilage. Orthopedic surgeons introducing thisprocedure wished to charge the full fee for thestandard open arthrotomy and an additional feefor the arthroscopy. In this instance, Blue Shieldof California agreed to pay the full arthrotomyfee and an additional 50 percent of the arthros-copy fee. The rationale for Blue Shield’s conces-sion was that the performance of the simpler pro-cedure might eliminate the need for many daysof hospitalization and laboratory tests, with a con-siderable net savings in total charges.

Allowing a simpler procedure to be billed asa more complex procedure has resulted in ques-

tionable increases in physicians’ fees. In the ex-ample of arthroscopy of the knee, the large dif-ference in allowable charges when an operativeprocedure is added to a diagnostic procedure of-fers a strong invitation to remove some tissue dur-ing arthroscopy. During the diagnostic examina-tion of the knee, a small piece of redundantsynovial membrane may be seen—a finding of nogreat importance. Removing a piece of this tissuemakes the procedure a “synovectomy,” for whichthe customary charge is $1,300, rather than simplya diagnostic arthroscopy, for which the customarycharge is only $500. The above scenario presentsa situation that may be reasonably justified med-ically, but, even interpreted generously, there isa clear fiscal invitation to perform a procedurethat is more, rather than less, complex.

There is also a much more serious consequenceof the manner in which charges are submitted forexperimental procedures. With increasing scrutinyby third-party payers of bills submitted for newprocedures and more than occasional denial ofpayment for such bills, there is a strong incen-tive for physicians to request payment for a stand-ard procedure rather than a new one. This prac-tice is also encouraged by the fact that new pro-cedures often do not have a procedure code num-ber, by which most bills are processed. Requestingpayment for a standard procedure may simplyreflect an honest effort to use whatever codenumber seems most nearly to approximate theprocedure actually performed. The net result,however, is that the identity of the new proceduremay be concealed, and the fact that an experimenthas been carried out may not emerge.

In bills submitted to Blue Shield of California,there is an approximately 15-percent error rate inthe coding of all procedures (39). The medicaldirector estimates that 1 percent of the errors in-volve the use of existing codes for procedures towhich new codes have not been assigned. Any in-novation that falls outside of “accepted medicalpractice” is particularly vulnerable to being mis-labeled. Because it is difficult to define exactlywhat constitutes accepted medical practice, thenew procedures that have the best chance of beingreimbursed are the ones that deviate the least fromexisting procedures which are already being reim-

Ch. 6—Factors Affecting the Deve/oprnent, Diffusion, and Use of Medical Technologies Ž 83

bursed. The Federal Government, for example,has traditionally favored coverage of new tech-nologies perceived to be modifications of existinginterventions (270). The incentives, therefore, aretoward the development of parallel procedures orextensions of existing technologies.

For procedures that deviate substantially fromaccepted medical practice, the reimbursement sys-tem may require considerable testing for safety,efficacy, and costs to determine if they offer suf-ficient contributions to compensate for theirdeviation from standard medical practice. Thesecircumstances have several implications. First,when procedures remain outside the coverablerange, they may also suffer the fate of anonymity,neglect, lack of funding, or underutilization. Anobvious example is the traditional exclusion frommost insurance plans of preventive medical care,most notably screening services. Second, thescrutiny of radical innovations rather than of in-cremental improvements may be misplaced to theextent that the growth in medical expenditures isthe primary reason for such scrutiny. The collec-tive expense of small tests and procedures isarguably far greater than that of a few ‘big ticket”technologies (249). Third, if radical innovationshave the most difficulty in receiving favorablecoverage decisions, innovators might be inclinedto pursue less radical but more easily accepted in-novations. This is a difficult hypothesis to test,as radical innovations have less chance of com-mercial success than minor innovations; but onceradical innovations penetrate the market, themagnitude of their commercial success is greaterthan for minor innovations. Fourth, a technology-by-technology approach to coverage decisions,with priorities determined by how radically eachtechnology differs from existing ones, may invitethose seeking payment for the use of new tech-nologies to submit their claims for payment underthe guise of accepted procedures.

Under either cost reimbursement or charge pay-ment, third-party payments generally are intendedto cover the full costs of new technologies, in-cluding purchase, maintenance, or operation ofequipment; the leasing of equipment; the costs ofdrugs; or the facilities and equipment needed fora procedure (19). One would expect that greateradoption of technologies would occur under these

relatively price-independent conditions thanwould occur under a more price-sensitive system.Cromwell, et al.’s (75), interstate analysis foundthat the hospital’s percentage of revenues fromthird parties was significantly and positivelyrelated to the hospital’s adoption of expensivetechnology. Russell (331) found that the adoptionof cobalt therapy and electroencephalograph oc-curred faster when the level of insurance coveragewas higher and proceeded more rapidly as thatlevel grew. She also found that a greater contribu-tion of hospital costs by Medicare was associatedwith increased adoption of cobalt therapy, inten-sive care beds, and diagnostic radioisotopes. AndWillems (392) concluded that open-heart surgeryspread more quickly in areas with faster growthin insurance coverage.

Third-party reimbursement can indirectly af-fect the adoption of technology by changingthe availability of financial capital to potentialadopters. A prominent example is the Medicareprogram, which reimburses institutional providersfor capital as well as operating costs. Medicarepayment for allowable capital costs such as de-preciation and interest provides a source of in-ternally generated funds (28). Third-party cov-erage, especially by Medicare and Medicaid, hasalso reduced hospitals’ risks of bad debts, therebyimproving their standing as credit risks to privatelenders. Other changes in governmental pro-grams, such as the Hill-Burton program for fund-ing medical facility construction and moderniza-tion, as well as various tax-exempt bond pro-grams, have also affected the sources of financialcapital.

In addition to affecting the adoption of tech-nologies, the extent of third-party coverage wouldbe expected to affect the use of technologies. Dataon the use of specific technologies are generally

lacking, however. Cromwell, et al. (75), foundthat many hospital technologies are underutilizedafter being adopted. Nonprofit hospitals in theBoston area were using automated analyzers andpatient monitors (and, in teaching hospitals, diag-nostic X-rays) at only half of capacity. Willems(392) considers such underuse as presumptive evi-dence of the hospitals’ overinvestment in newequipment.


It is not clear how this relatively price-inde-pendent adoption of medical technologies is usedby medical care providers to compete with oneanother. As summarized by Banta, et al. (19):

Studies of hospitals found no definite relation-ship between measures of competition and adop-tion. The situation is complex, because the char-acteristics of the market may relate not only tocompetitiveness, but also to the availability andsharing of information and to local standards ofpractice. The evidence conflicts, depending on thecharacteristic used and the technology studied.Russell (331) found that concentration of marketpower among a few large hospitals did not ap-pear to influence the adoption of three commonand two prestige technologies, but that hospitalsin more concentrated markets were less likely toadopt open-heart surgery. Prior adoption in alocality reportedly speeded the adoption of inten-sive care units and electroencephalographs, butnot diagnostic radioisotopes, open-heart surgery,renal dialysis, cobalt therapy, and computers(75,331). In urban areas, greater adoption ofradioisotopes and electronic data processing oc-curred where there were many hospitals percapita, the hospitals were of similar size, and theywere close to other hospitals (212,301).

Different patterns have also been observed be-tween adoption and the number of physicians percapita. Facing a low physician-population ratio,hospitals may compete for physicians throughtechnology adoption. On the other hand, fewerphysicians may exert less pressure for adoption.The adoption of CT scanners and radioisotopesappeared unrelated to the physician-populationratio (301,392). However, greater adoption of in-tensive care units, open-heart surgery, cobalttherapy, and renal dialysis occurred among Stateswith higher ratios (75).

Thus, even though current payment mecha-nisms for medical care services can lead to ex-cessive adoption of medical technologies, thereare still constraining factors which make it clearthat cost is not the only factor which influencesadoption.

Coordination Efforts and Disseminationof Information

Federal Activities

The Technology Coordinating Committee ofDHHS served as an interagency forum for the

identification and discussion of problems andissues associated with health care technologies(110,111,112). This committee, previously chairedby the Director of NCHCT, fostered informationexchange and interagency cooperation on healthcare technology matters and has served as thedepartment’s principal mechanism for joint actionon appropriate issues. Now that NCHCT is nolonger funded, DHHS is studying whether to keepthe Technology Coordinating Committee and, ifso, how to organize it.

NCHSR has responsibility for coordinatinghealth services research within agencies of PHS.To coordinate this research, NCHSR chairs thePHS Health Services Research Coordinating Com-mittee, which includes representatives from eachof the PHS agencies. * NCHSR also meets regular-ly with HCFA to review research priorities andto determine how each organization’s research ac-tivities might contribute to the other’s programs.In fiscal years 1980 and 1981, NCHSR and HCFAproduced a joint health services research strategyand budget (113).

NCHSR also disseminates research results torelevant Government agencies, the research com-munity, and other interested parties by means ofpublications, press releases, conferences, andworkshops. In 1978, the legislation authorizingNCHSR was modified to require that at least $1million or 5 percent of its budget, whichever isless, be used for dissemination activities. Inresponse, NCHSR established a User Liaison Pro-gram, aimed at providing substantive assistanceto non-Federal health care leaders concerned withcritical policy issues and operational problems inthe organization, administration, regulation, anddelivery of health care services at the State andlocal level. In 1979, NCHSR’s User Liaison Pro-gram conducted nine workshops that were at-tended by users of health services research suchas State legislators, executives of State healthagencies, leaders of both the insurance industryand hospital sector, and city health officials (113).

● NCHSR’S Health Care Technology Study Section, which servedas the scientific peer review committee in the grants review processfor NCHSR and NCHCT, provided an additional formal coordina-tion link.


The Office for Medical Applications of Re-search (OMAR), established in the NIH Director’sOffice in 1978, monitors, facilitates, and evaluatesNIH research, technology assessment, and tech-nology transfer activities. As noted in chapter 5,OMAR coordinates NIH consensus developmentconferences. The OMAR Advisory Committee—consisting of representatives from the bureaus, in-stitutes, and divisions of NIH and from otherFederal agencies, including FDA, the Centers forDisease Control (CDC), and the NCHCT (whileit was funded) —assists OMAR in its planned con-sensus development activities. The committee alsoassists OMAR in the exchange of information re-lating to other NIH involvement in assessing bio-medical technologies (228).

The consensus conference panels of NIH arecomposed chiefly of medical experts, althoughthey also include members of the lay public andselected professions (e.g., clerical and legal). Thetechnologies these panels examine may be emerg-ing technologies or technologies in general use andmay be drugs, devices, or medical, surgical, ordental procedures. (As described in ch.5 , therehave been over 30 consensus conferences since thefirst on breast-cancer screening in September 1977.The topics and dates of all conferences from 1977through the end of 1982 were listed in table 1.)On topics representing areas of mutual interestand concern, consensus conferences have beensponsored by NIH in conjunction with NCHCT,in collaboration with an agency outside NIH, orunder the cosponsorship of two or more instituteswithin NIH.

An essential part of the OMAR consensus de-velopment program is the dissemination of con-sensus statements and supporting materials topracticing physicians and others in the health caresystem, the biomedical research community, andthe public. It is hoped that by supplying medicalpractitioners with critiques of complex medicaltechnologies, dissemination of these reports willcontribute to an improvement in the quality ofmedical practice. OMAR has compiled a mailinglist of over 21,000 names. Consensus materialsand information have been published in threemajor American medical journals (Journal of theAmerican Medical Association, New EnglandJournal of Medicine, and Annals of Internal

Medicine), as well as in State medical society

periodicals and the general press (229,287).

NCHCT initiated the proposed development ofa Clinical Data Acquisition Plan, a conjunctiveeffort with both public and private group supportto develop a model method for collecting clinicaldata on emerging technologies. Under the modelprocess, third-party payers would provide interimreimbursement for appropriate technologies andrelated services to those providers who agreed tosubmit certain prescribed data (110,372).

As called for by its 1978 authorizing legislation,NCHCT also compiled an annual “emerging tech-nology list. ” All DHHS and other relevant agen-cies submitted a list of candidate technologies, in-cluding background information and a prelimi-nary assessment of each. Although the “emerg-ing technology list” was intended only to iden-tify emerging technologies, not necessarily toassess them, it came under increasing attack byindustry as a threat to innovation. The mere ap-pearance of a technology on the list, it wasargued, increased managerial reluctance to devel-op the technology because it created additionaluncertainty in further marketability (111).

Other Processes

Apart from those mechanisms involving Federalagency interaction, various other mechanisms bywhich medical technology information is dis-tributed include: 1) the public media, 2) the mail,3) advertising, 4) personal contacts, 5) the educa-tional process, and 6) libraries and other types ofinformation providers, including Federal informa-tion centers and private for-profit and not-for-profit organizations. While the relative impor-tance of each is arguable, the appropriateness ofindividual mechanisms may partly depend onwhether the information is to be used in conduct-ing an assessment of a medical technology or isto be used in conveying the results of a medicaltechnology assessment.

The popular print media, including daily andweekly newspapers and journals, are a primary

channel of information on health, includingmedical technology assessment, for the generalpublic. At times, they also serve a similar func-tion for physicians, nonphysician health profes-

86 ● strategies for Medical Technology Assessment

sionals, legislators, and others in the health field.Joining the mass circulation publications are anincreasing number of biweeklies and minimaga-zines serving special interests, regions, and evenlocalities (37o). Many of the mass circulation pub-lications employ a trained science/health colum-nist at regular or occasional intervals. In addition,news reports of immediate happenings in health,health advice columns, retrospective analyses oftechnologies, and more often, predictions aboutthe future of new technologies are found in allforms of print. Indeed, some popular publicationsare devoted exclusively to health and/or specificaspects of health.

The diversity in the print media is paralleledin radio and television. Some networks and/orstations, especially publicly owned or operatedoutlets, occasionally explore a medical technol-ogy in depth. One can question whether the5-minute-or-so news programs provide health in-formation of any real value, but such programsusually carry spot announcements about healthfairs and the need for their listeners to take ad-vantage of technologies, such as immunizationsand high blood pressure medications. Health fairsalso supply information about medical technol-ogies, as well as how and where professionalassistance can be obtained.

Mailings are a common mechanism fordisseminating both unsolicited and solicited in-formation for and about medical technology as-sessment. Unsolicited information is that whichis received without having been requested or paidfor by membership or subscription. Among thematerials available by this mechanism are directmailings about medical technologies (e.g., news-letters from drug companies) and advertisementsfrom product distributors. Unsolicited informa-

CONCLUSION

This chapter has discussed the innovation anddiffusion of medical technologies, especially med-ical and surgical procedures, and policies that af-fect the innovation and diffusion process. Manyof the points raised in this chapter are discussedat greater length in appendix D,

tion through the mails is an important source ofhealth information for both lay people and healthprofessionals. Most Federal agencies dealing withmedical technology use the mails for sending lit-erature in response to direct requests or to peo-ple on their mailing lists.

Advertising of drugs, and to a much lesser ex-tent of other medical technologies, is prominentin all the popular media. The large budgets thatmost pharmaceutical companies allocate for thispurpose seem presumptive evidence that there isa market for this source of information amongthe general public. Advertising is termed educa-tion by the companies involved, especially whenthe target audience is physicians and other healthprofessionals. Pharmaceutical manufacturersspend several hundred million dollars a year inadvertising their products to the medical profes-sion in professional journals, at professionalmeetings, and through their representatives (“de-tail men”). These expenditures would not be likelyto continue if they did not bear results (65).

Recently, some drug companies have been sup-plying hospitals and other medical facilities withvideo cassettes that contain information on var-ious aspects of health care including medical tech-nologies. They also are producing closed circuittelevision programs on similar topics to be re-ceived at medical facilities that have video tapereceivers (214).

For a discussion of the ways in which physi-cians keep informed about, and are influenced by,new developments of medical technologies, thereader is also referred to an earlier section of thischapter concerning the diffusion of medical tech-nologies.

The innovation process is complex and not wellunderstood, but is certainly important to anassessment strategy. Most regulation is intendedto substitute for an imperfect market. Govern-ment has adopted a general sense of public respon-sibility by seeking to ensure that unsafe and inef-

—.—


ficacious drugs and medical devices not be al-lowed on the market. The thrust of nearly all FDAregulations is to require manufacturers of newdrugs and certain medical devices to test theirproducts for safety and efficacy according to ap-proved protocols. FDA then synthesizes this in-formation, decides whether to approve the mar-keting of the technology, and then regulates thelabeling process. Thus, FDA is involved with allstages of medical technology assessment as definedin chapter 1 (i.e., identification, testing, synthesis,and dissemination). However, FDA’s involvementis limited to certain types of technologies-emerg-ing and new drugs and devices. Also, FDA’s ac-tivities are generally limited to assessments of safe-ty and efficacy; cost, cost effectiveness, and othersocial/ethical effects are generally not explicitlyconsidered. *

The information requirements of FDA tend toslow the innovation and diffusion of certain med-ical technologies. Industry, especially the medicaldevices industry, is concerned that FDA’s infor-mation requirements unnecessarily threaten in-novation. A major problem in analyzing indus-try’s concerns, however, is the difficulty of deter-mining the costs and benefits of testing require-ments.

Reimbursement policies also distort the innova-tion process. In general, it appears that the wideavailability of medical insurance contributes tothe overadoption and use of many medical tech-nologies. In many cases, the lack of technologyassessment information at the point of reimburse-ment tends to speed up the diffusion process. Assuggested earlier, however, the diffusion of trulyinnovative technologies that fall outside general-ly accepted medical practice may actually be dis-couraged by the present reimbursement system.

This dichotomy seems to be related to the lackof adequate scientific evidence of the value of newtechnologies. When a new technology is an add-on (i. e., when it does not directly substitute for

● Sometimes, however, its review extends farther. For example,in the case of the injectable contraceptive Depo Provera, FDA basedits decision to deny market approval, not only on safety criteria(i.e., its concerns over Depo Provera’s cancer-causing potential),but partly on the basis that the patient population originally targetedfor Depo Provera had diminished substantially as other methodsof contraception and sterilization became increasingly available andaccepted (193).

an existing technology), produces more informa-tion, or for some reason captures the imaginationof the medical profession, the technology tendsto be accepted and even encouraged by the med-ical profession without substantial evidence of itsvalue. On the other hand, radically new technol-ogies that challenge preexisting beliefs—or in somecases merely the status quo—are less likely to beacceptable to the medical profession without verystrong evidence of their worth. Since present reim-bursement policy rests in large part on acceptedmedical practice, these more radical technologiestend not to be acceptable for reimbursement, andtheir innovation may thus be discouraged.

The effects of regulation and reimbursementpolicies on the innovation process are clearly in-terrelated. As new medical procedures develop,they often make use of new drugs and devices oruse existing ones in modified ways. Such drugsand devices generally have to pass through FDA’sregulatory process. Until these technologies areapproved for marketing, regulatory review actsas a constraint on the adoption and dissemina-tion of the procedures in which they are used.Once these accessories are released into the mar-ketplace, however, they can act to stimulate useof procedures which are still experimental and notaccepted medical practice. For example, FDA re-leased the catheter used in percutaneous trans-luminal coronary angioplasty (PTCA) from in-vestigational device status and approved itsmarketing for PTCA while the procedure itselfwas still considered by many to be experimental.

The Federal agencies responsible for medicalresearch, regulation, and financing engage in avariety of technology assessment-like activities.Although there are increasing efforts to improvecoordination between these agencies and col-laboration with the private sector, these effortscurrently fall short of a strategy for medical tech-nology assessment as discussed in chapter 1. Cur-rent coordinating efforts are heavily orientedtoward the question of reimbursement. One of theformer NCHCT’s formal responsibilities was toadvise HCFA on coverage decisions. Current ef-forts such as NIH consensus development con-ferences are oriented toward determining the ef-ficacy, safety, and appropriate use of medical andsurgical procedures, but they can also help to pro-


vide information for HCFA’s decisions regarding sessment data, there is little coordination of itsthe reimbursement of new technologies. A major functions with those of the other governmentalweakness of all these activities is that no body is health agencies.charged with evaluating the economic and social/ The next chapter presents OTA’s critique of theethical effects of medical technologies. In contrast,the regulatory agencies, principally FDA, have

current system for the identification and testing

limited roles in current coordinating efforts.of medical technologies and the synthesis and dis-

Although FDA’s regulatory responsibilities makesemination of technology assessment information.

FDA an important generator and repository of as-

7Critique of the Current System

It is one thing to show a man that he is in an error, and another to puthim in possession of truth.

—John Locke

Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..........+’.. “.’”””Assessment of Medical Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Critique of the Current system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...”...Identification of Emerging Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Identification of New Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Identification of Existing Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Identification of New Applications of Existing Technologies . . . . . . . . . . . . . . . . . . . .

Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . “..”Dissemination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Conclus ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 . . . . .+ $

P’age919293949596979898

100101102

—-

7Critique of the Current System

INTRODUCTION

It has been well established that there is insuffi-cient information regarding the costs, risks, andbenefits of medical technologies. One purpose ofassessing medical technologies is to produce infor-mation to help guide their appropriate use. Clear-ly, society wishes to promote the developmentand diffusion of safe and effective medical tech-nologies. At the same time, society wishes to re-duce ineffective and inefficient use of medical tech-nologies. Finding a balance between these twogoals is difficult. The complexity of our societyand the mixed private/public nature of the healthcare system magnify the challenge of improvingexisting policies and processes for medical tech-nology assessment.

Federal policies toward medical technologyhave developed in incremental fashion to meetrather specific goals. Only recently have the nu-merous policies that pertain to the development,diffusion, and use of medical technologies begunto be seen as elements of an overall system forguiding and promoting technological change, Thecollection of programs and activities that formsthe current “system” for assessing medical technol-ogies’ has also been built-up in piecemeal fashion.Although many have realized that better informa-tion on the benefits, risks, costs, and social impli-cations of medical technology is essential to guid-ing the development and use of technology with-out unnecessarily impeding innovation, progresstoward developing a coherent system for assess-ing medical technologies has been slow.

The idea of a “strategy” for assessing medicaltechnology is closely related to having a system.A strategy is in effect the underlying basis for thedesign and implementation of any coherent sys-tem for assessment. Any strategy will represent

● By the existing “system” of assessing medical technologies, OTAmeans the current set of programs and activities which are relatedto assessing medical technologies. In this context, the word “system”is used in the general sense and is not intended to imply that therecurrently is some overall design involved.

a compromise among competing perspectives onthe goals of medical technology and on the roleand format of assessments. Strategies reflect de-sired policy directions and ideas on how best tomove in those directions. Without a conceptual-ly clear strategy, any system of assessment willsuffer from inconsistencies and unclear objectives.

The basic objective of a strategy for medicaltechnology assessment is to ensure that technol-ogies of public policy importance are evaluatedby appropriate methods in a timely fashion with-out unnecessarily harming innovation. This ob-jective must be sought despite a number of formid-able difficulties. One problem is the very limitedamount of money available for original evalua-tions of medical technology. Costs of a single ran-domized clinical trial (RCT) can run as high ashundreds of thousand of dollars, yet the entiresocietal investment in original studies of medicaltechnology probably does not exceed $2OO millionat any one time.

A sound strategy for assessment must take intoaccount the stage of development of particularmedical technologies-emerging, new, or existing.If emerging technologies are assessed too early intheir development, innovation may be slowed;furthermore, the information gained by assessingthe technologies may be valueless, because knowl-edge about their modifications and eventual useswill be limited. If new or existing technologies areassessed too late, the assessments will have littleeffect on the technologies’ diffusion or use. Assess-ment information must be disseminated to appro-priate parties in a timely manner.

A strategy for medical technology assessmentmust also deal with the universe of drugs, devices,and medical and surgical procedures—diagnostic,therapeutic, and preventive. Other importantclasses of medical technology to be dealt with aremedical care delivery systems and organizational

91


innovations, although, as noted earlier, they fallbeyond the scope of this assessment.

A further objective of a strategy is to developcriteria to choose an appropriate method or meth-ods for assessment. A strategy must permit oneto determine when it is enough to know that atechnology is efficacious, when it is desirable tohave a formal cost-effectiveness study, and whena full-scale technology assessment with evalua-tion of social implications would be helpful anddesirable.

A strategy should also address how the infor-mation gained from technology assessment is uti-lized and by whom. And finally, it is necessaryto consider how technology assessment should af-fect health policy.

These are some of the challenges. This reportcannot deal with all of them. The-section that-fol-lows discusses the development of information onwhich to base decisions. The chapter ends witha critique of the current technology assessmentsystem.

ASSESSMENT OF MEDICAL TECHNOLOGIES

Assessing medical technologies is a complexprocess, and no simple model can be devised tooutline the steps that must be taken in all circum-stances. Technologies are diverse, often lendingthemselves to be evaluated in diverse ways. Theneed for information about technologies by differ-ent people at different points in time varies, aswell. Nevertheless, there are a finite number oftechnologies and a finite number of assessmentmethods. The information that is required tomake rational and reasonable informed choicesis also finite.

The previous chapters of this report have identi-fied the needs for assessment information and theresources available to fill those needs. The needsare defined by those called on to make decisions.For example, the Food and Drug Administration(FDA) must decide whether to allow a drug ordevice to be marketed; it asks whether the technol-ogy works and whether it is sufficiently safe. TheNational Institutes of Health (NIH), in its basicresearch efforts, must set priorities regardingwhich technologies, including especially medicaland surgical procedures, it will further investigateand develop. Having been called on by Congressto synthesize what is known about medical tech-nologies in order to assist in their transfer, NIHalso asks what the appropriate conditions andstandards of use are. Similarly, ProfessionalStandards Review Organizations (PSROs) askwhether local practice patterns conform to reason-able standards of care. Those who pay for care,whether they be the Health Care Financing Ad-

ministration (HCFA), Blue Cross, or individualpatients, need to know whether the use of a med-ical technology is worth the cost. And, finally,the practicing clinicians, in consultation with in-dividual patients, must make the final decision touse a technology. Throughout this process, thevalues and needs of society, the medical profes-sion, and the patients themselves are interwoven.

A strategy for medical technology assessmentmust take into account what information isknown, what is not known, what is needed, whatcan be obtained, and what the cost of obtainingit will be. Information will never be perfect, andmoney and time will always be limited. Thus, itis important to make judicious use of evaluationmethods. Fortunately, there are means to compen-sate for uncertainty when important informationis lacking.

No clear-cut rules seem to be possible in devis-ing a strategy for choosing a method to assess thesafety, efficacy, and effectiveness of medical tech-nologies. Often, the method of choice will be re-lated to both the stage of diffusion of the technol-ogy and the extent of knowledge and belief as toits risks and benefits.

RCTs, for example, tend to be appropriatewhen a technology’s risks and benefits are not wellunderstood, when the technology is not yet in gen-eral use, and/or when costs of the technology arevery high in relation to expected benefits, andwhen risks are expected to be low. Under theseconditions, the purpose of an RCT is to establish

Ch. 7—Critique of the Current System ● 93

a cause-and-effect relationship. Reasonable can-didates for RCTs would thus be new drugs, newinvasive devices, new expensive equipment, andnew elaborate services requiring capital expendi-tures (e.g., neonatal intensive care units). Whenrisks and benefits are not well known or are notbelieved, randomization can be used without vio-lating some of the ethical principles noted in chap-ter 3. If RCTs are used early in a technology’s dif-fusion, nonrandomized designs, especially case-control studies, can later be used to establish theeffectiveness of the technology as it diffuses intodiverse settings.

However, when a technology is in widespreaduse, risks and benefits are either already knownor are widely believed to exist, and randomiza-tion may be neither possible nor appropriate. Inthis case, nonrandomized designs” can be used toestablish relationships which can later be tested,if desired, by more rigorous methods, includingrandomization.

Economic analyses are similarly varied, and noone technique is applicable in all cases. However,economic information may be worthless withoutgood safety and effectiveness information. For theuser of the technology, the price is the cost. That

price must somehow be compared to the perceivedvalue of the use of technology. But for more gen-eral decisions, especially at the societal level, eco-nomic analyses are very complex, requiring bothtechnical expertise and good judgment. A cost inone instance may be ignored or even counted asa benefit in another.

As discussed in chapter 4, decisions concern-ing the development and use of certain medicaltechnologies often have profound social and ethi-cal implications. Especially at the Federal policy-making level, these implications are important toconsider, even though they cannot be precisely

quantified.

Finally, informed decisions rest on the analysisof all available information. Chapter 5 discusseda number of techniques that can be used to syn-thesize information from research studies in a sys-tematic manner. Additionally, group process tech-niques such as Delphi and nominal group proc-ess are available to assist policymakers and tech-nically expert groups in making decisions. Noneof these methods for synthesizing information isperfect, but each has potential value in the devel-opment of more orderly processes for setting pol-icies regarding medical technologies.

CRITIQUE OF THE CURRENT SYSTEM

The present system of medical technology as-sessment, like the medical delivery system, is plu-ralistic, and many of the public and private sec-tor activities reviewed in this report were under-taken for purposes other than medical technol-ogy assessment. The diversity of activities is notnecessarily a weakness. Such diversity capitalizeson the wealth of ideas and interest of many differ-ent people and organizations. Nevertheless, itmakes the job of fashioning a more coherent sys-tem of assessment more difficult.

Perhaps the principal reason for the difficultieswith the present system is that the main parts weredeveloped separately over a long period of timewith specific, sometimes inconsistent, goals. Theexisting programs and activities were not devisedas elements of an overall system of technologyassessment. In the case of the Food, Drug, andCosmetic Act, amendments over the past three

quarters of a century have been internally consist-ent, tending to build on and complement previouslegislation. However, most other legislative andnongovernmental efforts that affect medical tech-nologies have not been so well coordinated. Thus,the country has a system of physicians, hospitals,planning agencies, PSROs, health survey activi-ties, research activities, and insurance claims net-works, all of which use, dispense, regulate, evalu-ate, collect information on, or otherwise affectmedical technologies, but which often do not com-plement one another’s needs for technologyassessment. *

*This criticism does not necessarily apply to the Veterans Admin-istration’s system, which OTA did not study to any appreciabledegree. Nevertheless, it is known that the Veterans Administrationis developing a system whereby potential investigators are informedof program needs, a research agenda is developed to satisfy thoseneeds, and useful information generated by research is made avail-able to those who need it (168).

94 • Strategies for Medical Technology Assessment

To examine the extent to which the needs ofan overall system for medical technology assess-ment are met, the programs and activities com-prising the present system are discussed in the re-mainder of this chapter with reference to the fourphases of the technology assessment process men-tioned earlier: 1) the identification of technologiesneeding assessment, 2) the testing of technologiesto develop information concerning their healthand economic effects, 3) the synthesis of informa-tion, and 4) the dissemination of the informationthat is available.

OTA finds that the current system for evaluat-ing medical technologies exhibits major deficien-cies in each of the four phases of the assessmentprocess. For technologies at different stages ofdevelopment (i.e., emerging technologies, newtechnologies, existing technologies, and new ap-plications of existing technologies), as well as fortechnologies classified as either drugs, devices, orsurgical or medical procedures, the adequacy ofthe present system differs.

The existing system for identifying technologiesto be assessed, except for FDA’s system of identi-fying new drugs and devices, is unnecessarilypoor. (Among the many reasons for this is theinadequacy of the synthesis of research informa-tion. ) In the testing of medical technologies, manystudies generate evaluative information, but thequality of such information varies widely. FDA’sresearch requirements for new drugs and devicesseem adequate for the premarket approval proc-ess, and much NIH-sponsored research has re-sulted in significant information for society. Inother areas, however, high-quality studies arefew, and most of them are not helpful in settingpolicy. High-quality, objective syntheses of re-search findings-a prerequisite for developing pol-icy or setting medical practice standards—arerare. Many syntheses are informal, overly subjec-tive, group-generated norms and are not basedon a rigorous assessment of the scientific evidence.Although there are increasing efforts to dissemi-nate technology assessment information, much ofthe information has questionable value. The ex-cessive adoption, diffusion, and use of some med-ical technologies indicate a need for improveddissemination efforts.

In the expanded critique that follows, specialattention is paid to the identification of medicaltechnologies to be assessed, since OTA finds thatthis is the critical phase of any overall assessmentstrategy.

Identification

Any system for medical technology assessmentmust have mechanisms to identify technologiesto be assessed and to set priorities among candi-dates for assessment. Clearly, no single mecha-nism is appropriate for all occasions and all tech-nologies. What works for drugs may not be suit-able for surgical procedures, and what is appropri-ate for identifying emerging technologies may notbe adequate for established ones.

Methods of identifying technologies for assess-ment can be thought of as falling into one of threegeneric categories: 1) routine mechanisms, 2) pri-ority-setting mechanisms, and 3) mechanisms ofopportunity. Routine mechanisms systematicallyidentify a class of technologies and are usuallyconnected with a particular event with which alltechnologies in the class are associated. (Examplesare FDA’s requirement that all drugs and devicesbe registered with it prior to marketing or testingin human beings and, if taken advantage of,HCFA’s reimbursement coverage determinations.)Priority-setting mechanisms are not routine andare often mechanisms or processes used by somegroup to determine priority technologies forassessment based on some implicit or explicit cri-teria. (Examples are the processes the institutesof NIH and HCFA use to establish their researchagendas, the processes the Office for Medical Ap-plications of Research (OMAR) of NIH uses toset priorities for consensus development confer-ences, and the process the National Center forHealth Care Technology (NCHCT) formerly usedto establish priorities for technology assessments. )Mechanisms of opportunity are means for identi-fying technologies for assessment as opportunitieshappen to occur. These are less well defined thanmechanisms in the previous two categories, butare not necessarily less important, because tech-nologies that suddenly become important to assessoften do so for safety or ethical reasons. (Examples

Ch. 7—Critique of the Current System ● 9 5

of mechanisms of opportunity are FDA’s sponta-neous drug reporting system, PSROs’ medical careevaluation studies, and the systematic generationof patient outcome data which researchers cananalyze or the public can challenge. )

The specific purpose of identifying technologiesfor assessment will vary according to the stageof development of the technology in question.Emerging technologies, especially drugs and cer-tain devices, need to be assessed for safety andefficacy. In the case of drugs and certain devices,assessment in the emerging stage is necessary be-cause of the well-accepted social responsibility,as expressed in FDA law, to protect the public.There may be other social and ethical reasons toassess emerging technologies as well. New and ex-isting technologies need to be assessed for safetyand effectiveness, sometimes for cost effectiveness,and at times for social and ethical consequences.Physicians and their patients need to know whatworks and what the benefit/risk balance is. Pa-tients and insurers, including HCFA, need to un-derstand the economic implications of technolo-gies, especially when there are alternatives. Infor-mation is also needed by PSROs. PSROs arecharged with assessing whether HCFA funds arebeing used on “needed” services, and these includeboth new and existing technologies. Additionalinformation is needed by health planning agen-cies, which are charged with determining whethermajor new technologies should be purchased byhospitals.

A critique of the current system’s record in iden-tifying medical technologies for assessment yieldsmixed results. For the purposes of the discussionbelow, it is helpful to think of the identificationof technologies at the four typical stages of devel-opment defined in chapter 1: 1) emerging technol-ogies, 2) new technologies, 3) existing technolo-gies, and 4) new applications of existing technol-ogies.

Identification of Emerging Technologies

Certainly, the most thorough system for identi-fying emerging medical technologies is FDA’s pre-marketing approval process for drugs and ClassIII medical devices (see ch, 6). This process isclearly a routine identification mechanism, as de-fined above.

The only other notable systems for identifyingemerging medical technologies are priority-settingmechanisms. These include the processes for es-tablishing the research agendas of NIH, the Na-tional Center for Health Services Research, andwhile it was funded, NCHCT. Although each in-stitute and research agency has its own internalsystems, the process of establishing priorities forintramural research and extramural contracts isessentially determined by institute or agency staff.Research priorities tend to be set by informed pro-fessional staff who know their particular field welland thus know which questions are important toaddress. Grants can be either solicited or unsolic-ited. In either case, the projects are generally se-lected on the basis of technical merit and judg-ments about their importance. The research agen-da priority-setting processes of NIH and otherFederal research agencies generate information fora base of knowledge which can lead to unpredicta-ble but substantial future dividends that may bedifficult or impossible to measure. Often, how-ever, the processes do not address the immediatepolicy priorities of operating agencies such asHCFA and other social priorities such as Congressmay have for the health care system.

Two other priority-setting mechanisms for iden-tifying emerging technologies for assessment arealso deserving of mention. HCFA’s research arm,the Office of Research and Demonstrations, ischarged with assessing technologies of interest toits operations. Some of these technologies maybe classified as emerging, although most probablywould not be. Seldom, however, are the technol-ogies clinically related (e.g., some are concernedwith information systems). The other mechanismwas the NCHCT’s “emerging technology list. ”This was a systematic and broad-based approachfor identifying health-related technologies underdevelopment which were expected to be used inthe practice of clinical medicine within 5 years.However, critics from industry charged that thecompilation of such a list threatened the innova-tion process, and the 1981 reauthorization ofNCHCT specifically withdrew the center’s author-ity to compile this list.

The third type of identification mechanism, tak-ing advantage of unforeseen “opportunities,” isgenerally not relevant for emerging technologies,


except in rare special cases such as the artificialheart.

Discussion: FDA’s routine identification of allemerging drugs and emerging Class III devicesseems adequate and appropriate. Emerging medi-cal and surgical procedures do not seem to lendthemselves to being identified through routinemechanisms. The most appropriate identificationmethod for emerging procedures would seem tobe the subjective priority-setting mechanisms suchas those being used by the institutes of NIH andother Federal research agencies.

NCHCT’s “emerging technology list” had theadvantage of cutting across categorically relatedprograms and also forced each program of the De-partment of Health and Human Services to explic-itly identify technologies which were emergingand of importance. The 1980 format for submis-sion of an entry included the name and identifica-tion of the technology, a technical description, astatement of importance or potential impact, anevaluation of the technology’s present status anddata needs, and any special considerations. NIHstaff have commented that the exercise of compil-ing such a list was useful in taking stock of whatwas happening in their respective fields. If one ob-jective of the technology assessment system is tomore actively manage technologies, compilingsuch a list would seem to be quite useful in thatit allows one to make predictions and to plan forthe future. The charge of industry that the list in-hibited innovation is not supported by any datathat OTA could find, but the issue could be fur-ther examined.

OTA concludes that emerging drugs and de-vices are adequately and appropriately identified,but that emerging medical and surgical procedurescould be better identified. Overall, the identifica-tion of emerging technologies is not a criticalweakness of the present system.

Identification of New Technologies

As a group, new technologies, those in theadoption phase, are the most easily identified. Inparticular, such technologies are the most obviouscandidates for identification through routinemechanisms. Most new medical devices (i.e.,Class I and 11 devices) are routinely identified as

required by FDA law (see ch. 6). * New medicaland surgical procedures, including the use of newdrugs and/or devices, are potentially identifiableroutinely through the reimbursement process,since the question of coverage should arise whena new procedure is identified.

All new medical technologies are also poten-tial candidates for being identified through the pri-ority-setting mechanisms discussed in the preced-ing section on emerging technologies. And, in fact,new technologies are identified for assessmentthrough the priority-setting processes of the insti-tutes of NIH and the other research agencies ofthe Public Health Service (PHS) and HCFA.

All new technologies are also logical candidatesfor being identified through mechanisms of oppor-tunity. The recent maternal serum alpha-fetopro-tein (MSAFP) controversy illustrates the use ofsuch a mechanism (see app. E). FDA was on theverge of approving widespread use of the MSAFPscreening test when a special interest consumergroup (the parents of children with spina bifida)questioned the validity of FDA’s data. A majorassessment of MSAFP was subsequently carriedout, and new regulations were issued. This caseillustrates a mechanism of opportunity (i.e., pub-lishing data and making decisions under publicscrutiny).

Discussion: As a class, new technologies are themost easily identified as candidates for assess-ment, especially by routine mechanisms. In thecase of new Class I and Class II medical devices,FDA’s routine identification process seems ade-quate and appropriate. In the case of new medicaland surgical procedures, however, there is cur-rently no systematic mechanism for identification.To some extent, new procedures are identifiedthrough the reimbursement system; however, incontrast to the structured identification processof FDA, identification through reimbursementdecisions of HCFA and other public and privateinsurers is much more haphazard. While there isconsiderable potential for the reimbursement sys-tem to be used routinely as a primary means ofidentifying new procedures for assessment, prob-

‘More invasive devices (i.e., Class 111 devices) are identified inthe “emerging” phase, because they must receive FDA’s approvalbefore being tested in clinical trials.

Ch. 7—Critique of the Current System ● 9 7

lems persist. Even the process of identifying whichprocedures are new seems to be unsatisfactory:terminologies and codes on claims forms are oftennot accurately labeled or are not standardized;new procedures often do not have a procedurecode number. However, to the extent that thereis increased scrutiny by third-party payers of billssubmitted for new procedures and more than oc-casional denial of payment for such bills, the pro-vider has a strong incentive to request paymentfor an already existing standard procedure, ratherthan a new one, thus complicating the identifica-tion process.

The process of identifying new technologiesthrough the priority-setting processes of the insti-tutes of NIH and agencies of PHS has essentiallythe same strengths and weaknesses as were dis-cussed in the connection with the identificationof emerging technologies. From an academic pointof view, the system seems appropriate. The weak-ness stems from the lack of an adequate systemto identify priority candidates for the operatingagencies, especially HCFA, PSROs, and planningagencies. Theoretically, HCFA has its own re-search arm, the Office of Research and Demon-strations, to accomplish this. As stated previously,however, that office has not been very involvedto date with either identifying technologies forassessment or assessing them.

Whether the mechanisms of opportunity foridentifying new technologies are adequate is dif-ficult to assess. Since standardized, high-qualitydata on technology use and health outcome arenot generally available, it is likely that they arenot.

OTA concludes that new drugs and devices areadequately identified for assessment, but that newmedical and surgical procedures are not. The mostpressing need is for a routine mechanism toidentify new procedures before they are wide-ly adopted. The reimbursement system, becausecoverage and payment decisions are critical pointsin the diffusion of many technologies, might begiven primary consideration. In addition, the pri-ority-setting systems of the institutes of NIH andof other Federal research agencies (e.g., NCHSR)are adequate and appropriate for their respectivemandates, but there is not an adequate similar

system to fulfill the needs of operating agencies(e.g., HCFA, pIanning agencies). Finally, suffi-cient mechanisms of opportunity for identifyingnew technologies do not exist but could be devel-oped. Medical specialty societies could be helpfulin this area.

Identification of Existing Technologies

As a group, existing medical technologies tendto be the least likely candidates for routine identi-fication, primarily because there is no natural trig-gering mechanism such as introduction. Conse-quently, the timely identification of existing tech-nologies must depend largely on priority-setting

procedures or mechanisms of opportunity.

Theoretically, if emerging and new technologieshad been adequately identified (and assessed) asthey developed, there would be less need to identi-fy (and assess) them after their adoption and gen-eral diffusion. But, as indicated in previous OTAreports (e.g., 266,270,279), most existing medicaltechnologies have not been adequately assessed.At a minimum, existing medical technologiesshould be monitored for risks that may not havebeen previously apparent. A review of the activityin this area reveals a very poor record, with a fewexceptions and a few encouraging signs.

One encouraging sign is the interest in post-marketing surveillance systems for drugs. * Post-marketing surveillance systems are noteworthy,because there is increasing concern that FDA’s pre-market approval process is not sufficient to pro-tect the public after a drug or device is marketedand in use (281). Although often regarded as test-ing techniques, postmarketing surveillance sys-tems can also be thought of as sophisticated sys-tems for identifying technologies needing furtherinvestigation. Such systems represent a hybridiza-tion of a routine mechanism, a priority-setting

mechanism, and a mechanism of opportunity, Al-though data may be collected routinely underpostmarketing surveillance systems, not all drugswould automatically be screened. FDA can set itsown research agenda, and independent researchinvestigators, at their own initiative, can be ex-

‘This topic is considered at greater length in a separate OTA reportentitled Postmarketing Surveillance of Prescription Drugs (281).


pected to use the data to identify fertile areas forfuture study.

Another encouraging sign for the identificationof existing technologies for assessment are privatesector initiatives using the priority-setting method.These include Blue Cross/Blue Shield’s MedicalNecessity Project and the American College ofPhysicians’ new Clinical Efficacy Assessment Proj-ect (see ch. 6). In the Federal sector, discussed pre-viously, priority-setting processes (includingNCHCT’s and OMAR’s) are also used to identifyexisting technologies.

The most glaring omission in the system foridentification of existing technologies for assess-ment is the lack of identification by operatingagencies, especially HCFA and State Medicaidagencies. Even with its PSRO arm, HCFA doesnot have an adequate system to question technol-ogies that are already in widespread use.

One mechanism of opportunity that can beused to trigger identification of an existing tech-nology in need of assessment is the identificationof a competing technology. To some degree, thismechanism is used implicitly. For instance, com-puted tomography scanning was likely to havebeen compared with existing technologies such asskull X-rays. Whether such opportunities for iden-tification are always, or even frequently, takenadvantage of is not clear.

It was stated earlier that mechanisms of oppor-tunity are particularly useful for identifying tech-nologies that are currently in use. FDA has a spon-taneous reporting system for adverse drug reac-tions which illustrates how technology assessmentopportunities surface “spontaneously. ” Similarsystems could be used by other agencies such asHCFA. A functioning identification system of op-portunity requires a method by which a technol-ogy assessment issue can be reported and a meansto act on that information.

OTA concludes that the system for identifyingexisting technologies in need of assessment is in-adequate. One promising possibility is postmar-keting surveillance techniques. As was true withemerging and new technologies, the priority-setting procedures of Federal research agenciesmay be adequate for those agencies’ respective

needs, but not for the needs of operating agen-cies such as HCFA. And the operating agenciesthemselves do not adequately identify existingtechnologies for assessment. Medical specialtysocieties could be helpful in this area. Final-ly, NCHCT’s activities of identifying nationallyimportant priority technologies for assessmentwere valuable but are not now funded. Thus, nobody is currently undertaking this important task.

Identification of New Applicationsof Existing Technologies

The consideration of new applications ofexisting technologies is important for two rea-sons. First, a new application of a technology

means that previous information about the tech-nology may no longer be applicable; and second,a technology’s new use may provide an opportu-nity to identify it through a routine mechanism.At present, OTA is unaware of any systematicmethod of identifying new applications of existingtechnologies as candidates for assessment.

These technologies can be identified throughpriority-setting procedures and mechanisms of op-portunity, as can existing technologies. It wouldseem, though, that the most rewarding approachfor identifying new applications of existing tech-nologies would be through a routine mechanism,namely, the reimbursement system.

OTA concludes that new applications of exist-ing technologies are not adequately identified forassessment. To facilitate the identification of suchtechnologies, the most promising approach maybe the use of the reimbursement system to linkthe diagnosis with the use of the technology. Med-ical specialty societies could be helpful in this area.

Testing

Many of the deficiencies of the testing phaseof the current system for medical technology as-sessment are intimately related to the inadequaciesof the identification phase. In order to know whatto test for, one must have identified the appro-priate technology for assessment, the relevant pol-icy concern (e.g., safety, efficacy, or cost effec-tiveness), and the information which is lacking.Thus, an adequate testing phase requires an ade-

Ch 7—Critique of the Current System . 9 9--

quate identification phase. It is not surprising,therefore, that the strengths and weaknesses ofcurrent testing activities closely parallel those ofthe identification phase.

FDA adequately identifies emerging and newdrugs and devices that need assessment and alsodetermines what information it needs. Further-more, FDA carefully reviews the research proto-cols of the industries it regulates and requires thatthe protocols be used. Resulting testing by indus-try seems adequate. As suggested previously,however, FDA does not have an adequate meansto identify which drugs and devices need furthertesting once they are released into the market.Thus, FDA cannot develop protocols for furthertesting of products in new settings or under dif-ferent applications. As indicated in chapter 3, ade-quate protocols can be developed (see also ref.281).

Chapter 6 discussed the testing of medical andsurgical procedures through the funding activitiesof NIH. Since the individual institutes of NIH sub-ject all research protocols to an intensive peerreview process (270), the quality of the researchis generally good. Any problems with such activ-ities center around either the need for additionalfunding or the agenda-setting process* (the lat-ter is essentially an issue of identification). It isimportant to note that NIH does not have the mis-sion to ensure that all medical and surgical proce-dures are proven to be safe and effective. (Nordoes any other agency or organization. )

Currently, the overriding weakness of the test-ing phase is in the testing of new and existing med-ical and surgical procedures. Since procedurestend to be developed within the practice of medi-cine, they are generally adopted and accepted bythe medical community without a routine, for-mal examination of their merits. A good deal ofthe problem in this area stems from a lack of re-search funding. Another problem concerns thedevelopment and use of research methods, sinceRCTs are not appropriate for all clinical inquiries.

*During the current period of fiscal restraint, substantially in-creased Federal research budgets seem unlikely; this, it may beworthwhile to explore the possibility of joint private/public efforts.The theme is explored in ch. 8.

Data systems can be linked and then used toidentify technologies for assessment, and such sys-tems can also be used, though to a lesser extent,to evaluate safety, effectiveness, and cost effec-tiveness. Prospective studies could be initiated tolink technology use to health outcome and cost.One model which could be further examined isthe Clinical Data Acquisition Plan which wasbeing developed by NCHCT (see ch. 6). Data sys-tems may be adequate in some cases to providesufficient evidence of safety and effectiveness oftechnologies, especially if they are used to com-plement more rigorous testing methods.

FDA’s postmarketing surveillance activities fordrugs, mentioned earlier, are being developed tomonitor adverse reactions to drugs (281). Suchsystems may be adaptable for other technologiesas well.

The Federal Government has not used its poten-tial leverage to test technologies through the reim-bursement system. For instance, if HCFA coulduse its system to study whether new procedureswere safe, effective, and cost effective, or need-ed further testing before final reimbursement deci-sions were made, many ineffective technologiesmight be identified and discarded well before theywere accepted by the medical community. Theambiguous “reasonable and necessary” clause ofHCFA’s statutory language has been an obviousimpediment to such activity.

Although the private sector has been activelyinvolved in testing medical technologies, its directsupport for well-controlled clinical trials has notbeen very extensive (except for the research mon-itored by FDA). Research protocols tend to be ofa nonrandomized design and often rest on the in-formation derived from available data bases andrecordkeeping systems (e.g., 209). Nevertheless,there is evidence of increasing private sector inter-est in research on technologies. Much of the inter-est seems to stem from the belief and concern thatresources are not used efficiently.

Finally, it should be noted that currently nopublic or private body has responsibility for deter-mining either the cost effectiveness or social/ethi-cal implications of medical technologies. FDA andNIH are both primarily oriented towards safetyand efficacy issues. It is true that NCHSR and to


a lesser extent HCFA do selectively fund somecost-effectiveness studies, but no one body ischarged with systematically examining the largersocial issues.

OTA concludes that, in general, drugs anddevices are adequately tested for safety and effi-cacy prior to being marketed. Medical and surgi-cal procedures, however, are not well tested foreither safety or effectiveness. No class of technol-ogies is adequately evaluated for either cost-effec-tiveness or social and ethical implications. Final-ly, there is no organization whose mission it isto ensure that medical and surgical procedures areassessed for safety and efficacy or to evaluate anyclass of technologies for cost effectiveness and forsocial/ethical implications.

Synthesis

Synthesis activities in the area of medical tech-nology assessment are generally of two majortypes: 1) synthesis of the results of individual re-search studies; and 2) synthesis of a body of re-search findings with other concerns such as risk,social, ethical, or cost factors. The former, whichis more focused and technical than the latter, seeksto answer questions such as those concerning thesafety, efficacy, or effectiveness of a given tech-nology. The second, which is more policy di-rected, often seeks to develop guidelines or stand-ards for medical practice or reimbursement policy.The value of the latter depends, in large part, onthe adequacy of the former. That is, one cannotconsistently set good policy regarding medicaltechnologies without knowing what the collectiveresearch says about a given set of issues.

The challenge for synthesizing research evi-dence concerning a technology is to make senseout of a growing body of information-some bad,some good. Techniques available to do this weredescribed in chapter 5.

Synthesis activities are inherently a part of con-ferences sponsored by individual institutes of NIHand other Federal agencies (and numerous otherorganizations). Among the more formal synthesis-type activities within the Federal Government arethe consensus development conferences sponsoredby OMAR of NIH.

The goal of consensus development is to synthe-size the scientific literature on safety and efficacy/effectiveness and to recommend to physicians theappropriate use of technologies. In many respects,consensus development conferences are well doneand important activities. As discussed in chapter5, however, the NIH consensus conferences havedemonstrated weaknesses in terms of objectivelysynthesizing scientific information and in recom-mending guidelines for the appropriate use of thetechnologies they consider.

For instance, although the NIH panels are gen-erally composed of eminent physicians, a meth-odologist (i.e., a biostatistician or an epidemiolo-gist) is not always included, and the validity ofevidence from scientific research is not always ex-plicitly examined (see app. C). Thus, the metho-dological limitations of a given study may beoverlooked. Another limitation of NIH’s formatis the process itself. For instance, the use of adver-sary groups and task forces has been almost en-tirely abandoned, and the questions that havebeen posed are strictly limited to issues on whichthere is enough factual evidence to reach agree-ment. For the purpose of synthesizing availableknowledge, this approach may be adequate (as-suming that the available knowledge is all in-cluded and understood), but for the purposes ofidentifying gaps in knowledge and needs for futureresearch, this approach is weak. Of equal impor-tance is that consensus development conferencestend to examine in depth only two aspects of med-ical technology assessment: safety and efficacy.This limits the usefulness of the conferences andcalls into question the appropriateness of their set-ting guidelines for clinical use of a medical tech-nology (e.g., frequency of Pap smears or the useof mammography).

Setting medical standards (e.g., indications forusing respiratory therapy) by professional organi-zations and governmental agencies, though notcustomarily characterized as a synthesis activity,does depend on the integration of available infor-mation. Ideally, these organizations and the indi-viduals within them should first systematicallyand objectively review the clinical research evi-dence. A knowledge base (see ch.5), such as theNational Library of Medicine’s (NLM’s) Hepatitis

Ch. 7—Critique of the Current System Ž 101

Knowledge Base, may be useful in this regard. Animportant output should be the identification offertile areas for further research. However, thecommon pattern is for standards to be set, wheth-er by PSROs, HCFA, professional organizations,or NIH, that are based on the group’s belief ofgood medical practice, much of which is unsup-ported by scientific evidence. Thus, not only areimportant opportunities lost for further research,but perhaps more important, current practice pat-terns tend to be validated when they should notbe. Finally, not only is the research evidence gen-erally not reviewed systematically and objective-ly, neither is the standard-setting process. Formaldecision-assisting techniques such as Delphi andnominal group techniques are seldom applied.

OTA concludes that the synthesis phase of thepresent system of technology assessment is unnec-essarily weak within both the private and publicsectors. Research evidence regarding the safety,efficacy, and effectiveness of medical technologiesis seldom examined systematically and objective-ly. Federal agencies and private insurers and orga-nizations set policies, guidelines, and regulations,and/or make reimbursement coverage determina-tions, many of which profoundly affect the adop-tion and level of use of medical technologies. Yet,their decisions are usually based on informal, sub-jective, group-generated norms which tend to sup-port the status quo. Formal, more objective tech-niques both for evaluating research evidence andfor making decisions and setting policy could beused more often to aid in better decisionmaking.

Dissemination

The issues associated with making sure that theright people have access to technology assessmentinformation transcends technology class (i. e.,drug, device, procedure). However, the dissemi-nation issue is particularly important for the deci-sionmaker at the point of a technology’s adop-tion. At that point, the insurers, hospitals, physi-cians, or patients need to assimilate safety, effi-cacy, and cost information in order to make a ra-tional decision based on their individual condi-tions, values, and objectives.

This report does not deal with the entire scopeof information transfer. It does, however, brief-

ly examine the ability of the Federal Governmentto make available research findings and the activi-ties of NLM in indexing and providing access tothe biomedical and other health-related literaturethat may be useful for medical technology assess-ment. These issues are addressed in greater detailin a separate technical memorandum entitledMEDLARS and Health Information Policy (276).That document also discusses the relationship be-tween NLM and private sector organizations thatindex and provide computerized access to the bio-medical and other health-related literature.

Specific problems associated with communicat-ing information about medical technologies ap-pear to be similar to those in other fields of scienceand technology. Paradoxically, the amount of in-formation available is at once too much and toolittle. The “publish or perish” syndrome has ledto an explosion in the quantity of literature with-out an accompanying improvement in quality.One way to ameliorate the problem of an over-abundance of primary literature has been to rely

more on secondary sources, particularly biblio-graphic data bases that can be read by a computer.

NLM has excelled in collecting, indexing, andmaking accessible biomedical literature by a com-puterized bibliographic system. An earlier OTAstaff paper of this assessment indicated that about76 to 98 percent of the relevant biomedical jour-nals are covered by NLM’s major biomedical database MEDLINE (278). But subject coverage of thehealth care delivery field by contrast is poor: lessthan 40 percent of all relevant citations were con-tained in MEDLINE. Many of the missing cita-tions were in economic, business, and sociologicaljournals. The coverage of citations in the healthcare delivery field is limited, not only becausemany of the citations are in economics and busi-ness journals, but because a large number are alsoolder than the NLM health file. The percentageof relevant citations held in MEDLINE will be sig-nificantly greater for articles citing the more re-cent literature.

References to other sources of information onmedical technology assessments such as mono-graphs, reports, conferences, and Governmentdocuments are not nearly as accessible in otherbibliographic data bases as references to the jour-nal literature. Thus, many useful Government re-


search documents may not be used or may haveto await for the authors to publish the results ina refereed professional journal.

Along with the growth in literature in the bio-medical field has come confusion on the part ofmany users about obtaining information. Thelarge number of primary publications and evensecondary publications (e.g., bibliographies)makes it difficult for the occasional user to findinformation efficiently. Users with access to awell-trained and competent information specialistor librarian find their search simplified. However,the quality of libraries or information centers andthe quality of the staff vary. Furthermore, thereis no comprehensive single source where informa-tion about existing federally generated data basesin a field can be obtained. This complicates evenan informed user’s search and has resulted in theunnecessary duplication of information.

Two important issues related to NLM’s useful-ness in the dissemination of technology assessmentinformation are: 1) whether NLM should includemore Government reports and other nonserial lit-erature (especially in the area of health services)in its data bases, and 2) whether NLM shouldmodify its indexing process to indicate more usefulinformation as to articles dealing with researchfindings. With regard to the first issue, it shouldbe noted that the National Technical InformationService (NTIS) has major responsibility for Gov-ernment reports and perhaps NLM should notduplicate the collection, although NLM is expand-

CONCLUSION

Thus, OTA finds that there are major problemswith each of the four components of the presentsystem of medical technology assessment. The last

ing its data base somewhat in this direction. Aneffort could be made to link existing data basesso that a single search could access both NLM andNTIS data bases as well as any other sources re-lating to health questions. With regard to the sec-ond issue, one possibility would be for NLM tocarry a code within its citations that is related tothe methodological and statistical nature of thearticle. The editors of research journals could beasked to supply the necessary information (276).

Finally, the potential impact of the widespreaddistribution of microcomputers in physicians’ of-fices in the future could be significant. For in-stance, NLM’s data bases could be immediatelyaccessible, and if knowledge bases such as NLM’sHepatitis Knowledge Base were available, the dis-semination of technology assessment informationcould be much enhanced.

OTA concludes that better methods need to befound to communicate information about medicaltechnologies to physicians, researchers, and pol-icymakers. OTA also concludes that Government-generated reports, many of which maybe impor-tant to technology assessment, are not as accessi-ble as they could be. There is no mechanismthrough which all health-related Government re-ports can be identified or obtained. Finally, NLM’smission and capabilities should be examined todetermine whether more Government reportsshould be included in its data base, and whetherNLM should index articles to indicate their meth-odological and statistical nature.

chapter of this report provides Congress with op-tions to address what appear to be some of themost striking weaknesses.

8Policy Options

The great end of life is not knowledge but action.

—Thomas Henry Huxley

—

Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Legislative Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Organizational Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Research Funding Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Educational Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Congressional Oversight Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Private Sector Oversight Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Identification of Medical Technologies for Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . .Testing Medical Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Synthesizing Research Information and Group Decisionmaking Activities . . . . . . . . . . .Dissemination Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Establishing a Coordinated System of Technology Assessment . . . . . . . . . . . . . . . . . . . .

Page105106106107108108108109109109109110

8Policy Options

INTRODUCTION

As described in the previous chapter, the pres-ent “system” of assessing medical technologies ex-hibits deficiencies in a number of areas, One ofthe problems is that there has been no strategyor systematic plan for developing an effectivesystem:

1. to identify technologies to be assessed;2. to ensure that high-quality, relevant assess-

ments are carried out;3. to synthesize or coordinate the synthesis of

the resulting information; and4. to disseminate the information to Federal

agencies, health care providers, third-partypayers, patients, and other health care deci-sionmakers.

Elements of an effective system are already inplace—e.g., the Food and Drug Administration’s(FDA’s) processes for the regulation of drugs andthe National Institutes of Health’s (NIH’s) supportfor clinical trials. The problem is that theseelements have not become part of a coherent over-all system. The most important need is to bringforth, from the present multiplicity of agenciesand activities, a more rational and systematic ap-proach to promote and coordinate medical tech-nology assessment.

Achieving the goal of an effective system forassessing medical technologies will require a moreintegrated structure than now exists. An in-tegrated system for assessing medical technologiesneed not be centrally managed or controlled.However, an integrated system will require stronglinks between multiple organizations and agen-cies. Candidate technologies for assessment couldbe identified by a number of Federal organiza-tions—including NIH, FDA, Professional Stand-ards Review Organizations (PSROs), health plan-ning agencies, the Health Care Financing Ad-ministration (HCFA), the Veterans Administra-tion, and the Department of Defense—as well asprivate sector organizations. To ensure that the

most significant technologies are assessed, all in-volved organizations could participate in a pri-ority-setting exercise. Many medical technologiesneeding assessment are already in widespread use.In setting assessment priorities, therefore, it mightbe useful to establish new links to nongovernmentbodies such as medical specialty societies.

Mechanisms to fund assessments of high-pri-ority technologies would have to be developed.Federal research organizations, such as NIH andthe National Center for Health Services Research(NCHSR), should be involved. Private organiza-tions may also be interested in participating inassessments.

An important function of any system for assess-ing medical technologies would be to select ap-propriate testing methods. Although the ran-domized clinical trial (RCT) is accepted as the op-timal method for testing efficacy in most situa-tions, resource and other constraints make it im-possible to test every technology by this method.In some cases, alternative study designs may bemore useful.

Syntheses of information could be done by bothGovernment and private organizations to meettheir respective needs. Information could be fedback to organizations participating in the assess-ment process in a form most useful or acceptableto them. HCFA, for example, in making reim-bursement decisions, might be most interested inthe question of whether, on the basis of scientificevidence, a specific technology could be con-sidered to be efficacious.

To imagine how a coordinated technology as-sessment system could work, consider the assess-ment of a hypothetical high-priority medical tech-nology about which relatively little is known.First, it would be necessary to gather and syn-thesize information about the technology. Theprocess of synthesis, by pointing to gaps in

105


available knowledge about the technology, mightsuggest a need for further research. It might be,for example, that the extent of use of the tech-nology is not known; in that case, a simple data-gathering exercise by the health insurance systemmight be useful. It might be that the technologyhad been tested in normal subjects, but not inelderly people with chronic disease; in that case,its safety in the latter might need to be investigatedby surveillance.

All agencies and organizations participating inthe system could contribute to the assessment ofthis technology. Thus, for example, HCFA couldprovide information from its data base about theextent of the technology’s use. If questions aroseconcerning benefits in the usual practice of med-icine, selected PSROs might be asked to evaluatethese. Different testing methods could be usedsimultaneously to complement one another. Forinstance, a small RCT could be used to establishcausation, while an observational survey couldbe used to detect associations within a more di-verse population. Unlike RCTs, which generatetheir own data for analysis, observational studiestypically rely on existing, often large-scale, datacollection systems (e.g., Medicare claims files,vital statistics). For purposes of analysis, it is im-portant that these data systems be compatiblewith one another and be accurate.

When policy decisions about the technologyneeded to be made, the evidence of safety, ef-ficacy, and effectiveness would be synthesized,and the information disseminated to the appro-priate decisionmakers. The system would requiremechanisms to determine when a rigorous assess-

LEGISLATIVE OPTIONS

Organizational Options

1. Sponsor a private-public body or grant acharter to an organization to undertake med-ical technology assessment activities.

An organization could be chartered either asa separate nonprofit corporation or as part of anorganization (e.g., the Institute of Medicine of theNational Academy of Sciences) to undertake as-sessment activities that would complement Federal

ment was needed and when a more informal re-view was sufficient. If controversy existed con-cerning appropriate patterns of use for the tech-nology, group decision techniques such as thosedescribed in chapter 5 would be useful.

The initial concept of the 1978 legislation es-tablishing the National Center for Health CareTechnology (NCHCT) resulted from a recogni-tion in Congress of the need for a systematic ap-proach to the assessment of medical technologies.However, the NCHCT legislation left certainproblems unaddressed, e.g., who would set re-search priorities for the Government. Further-more, NCHCT’s mandate to perform assessmentswas curtailed by its austere budget. Consequent-ly, NCHCT’s impact on the health care systemhas been fairly small. If NCHCT’s funding is notrestored, however, an organization potentiallyable to carry out or coordinate the tasks men-tioned above will have been lost.

The policy options that follow are intended toaddress the deficiencies of the existing system forassessing medical technologies. The options aredivided into two broad categories: legislative andoversight. OTA finds that there are few realisticlegislative options necessary for Congress to con-sider. In most of the deficient areas noted withinthis report, congressional oversight may suffice.There is already substantial statutory authorityvested in the Secretary of Health and HumanServices to develop a coherent system of medicaltechnology assessment. The options below are notpresented in any particular order of importance,nor should they be regarded as mutually exclu-sive.

activities and serve the needs of consumers, pro-viders, and third-party payers. The organizationcould be composed of a number of groups con-cerned with the evaluation of health care: physi-cian and hospital professional associations, con-sumers represented through industry and labor,private health insurers, and academic centers.

One of several objectives that such an organiza-tion could have would be to stimulate the devel-

Ch. 8—Policy Options ● 107

opment of uniform and accessible data bases formedical technology assessment. This could includeencouraging the use of uniform diagnostic andprocedure coding, encouraging commonality ofpatient registration and claims forms, and devel-oping clinical data banks. A second objectivewould be to identify technologies for assessmentand to establish assessment priorities. A thirdwould be to develop and refine methods of assess-ment, including scientific, economic, and socialtools. This objective could include developmentof community-based collaborative studies, im-proved clinical data banks, better measures ofquality of life, etc. A fourth objective would beto conduct comprehensive assessments of medicaltechnologies, considering their scientific, eco-nomic, social, ethical, and legal implications; andto perform scientific and economic analyses at therequest of providers and third parties. The per-formance of such assessments would include thegeneration of new data as needed. A fifth objec-tive would be to disseminate new information andto serve as a clearinghouse of information on newtechnologies, assessments of technologies, etc.

Initial funding for the organization could comefrom private foundations. Ongoing support mightinclude some support from foundations, contribu-tions from insurers for support of assessment ac-tivities, congressional appropriations for specialassessments of interest to the Federal Government,and support from hospital associations for adviceon use and distribution of technologies.

One of the advantages of this option’s generalapproach is that it would capitalize on private sec-tor initiative and interest and would rely on pri-vate as well as possible public funding. A com-bination of private and public sector involvementmay be essential for any system of medical tech-nology assessment to be acceptable to all partiesconcerned. Apart from the very real possibilitythat an effective arrangement could not be forged,disadvantages of this approach include potentiallegal problems with funding—e.g., possible,though not likely, antitrust violations, and in-terference with State laws governing the healthinsurance industry.

A variation of this option, presented in appen-dix F, would establish a private-public bodytermed an “Institute for Health Care Evaluation. ”

A limitation to the particular model proposed inappendix F is that it deals with medical technologyassessment primarily as it relates to the reimburse-ment system. Thus, it may be unnecessarily re-strictive. Both the legal issues noted above andethical concerns associated with selectively reim-bursing for health care technologies are discussedin appendix F.

2. Maintain the authority of and fund NCHCT.

Several advantages would result from refundingNCHCT. In the few years of its operation,NCHCT was making progress on several fronts.Perhaps most importantly, the Technology Coor-dinating Committee of the Department of Healthand Human Services (DHHS), chaired by the Di-rector of NCHCT, provided a valuable frame-work for the coordination of technology assess-ment within the Government; NCHCT’s confer-ence (e.g., on coronary artery bypass surgery)were successful as a needed adjunct to the moremedically oriented NIH conferences; NCHCTprovided an important focal point for HCFA tointeract with the Public Health Service (PHS) forcoverage determinations. Refunding NCHCTwould allow it to continue this work and tomature as a Federal agency. Furthermore, evenif option 1 above were implemented, the Federalinvolvement would still require interagency

coordination.

The disadvantages of this option include mostof the arguments which recently led Congress notto fund NCHCT for fiscal year 1982. A majorconcern at that time were the assertions by themedical devices industry that NCHCT’s “emerg-ing technology list” inhibited innovation, Theother major concern was that NCHCT’s activitiesmight not be needed, because professional medicalsocieties are increasingly active in technologyassessment and PHS may be able to manage manyof NCHCT’s former responsibilities.

Research Funding Options

3. Change the statutes so that HCFA can selec-tively reimburse for experimental technolo-gies in return for clinical data on these tech-nologies.

This option has several potential advantages.First, the actual implementation of this option


would not necessarily involve additional costs.Second, the implementation of this option mightprove over the long run to be an effective methodof cutting costs. Decisions to reimburse for manytechnologies which are essentially experimentalare now made before adequate safety, efficacy,and cost-effectiveness information is available. Ifimplemented properly, this option could substan-tially increase the quality of information availablefor reimbursement coverage decisions, therebyyielding substantial budgetary savings.

The possible disadvantages of this option arealso substantial and are similar to some of thoseof option 1. Primarily, the problems concern thelegal and ethical implications of selectively reim-bursing for health care. Before Congress serious-ly considers exercising this option, therefore, itwould probably need to conduct extensive hear-ings concerning possible adverse consequences.Possibly, elements of PHS could be involved indeveloping research protocols and in interpretingresearch evidence from the resulting experiments.If option 2 above is exercised, NCHCT could per-form these duties.

Educational Options

4. Increase funding to train researchers in meth-odological and statistical principles.

This option is a general one that could be ac-complished through a variety of existing educa-tional programs. One advantage of this option isthat the quality of both privately and publiclyfunded research could be expected to improveover time; the quality of the synthesis of researchfindings could be expected to improve as well. Thedisadvantages are that this option would requireadditional funding and would not produce imme-diate results.

5. Increase efforts to train health professionalsin methodological and statistical principles.

This option could be exercised either by cat-egorical funding for additional training or throughcongressional oversight with respect to the educa-tional curricula of professional and continuingeducational programs. One advantage of this op-tion is that it would help to increase the qualityof research performed by clinical professionals.Perhaps more importantly, it would help to en-sure that such professionals are more informedabout the value and limitations of researchliterature in their respective fields. Disadvantagesmight be the cost of increased training efforts andthe lack of immediately observable results.

CONGRESSIONAL OVERSIGHT OPTIONSAn option involving the private sector and eight

other options involving the powers already vestedin the Secretary of the Health and Human Serv-ices are discussed below. Congress could exercisethese options by using its oversight powers.

Private Sector Oversight Option

6. Encourage the private sector to take the leadin assessing medical technologies.

As noted in chapter 6, there is evidence thatthe private sector is increasing its technologyassessment activities. The advantages of this op-tion are that it would require no additional fund-ing, would probably be more attractive to ele-

ments of the private sector than other options,and would capitalize on an existing trend at anearly stage. Disadvantages of this approach in-clude the problem of differing private and publicobjectives; because of these differing objectives,much of the research conducted by the private sec-tor may not be of high priority to Congress orthe Secretary of Health and Human Services. Afurther problem may be a low level of funding.Since technology assessment is apt to be very ex-pensive and since the information it produces isgenerally regarded as a public good, any oneprivate party has an incentive to let someone elsepay for it. Finally, the private sector does not havean impressive record in the assessment field; mostpast efforts have been Federal ones or have beenrequired by Federal law.

Ch. 8—Policy Options . 109

Identification of Medical Technologiesfor Assessment

7. Examine how Federal research institutes(e.g., NIH), agencies (e.g., NCHSR), and re-search programs of operating agencies with-in DHHS (e.g., the Office of Research andDemonstrations of HCFA) could identifytechnologies better when setting theirresearch agendas; and how the PSRO pro-gram and the reimbursement system itselfcould be used to more advantage for identi-fying candidate technologies.

As discussed in chapter 7, Federal programs doan inadequate job of identifying technologieswhich need assessment, especially medical andsurgical procedures. This option is intended to ad-dress that problem.

Testing Medical Technologies

8. Continue to conduct oversight hearings con-cerning the duplication and fragmentation ofhealth-related data collection activities.

9. Examine the ability of operating agencieswithin DHHS (e.g., HCFA) to generate suf-ficient information for their decisions relatedto medical technologies, and the extent towhich the Secretary of Health and HumanServices utilizes the department’s otherresearch arms (e.g., NCHSR, NIH) to pro-cure that information in a timely manner.

10. Examine the activities, plans, and potentialfor elements of DHHS (e.g., NIH) in utiliz-ing various research methods to determinethe appropriate use of medical technologies.

The duplication and fragmentation of health-related data collection has been discussed inprevious OTA reports and is well known to Con-gress. If Congress believes there is a continuingneed to evaluate data collection activities and tomatch their value with both research and oper-ating needs, it may wish to exercise option 8.

As discussed in chapters 6 and 7, operatingagencies of DHHS (including HCFA), PSROs(within HCFA), and State and local health plan-ning agencies require information for the decisionswhich they make, yet they all have limited abili-ty (funding, expertise, and/or legal mandate) to

secure such information. Option 9 would allowCongress to investigate this matter further.

Option 10 addresses research methods. As dis-cussed in chapter 3, the selection of optimalresearch methods for evaluating different technol-ogies at different stages in their lifecycle is verycomplex. Although an RCT is often ideal forstudying efficacy, other methods may be more ap-propriate for such things as safety or for technol-ogies in widespread use. This option would per-mit Congress to encourage the use of appropriatemethods.

Synthesizing Research Information andGroup Decisionmaking Activities

11. Explore how research evidence could be bet-ter evaluated by HCFA and its carriers andfiscal intermediaries when making reim-bursement decisions, by PHS when making

recommendations to HCFA on coverage pol-icy, by PSROs when setting standards forcare, and by the Office of Medical Applica-tions of Research of NIH when conductingconsensus development conferences; andmonitor the progress and potential costs andbenefits of the National Library of Medi-cine’s (NLM’s) knowledge base prototype.

As discussed in chapters 6 and 7, many deci-sions are currently being made by Federal agen-cies regarding premarket approval, reimburse-ment, and appropriate use of medical technolo-gies. As discussed in chapter 5, however, thereis good reason to believe that the evidence fromresearch is seldom carefully and objectively ana-lyzed before these decisions are made. This op-tion would help ensure that these decisions arebetter informed and would assist in establishing

research agendas for Federal agencies.

Dissemination Activities

12. Examine the disposition of federally gener-ated reports to determine the degree to whichthey have been useful both to private andpublic researchers and policymakers; specif-ically conduct an oversight hearing on the


13.

ability of researchers and policymakerslocate, retrieve, and use these reports.

to

Examine whether NLM’s literature baseshould be further expanded, especially to in-clude more Government research reports andother nonserial literature; and examinewhether there are more useful ways to indexarticles which contain findings from research.

Federal agencies that conduct or fund researchgenerate reports that may be useful for technologyassessment. As discussed in chapter 2, numerousagencies and other organizational units of DHHSare involved in disseminating Government reportsand other health-related information that is usefulfor medical technology assessment. The deposi-tion of all research reports and other Governmentdocuments to distributing organizational units isnot mandatory. Option 12 would allow Congressto ascertain whether, and the degree to which,federally generated information is useful and ac-cessible to the people and agencies conductingmedical technology assessments and making re-lated health policy decisions.

As discussed in chapter 2, NLM is primarilyoriented toward the biomedical research com-munity. Increasingly, however, the health servicesresearch community is looking to NLM for assist-ance in locating and retrieving health services in-formation. Option 13 suggests two areas thatCongress may wish to explore. *

Establishing a CoordinatedTechnology Assessment

System of

14. Encourage use of the powers vested in theSecretary of Health and Human Services todevelop a coherent system of medical tech-nology assessment.

As already discussed, a recent decision wasmade by Congress not to fund NCHCT. If Con-gress does not choose to restore NCHCT funding(see option 2), it may wish to consider this option.

*NLM’s role in the dissemination of health-related informationis explored at greater length in OTA’S technical memorandum en-titled MEDLARS and Health hformation Policy (276).

Appendixes

System/organization Purpose and focus of data Method of data collection –

. Statistics generatedPopulation/locatlorr/time coverage

CDBasic Vital StatisticsNCHS To provide uniform data on births, infant

deaths, fetal deaths, birth weights,gestation

Birth and death registration in States 100percent registration of all births anddeaths in the country

Total United States since 1933 registration for all Iocations on an ongoingbasis

Frequency of deaths by cause and fetaldeath Frequency of death

National Health Interview SurveyNCHS To provide data on acute and chronic Ill-

ness prevalence among noninstiutional-ized persons

Morbidity Survey-Multistage probabilitysample of standard geographic primarysampling units census enumerationdistricts and households Interviewsconducted in households

120,000 civilian noninstitutionalized in`dividuals throughout the United Statescovered each year since 1957 Datacollected weekly throughout the year

Estimated frequency of physician visitsand hospitalization Prevalence of chron-IC conditions Incidence of acute con-dition Morbidity associated with acuteconditions

National Health and Nutrition Examination Survey (NHANES)NCHS To provide data on health status and phys-

iological measures of noninstitutional-ized persons

Morbidity Health Survey—Multistage,highly clustered probability sample ofpersons stratified by geographic regionand population density grouping Primary sampling unit IS the same as forNational Health Interview Survey Datacollected by interview physician exam-ination measurement and laboratorytesting

Two cycles of NHANES have been con-ducted on the noninstitutionallized U Scwdtan population, Cycle ( 1971-74,Cycle I augmentation 1974.75, CycleII 1976-79 A special survey of HIs-panics IS scheduled to begin In 1982

Morbidity measures Distribution of phys-iologic variables Prevalence of chron-iC conditions

3CD

National Hospital Discharge SurveyNCHS To provide medical diagnosis and surgical

data on patients discharged from non-Federal short-slay hospitals

Hospital Utilization and MorbiditySurvey-Two-stage cluster probabilitysample Hospital selection stratified bybed size region, and ownership Sys-tematic sample of discharged patients

Since 1964, over 400 hospitals surveyedper year throughout the United States

Estimated frequency of specific diagnosesand surgical procedures by patient andhospital characteristics

National Death IndexNCHS To provide possible fact of death the

death certificate number, and State ofdeath

Registry of all death records in the UnitedStates transmitted by the States orother death registration areas

information for all States beginning in1979 Updated annually thereafter

No statistics generated Computer searchesused to assist researchers to deter-mine whether persons m their studiesmay have died

National Natality SurveyNCHS To provide in-depth data on newborns and

maternal health Prenatal and postnatalcare, infant health, medical aspects ofpregnancy, labor, and delivery

Followback survey Probability sample of 1In 425 live births drawn from birth cer-tificates

54 birth registration areas span the UnitedStates Natality surveys done on recordsof 1963-69 and 1972 1980 survey IS

currently underway

Demographic statistics of parents Fre-quency of radiologic procedures (1963)Factors associated with maternal and in-fant health

National Fetal Mortality survey

NCHS To provide data on the health of womenwho have stillbirths Prenatal and post-natal care medical aspects of preg-nancy labor, and delivery

Followback survey Probability sample oftwo in five fetal death certificates

1980 survey, covering 52 birth registrationareas IS currently underway

Parental occupation Fact of maternal radi-ologic exposure Recent pesticides andinsecticide exposure

National Ambulatory Medical Care Survey (NAMCS)NCHS To provide data on use of office-based

physiciansMorbidity Survey—Multistage probability

sample of non-Federal physicians inoffice-based practice A systematic sam-ple of results during a year within physi-cian offices

Since 1973, an annual cycle of NAMCShas been conducted on noninstitutional-ized visits to non-Federal physicians inthe 48 contiguous United Stales

Frequency of visits by diagnosis Vmltrates to physcians by age race sexand type of physician

National Disease Surveillance ProgramCDC To examine disease and trends of 45 spe-

cific conditions, to identify regional prob-lems, and to evaluate the effectivenessof control measures

Reporting System—Through epidemiologyand laboratory Offices of State healthdepartments Supplemented by infor-mation from epidemic investigations oroutside sources such as NCHS

State county and city health authorityareas throughout the United States

Incidence of 45 specific disease condi-tions

-. . .-4 . . . - -. J-. . ..-. L-. . . A -,, -.. Population/locatlon/time coverage Statistics generatedSystem/organization Purpose and focus of data Method of data collection

Birth Detects Monitoring SystemCDC To provide data on type and incidence of

birth defectsReporting System—Self-selected sample of

hospitals Commission on ProfessionalHospital Activities (CPHA) hospitals withobstetric units are requested to partic-ipate. Report aggregate data on birthdefects

Includes approximately one-third of allU S births Quarterly reporting beganin 1970

Incidence of birth defects

Hepatic Angiosarcoma Surveillance Finding EffortCDC To understand relationship between angle.

sarcoma with health history and riskfactors

Incidence, risk factorsNational case findmg effort. 167 cases found, between 1964 and1974

Smokey Nuclear Test Cohort StudyCDC To understand relationship between can-

cer, particularly Ieukemla, and nucleartest exposure

Incidence and mortality dataComplete followup of all film badge hold-ers

Department of Defense workers at SmokeyNuclear Test Site m 1957 who worked1 week prior to or after nuclear explo-slon and has possible exposure

Alcoholism Program Monitoring SystemNIAAA (ADAM HA) To provide data to plan, manage and eval-

uate alcoholism

Client-Oriented Data Aquisition Process (COOAP)NIDA (ADAM HA) To provide data on federally supported

drug abuse and rehabilitation programs

National Drug Abuse Treatment Utilization SurveyNIDA (ADAMHA) To provide data on nationwide resources

devoted to drug abuse treatment, theiruse, and their dlstribution

Drug Abuse Warning Network (DAWN)NIDA (ADAMHA)/DEA To provide data on drug-related deaths,

medical emergencies, and psychologicalcrises

N/A Programs funded by NIAAA Data on clients, services, and costs

Client-related data

Use and distributuion of resources for drugabuse treatment

Mortality and morbldity incidence

Required reporting system

Systematic survey, participation voluntary

Collection at time of admission to, and dis-charge from, treatment programs

Annually—nationwide

Voluntary reporting system Participating emergency rooms, medicalcoroners, crisis intervention centers

Adverse Drug Reaction Spontaneous Reporting SystemFDA To provide for reporting of adverse effects

of pharmaceutical products Fact ofdeath, health outcome after reaction.concomitant diseases

Poisoning Control Case RegistryFDA To provide registration of acute poisoning

incidents Signs and symptoms of poi-sioning, required medical intervention

Reporting system—Drug manufacturershospitals, health care practioners filereports

Since 1968, 167,000 reports have beenfiled across the United States

Types of reactions by drug

Registry—Volunfary reporting of all con-tracts with 400 poison control centers

Since 1971, 1 2 million records of inci-dents have been placed in this systemfrom across the United States

FDA use only

Registry of Tissue Reactions to DrugsFDA/Armed Forces lnstl- To provide historical, clincal, laboratory,

tution of Pathology and morphological findings of adverseunder auspices of Unl- drug reactionsversities Associated forResearch m Educationand Pathology, Inc

Registry of Dermatological Reactions to DrugsFDA/National Academy of To improve the reporting of drug-caused

Dermatology skin reactions, and to test the feasibilityof using a specialty society m medicineas the focal point for collecting drug ex-perience data

3,753 cases from 37 States since 1966Reglsfry—Medical facilities reporting sys-tem

Quarterly reports of individual cases as tocause, basic disease, part of the bodyaffected, and suspected drug(s)

Registry—24-hour loll-free telephone re-.porting system

National coverage, 3-year study starting inDecember 1980

Incidence data

National Ragistry for Drug-Induced Ocular Side EffectsFDA To provide better Information on ocular

side effects of drugsRegistry-Cases are reported primarily by

opthamologists

Prospective registry

National—700 cases reported between1977 and April 1978

Registry of Adverse Reactions to Contrast MediaFDA/international Society To provide greater detail to the current

of Radiology study of adverse reactions to intravas-cular contrast media

30 teaching hospitals m the United StatesCanada, and Europe 5,546 patientsrecorded with reactions out of 112,003cases time period not reported

Nationwide Evaluation of X-Ray Trends (NEXT)Bureau of Radiological To detect and evaluate the extent of the

Health (FDA) population’s exposure to X-rays used inmedical and dental examinations

Systematic sample for participating States Hospitals, private offices, and clinics Incidence rates

Medically Oriented Data System (MOOS)Bureau of Medical De- To provide an estimate of the frequency of

vices (FDA) use of various drugs, devices, and pro-cedures as well as the frequency of ad-verse events following from such use

Systematic sample from participating hos0-pitals

Since September 1974 since 1977 hasbeen a sample of 26 U S short-termhospitals

Incidence rate

Registry for Implanted Artificial Cardiac PacamakersFDA To provide data on new implant cardiac

pacemakers, replacements, and inter-current procedures

Registry of three medical center ex-periences

Medical Centers at University of SouthernCalifornia, Montefiore Hospital, andNewark Beth Israel from July 1, 197410Dec. 31, 1979, 5,070 registered pace-makers overall

Use patterns Morbidity and mortalityrates

Surveillance, Epidemiology, and End Results Program (SEER)NCI (NIH) To measure type and site of cancer inci-

dence and survival m the United States,extent of disease, annual vital status

Morbidity survey-Sample of personsdiagnosed with cancer 10 SEER loca-tions chosen for contractual reasons, allcases of cancer m SEER area sampled

5 entire States and 5 metropolitan areashave been surveyed on an ongoing ba-SIS since 1973 Some of these Iocationssurveyed m earlier NCI cancer surveysPopulation in these Iocations constitute10 percent of U S total

Survival rates Incidence of cancer by siteof cancer, geographic area race, andsex

Framingham Heart StudyNHLBI (NIH) To understand relationship between per-

sonal Characteristics, physiologic mea-sures, physical signs and heart disease,cause of death

Prospective study —Representative sampleof 1940 Framingham Population Follow.up of offspring of initial sample.

Followup of 5,209 persons m initial cohortages, 29 to 62, in 1950, and 5,135 off-spring, ages 16 to 52, in 1973 Follow-UP began m 1940’s and has continuedup to present in Framingham, Mass

Cardiovascular and cancer incidence Riskstatistics Mortality rates

Multirisk Factor Intervention TrialNHLBI (NIH) To understand relationship between risk

factors and heart disease physiologicmeasures, physical signs and fact ofdeath from heart disease

Intervention trial—Two-stage samplingscheme. Nonprobability sample at 20competitive research sites Identificationof males at high risk of heart disease

Intervention trial Two-stage samplingscheme Nonprobability sample of 14communities. Identification of personswith elevated blood pressure in a settime period.

Followup of 12,866 men. ages 35 to 57,from 1973 to 1976

Not yet available

Hypertension Detection Second Followup ProgramNHLBl (NlH) Physiologic measures, physical signs, fact

of cardiovascular disease and death re-corded to understand relationship be-tween drugs and control of hyperten-sion

Since 1973, followup of 211,000 persons,ages 30 to 69

Morbidity and mortality rates

System/organization Purpose and focus of data Method of data collection Population/locallon/time coverage Statistics generated

Continuous Disability History SurveySince 1967, 15,000 records have been

placed in this file per year fromthroughout the United States

Frequency distribution of disabilty-relatedprimary diagnosis by State

SSA

Leed FileSSA

o provide data on applicants for disabilitybenefits under Social Security Act titleII Disability by diagnosiss for in-dividuals

To provide Integrated occupational and

Survey of applicants for title II benefitsProbability sample of claims stratified byState

percent digital probability sample of so-cial security numbers Integration ofwork history from Annual EmployeeEmployer File with Summary EarningFile File provides longitudinal infor-mation by person

Since 1957 1-percent sample of wage

and salary workers covered under So-cial Security

Employment and earnings information fromIongitudinal analysishealth status” information on social se

curity number holders Fact of disabilityand death

Annual Disability Determmatlors/Social Security Income ExtractSSA To provide data on applicants for benefits

under Social Security Act title XVI Dis-ability by cause

Since 1975 100000 records have beenplaced in this file per year fromthroughout the United States

Frequency of disease and accident.relateddisability claims by State

Survey of Claimants for title XVI benefits10-percent probability sample of claim-ants stratified by State with oversampling of claims in small States or forchildren

Census of the PopulationBureau of the Census To provide demographic data on entire

(Commerce) population

Annual Occupational Injuries and Illness SurveyBLS To provide data on work-related disease

and injury Acute disease and acute in-jury by cause, with or without lost workdays Fact of death

Enumeration of all residents United States every 10 years Demographic and socioeconomic character-istics

Morbidity Record Survey —Probability sample of employers under OSHA record-keeping stratified by Industry and es-tablishment size

Since 1972, ongoing surveys of nearly allprivate sector Industries across theUnited States

Incidence of illness and injury by type ofcase per plant-hour worked

CHAMPUS (Civilian Health and Medical Program of the Uniformed Services) Management Information SystemDependents of active duty personnel, re-

tired members of the armed forces, andothers

Utilizatlon statisticsDOD To provide reformation for eligible bene-ficiary programs Provider, claims, uti-Iization and management data

TRI-Service Medical Information Systems (TRIMIS) ProgramDOD Clinical and adminstraive automated data

processing for military medical treat-ment facilities

Compensation and Pension SystemVA To provide data on recipients of VA

benefits Disability by cause Factof death in or out of service

Patient Treatment FileVA To provide data an VA system discharges

Medical or surgical diagnoses

Blue Cross-Blue Shield SystemsOver 100 members of To provide information on beneficiaries for

Blue Cross-Blue Shield reimbursement and utilization review/Association quality control generally

Registry of program recipients

TRIM IS systems m operation include Phar-macy Formulary, Hypertension Manage-ment, Hospital Logistics and AutomatedCardiac Catherization Laboratory

Patient registry 80 military medical facilities potentiallyabout 89 million beneficiaries in DODcommunity

Registry—Unwersal coverage of allveterans discharged with disability andreceiving benefits

Registry --Universal sample Medicalrecords abstracts for each hospitaldischarge

Registry of claims forms for reimburse.ment

Since 1960, all veterans discharged whoreceive benefits far military-relateddisability

Level of compensation by diagnosticcodes

Since 1969, all VA systems patientsdischarged

Frequency of diagnostic category

More than 80 million people m the UnitedStates SIX million of these are Federalworkers (which for some research,approximates a national data set)

Utilization Statistics (data elements mayvary from member to member)

System/organization Purpose and focus of data Method of data collection Population/locatlon/time coverage Statistics generated

Computer-Stored Ambulatory Record System (COSTAR)Massachusetts General To provide automated medical records and

Hospital/NCHSR business support, quality assurance,patient followup reminders and selec-tion of preferred therapies

Problem-Orientation Medical Information Systems (PROMIS)

Data bank of beneficiaries Harvard Community Health Plan Utilization Statistics

Patient clinical and utilization statisticsUnversity ofVermont/ NCHSR

Computerized data bank of patient recordsentered by professional personnel

Available since 1977To restructure medical records and data toorganize and help direct the process ofclinical care and medical action

Hospital Discharge Data SystemsAn estmated 18 to 20

private nonprofit hos-pital discharge abstractsystems, including theC o m m i s s i o n P r o f e s -sional and HospitalActivities (CPHA), andthe Hospital UtilizationProject (H UP)

To provide summary information aboutpatients and their episodes of Illness inshort-term hospitals

Registry—Pat!ent information forparticpating hospitals IS abstractedfrom medical records by hospitalpersonnel after patients are discharged,according to a prescribed format

About half the hospitals m the UnitedStates, representing about 20 milliondischarges annually

Utilization and clinical Statistics

DES Vaginal Cancer RegistryOr Arthur Herbst/

University of ChicagoMortality and morbidity data, clinical

historiesTo investigate the clinical, pathologic, and

epidemiologic aspects of clear ceiladenocarcinomas occurring in the vagm+rand cervix of females born after 1940

Voluntary registry International in scope, with 341 casesreported, established since 1971Considered highly precise, i.e., hascaptured about 85 to 90 percent of DESuniverse

Rhode Island Diabites RegistryRhode Island

Department ofHealth

To provide information on young insulin Prospective 3-year recustry charts Patients who are1 ) insulin-dependent at 15 acute-carehospitals in the State, 2) under 30, and3) residents of the State

Incidence, patient-care patternsdependent diabet ics (Type I juveni le . reviewed on - weekly or biweekly basis,diabetic) with individual physicians subsequently

interviewed

Retrospective registry

S IX data banks at medical institutions

Pittsburgh Diabeties RegistryN/A To provide Information on the etiology of Pittsburgh, Pa , metropolitan area covering

a 12-year periodIncidence data

Incidence rates, clinical patterns

juvenile onset diabetes

American Rheumatism Association Medical Information System (ARAMIS)Consortium of several To collect Patient information on a variety

medical groups of rheumatic diseases among differentcentered patient populationsat Stanford UniversityMedical Center

Duke University Cardiovascular Data bankDuke University To provide onformation on patients with

Medical Center/NCHSR known or suspected Ischemlc heartdisease and to describe outcomes ofpatients with various sets of attributes,patients findings, histories, outcomes

Databank on patients in coma for causes other than head injuryNew York Hospital- To provide information on nontraumatic

Cornell University coma patients, allow future predictiveMedical Center power and clinical Improvement m treat-

ment, symptoms, diagnoses,procedures. functional states, causes ofdeath

10,000 patients in system m the UnitedStates and Canada, extensive followupin some cases

Computerized data bank Over 6,000 Patients a! Duke UniversityMedical Center since 1976

Incidence and clinical patterns

Accesses about 500 patients per year Morbidity and mortality data, patterns ofclinical practice

Computerized data banks located inseveral medical centers m the UnitedStates and Great Britian I

I

Statistics generated

Morbidity and morality data patterns ofclinical practice

System /organization Purpose and focus of data Method of data collection Population/locatlon/time coverage

Intensive Care DatabankMassachusetts To collect Information and evaluate

General Hospital practices regarding intensive care unitsdiagnoses, functional status, indicationdata, charges

2,305 patients admitted to one of the threeintensive care units at MassachusettsGeneral from July 1977 to July 1979

Computerized data bank Patient recordsand subsequent Interviews wererecorded and coded

SEARCHRhode Island Health

Services Research,Inc

Vital statistic utilization rates facilityand expenditure data

To provide a neutral resource organizationwhose health information sources couldservice a diverse need of all the State shealth providers and planners, censusand vital statistics, planning and utiliza-tion data

Computerized data bases S!ate-widecooperate reporting system 10 privatenot for-profit organization

State of Rhode Island since 1968

Drug incidence and utilization dataDrug Epidemiology UnitBoston University

Medical CenterEstablished July 1976, over 10,000 indi-

viduals studied m United States andCanada

To study a broad range of problems con-cerning the clinical effects of drugs mhumans, with particular emphasis onadverse effects, life-time histories ofdrug use, patients characteristics anddiagnostic information

Pediatric Drug Surveillance ProgramBoston To provide an estimate of the incidence of

Children’s adverse reactions m hospitalizedHospital childrenMedicalCenter incollaborationwith DrugEpidemiologyUnit

Boston Collaborative Drug Surveillance Program (BCDSP)BCDSP/FDA To provide some quantification of clinical

efficacy and toxicity for prescribeddrugs m specific types of patients.

Dunedin ProgramDunedin, Florida To screen participants over the age of 65

Clinic for medical disorders

Olmstead Country, Minnesota SystemMayo Clinic and To provide a complete medical record and

Olmstead Medical Information system for a circumscribedand Surgical Group population for Clinical and followup care

all contacts with the medical system arerecorded

Seatte Group Health Cooperative SystemSeattle Group Health To develop comprehensive system tabula-

Cooperative tion on inpatient procedures and diagno-ses, outpatient drug utilization enroll-ment features, and outcome measures

Case-control surveillance system in 11medical centers nurse-monitors inter-view subjects

Drug Incidence and utilization dataCase-control surveillance, nurse monitorscollection of information

Established 1974, to date over 4500 pedi-atric patients from premature newbornsto young adults

Drug incidence and utilization rates Effi-cacy ratings of drugs used

Surveillance among hosptalized medicaland surgical patients by speciallytrained monitors

Since July 1966 m over 40 hospitals andseven countries on nearly 100000patients

Data from patient records and interviewswere collected

Over 5,000 participants, an ambulatorygeriatric population

Drug Incidence and utilization rates

About 98 percent of Olmstead Country res-idents, medical records for populationdating back to 1907

Incidence and utilization patternsComputerized medical records

Over 270,000 members, 2 demographicbase that resembles metropolitanSeattle, Wash

Centralized automated data bank Incidence and utilization rates, mortality,morbidity data, patterns of clinicalpractice

AGRICOLA (formerly CAIN)Agriculture, animal and USDA

plant sciences

NLMBRS, DIALOG,

SDC

NLM

BRS, IRS,DIALOG, SDC,

BRS, DIALOG,SDC

NLM

NLM

DIALOG, SDC,

NLM

Citations to literature of agriculture Journals, monographs, governmentreports

international—1970to date

Primarily UnitedStates—wthln thelast 10 years


12,000 per month

AVLINE(AudioVisuals onLine)Science and technology NLM

(Iife sciences)Descriptions of over 8,000 audiovisual and

nonprint health science teaching mate-rials

200 records permonth

BIOETHICSLINEScience and technology Georgetown

(life sciences) biblio- Kennedy ln-graphic stitute/Center

for Bioethics

BIOSIS PREVIEWSScience and technology BIOSIS

(life sciences) biblio-graphic

60 indexes, 70 journals, other data basesCorresponds to “Bibliography ofBioethics ‘‘

450 per quarterCitations to the Iiterature on numerousmoral, ethical, and policy issues of con-cern to the medical community

Citations worldwide literature of researchin Iife sciences Orignal researchreports, reviews of research, document-ation, and retrieval information

8,900 periodicals, books, monographs,conference proceedings, research com-munications, symposia are screenedCorresponds to “Biological Abstracts(BA)” and “Biological Abstracts/RRM “


14.000 records fromBA every month,11,000 recordsf rom “BA /RRM”per month

CA SEARCH; CASIAScience and technology Chemical

(chemistry) biblio- Abstracts Ser-graphic vice (CAS)

Citations to literature in chemistry Journalarticles, monographs, conference pro-ceedings, technical reports, andpatents

Merger of Chemical Abstracts Condensates(bibliographic information from the print-ed “Chemical Abstracts)” and “CASIA(Chemical Abstracts Subjects IndexAlert)”

lnternational—1967to date

40,000 records permonth

CANCERLIT (formerly CANCERLINE)Science and technology National Cancer

(life sciences) biblio- Instltute, ln-graphic ternational

Cancer Re-search DataBank Program

Citations abstracts of literature on oncolo-gical epidemiology, pathology, treat-ment, and research

Primary literature, articles selected fromover 3,500 journals, monographs, tech-nical reports, conference proceedings,

International—l 963to date

About 4,000 citationsper month

and theses

CANCERPROJ (Canxcer Research Projects)Science and technology Current Cancer

(life sciences) Research Pro-ject AnalysisCenter oper-ated by Smith-sonianScience lnfor-mationExchange (SSIE)

CHEMEX; CHEMLINE: CHEMNAME TM; C H E M XScience and technology Chemical

(chemistry) properties Abstracts Ser-vice (CAS)

Summaries of ongoing and recently com-pleted cancer research U S Federaland non-Federal SSIE.

International—past 2

to 3 fiscal yearsQuarterly, average

size IS 18,000projects.

Chemical dictionaries based on CAS Regis-try Nomenclature File, nomenclature,synonyms, structural data, molecularformula, ring system Information

Vanes, from quar-terly to less fre-quently

Online avail-Data base/ ability Sources of Coverage (earnest Updatingsubject Producer (United States) Content references dates available) (approximate)

CHILD ABUSE AND NEGLECTSocial sciences (biblio- Department of

graphic and referral) Health andHuman Ser-vices, NationalCenter ofChild Abuseand Neglect

CLINPROT (Clinical Cancer Protocols)Science and technology National Cancer

(life sciences) Institute,InternationalCancer Re-search DataBank Program(ICRDB)

CDI (Comprehensive Dissertation Index)Multidisciplinary University Micro-

films, Inc

CPI (Conference Papers Index)Science and technology Cambridge Sci-

general entificAbstracts

DRUGINFO and ALCOHOL USE/ABUSESocial sciences (biblio- University of

graphic) Minnesota,College ofPharmacy,Drug lnfor-mation Ser-vice Center

ENVIROLINEScience and technology Environment

(environment) InformationCenter

EPILEPSYLINEScience and technology National lnsti-

(life sciences) biblio- tutes ofgraphic Health, Nat-

ional Instituteof Neurologicaland Communi-cative Dis-orders andStroke

DIALOG Citations and abstracts of materials def-inition, identification, prevention, andtreatment of child abuse and neglectReferences to the literature audiovisualmaterials, excerpts of current lawslistings of U.S. service programs

“Child Abuse and Neglect ResearchProjects and Publcations, Child Abuseand Neglect Programs’ and ‘‘ChildAbuse and Neglect AudiovisualMaterials “

United States-1965to date.

Research projects,audiovisual mate-rials and legalcitations, annually;literature, semi-annually

NLM Summaries of ciminal investigations onnew anticancer agents and treatmentmodalities.


Quarterly, severalhundred recordsper year

BRS, DIALOG,SDC

DIALOG, SDC

BRS

Citations to all dissertations accepted forPh. D at accredited U S and 210 for-eign institutions.

Corresponds to printed “Dissertation Ab-stracts international (DAI)’ and“American Doctoral Dissertations(ADD)”.

Corresponds to printed ‘‘ConferencePapers Index”

international—1961to date.

3,000 ‘ ‘DAI” permonth and 3,000“ADD” per year

Citations to papers presented at 800 sci-entific meetings.



Two files on alcohol and drug use/abusecitations to monographs, journals, con-ference papers, instructional guides,films on aspects of alcohol and druguse/abuse. Research in the area ofchemical dependency.

u S. DRUGINFO,1968 to date,ALCOHOL USE/ABUSE, 1968 to1979

Quarterly

DIALOG, SDC

NLM

Citations (and abstracts from 1975) topicsrelated to environment

Corresponds to printed ‘‘EnvironmentalAbstracts”

international—1971to date.

700 per month

Citations and abstracts of the literature on

epilepsy Covers basic sciences, sei-zures etiology, genetics, systemicchanges related to seizures, diagnosticaids; psychology, sociology, and epide-miology.


400 records perquarter

Excerpta Medica pubilcation, “EpilepsyAbstracts”

—Onllne avad-

ablllty(United States)

—

Sources of Coverage (earnestContent references dales available)

—.- . ..—.

Updating(approximate)

Data base/subject Producer

EXCERPTA MEDICA; EMBASEScience and technology Excerpta Medica


References to articles from over 3,500domestic and international journals Cor-responds to the 43 abstract journalsand 2 printed indexes that comprise theprinted “Excerpta Medica ‘‘ Approxi-mately 100,000 citations are added tothe data base each year that do notappear m the printed publication


5000 recordsweekly

DIALOG Citations and abstracts of worldwide bio-medical literature on medicine and areasof biological sciences related to med-icine, Including clinical practice,research, and economic and manage-ment issues

FOODS ADLIBRAScience and technology Kemp lnforma-

(food science) biblio- tion Services,graphic Inc

HEALTH INDEX INFOHealth Statistics (biblio- NCHS, Clearing-

graphic and referral) house onHealth Indexes

Citations and abstracts to the journal liter-ature on food technology nutrinional andtoxicological information

1974 to date 2,000 records permonth

DIALOG

NCHS Citations to unpublished literature relatingto measures of health status and qualityof life Journal articles, monographs,conference proceedings, and technicalreports

Reports of ongoing research, technicalreports, workshops, conference pro-ceedings, and symposia as well as theMEDLINE data base, current contents,and selected periodicals

300 per quarterinternational—1973to date

HEALTH (Health Planning and Administration)Science and technology NLM


NLM. BRS Topics relevant to the provision of healthcare services

3,000 Journals m MEDLINE, journals re-viewed for ‘‘Hospital Literature Index,other monographs, technical reportsForthcoming wiII include citations fromCATLINE, ‘‘Health Planning Series”published by NTIS



HISTLINEScience and technology NLM


Citations to articles, books. conferenceproceedings and other literature on thehistory of medicine

The MEDLINE data base and “CurrentCatalog ‘‘ Corresponds to NLM’s annual“Bibliography of the History ofMedicine ‘‘

International. somedata from 1964

QuarterlyNLM

DIALOGIPA (International Pharmaceutical Abstracts)Science and technology American Society

(pharmaceuticals) bib- of Hospitalliographlc Pharmacists

Citations and abstracts of the literature ondevelopment and use of drugs and toclinical, practical, theoretical, scientific,economic, and ethical aspects of pro-fessional pharmaceutical practice,

Corresponds to the printed “InternationalPharmaceutical Abstracts ‘‘


About 1,200 recordsevery 2 months

IRL LIFE SCIENCES COLLECTIONScience and technology Information

(life Sciences) biblio- Retrieval, Ltdgraphic (IRL)

Citations and abstracts to worldwide lifesciences literature

DIALOG

NLM

Corresponds to the 15 abstracting journalspublished by IRL


Monthly

MonthlyLADB (Laboratory Animal Data Bank)

Science and technology Battelle Colum-(life sciences) numeric bus Labor-

atories, undercontract to

NLM

Numeric data base system contains com-parative data on control animals used mbiomedical research

Data are collected from Government agen-cies, independent laboratories, pharma-ceutical manufacturers, universities,and animal producers

Uinted States

Online avail-Data base/ abilitysubject

Sources of Coverage (earliest UpdatingProducer (United States) Content references dates available) (approximate)

MEDLINE (MEDLARS on Line)Science and technology NLM


References to articles from 3,000 Journalspublished in the United States and 70other countries, chapters and articlesfrom selected monographs Authorabstracts (from 1975) are available forsome citations Corresponds to coverageof Printed ‘‘Index Medicus ‘‘

International, NLM,1966 to date (cur-rent year and twopreceeding yearsare searchableonline, earneryears are searchedoffline)

About 20,000records permonth

BRS, NLM,DIALOG

Citations and sometimes abstracts ofworldwide biomedical Iiterature onresearch, clinical practice, adminis-tration, policy issues, and health careservices

MEDOCScience and technology University of BRS

(life sciences) biblio- Utah, Spencergraphic S Eccles

Library

Citations to health-related U S Govern-ment documents medical and healthsciences, Food and Drug Administra-tion, mental health, biomedical engi-neering. Safety, child development,juvenile delinquency, welfare, Medicareand Medicaid, aging, rehabilitation:alcoholism, ethnic groups, basic sci-ences, nutrition, veterinary medicine,behavioral sciences

QuarterlyUnited States—1976to date

NIMH (National Institute of Mental Health)Science and technology National lnsti-

(life sciences) biblio- tute of Mentalgraphic Health, Na-

tional Clearing-house forMental HealthInformation(NCMHI)

NTIS (National Technical Information Service)Science and technology National

(general) includmg bio- Technicallogy (bibliographic) Information

Service

BRS, DIALOG international—1969to date

About 3,500 recordsper month

Citations and abstracts of the mentalhealth Iiterature, both the biomedicaland social aspects

Materials are from 1,000 journals, books,technical reports, workshops, confer-ence proceedings, and symposia

‘Government Reports Announcements andIndex (GRA&l), ” m part to the weekly“Abstract Newsletters “ Over 750,000citations

BRS, DIALOG,SDC

Citations, abstracts of technical reportsU S and non-U S Government-sponsored research Announcements ofcomputer-readable software data files.federally sponsored translations

United States—1964

2,600 records every2 weeks

POLLUTIONScience technology Cambridge Sci-

(environment) entificAbstracts

POPLINESocial sciences demo- Johns Hopkins

graphics University

POPULATION BIBLIOGRAPHYSocial sciences (demo- Unversity of

graphics and pop- Northulation) bibliographic Carolina, Car-

olina Pop-ulation Center

BRS, DIALOG Citations and abstracts to worldwide litera-ture on pollution research


1,400 records per 2months

Corresponds to printed “Pollution.

Citations, most with abstracts to literature.on demography, contraception, fertility


NLM

DIALOG

Worldwide literature Monthly

Literature on population research abortion,demography, family planning, fertilitypolicy and research methodologyMonographs, journals, technicalreports, government documents, confer-ences, unpublished reports Socioeco-nomic as opposed to biomedicalaspects

Worldwide literature Primarily UnitedStates–1966 todate

800 records every 2months

Appendix B—Compendium of Bibliographic Data Bases for Medical Technology Assessment ● 125

I

Appendix C.— Assessment of Medical Technology:Methodological Considerationsby Paul M. Wortman, Ph. D., University of Michigan

andLeonard Saxe, Ph. D., Boston University*

Abstract

This appendix is primarily concerned with method-ological issues underlying the research evidence usedto assess medical innovations. In particular, it ex-amines the process of research analysis in interpretingthe results from individual studies and the complemen-tary process of research synthesis in aggregating theresults from many studies. Both processes are impor-tant to medical technology assessment and require anunderstanding of their methodological limitations. Aconceptual framework is presented for determining thevalidity of the research evidence derived from variousmethodologies (e.g., clinical trials, consensus exercises)employed to assess medical technology.

Introduction

Medical technology has assumed an increasinglycentral role in the delivery and costs of health services.In order to assess the effectiveness of medical technol-ogies and increase the impact of Federal funds, Con-gress has undertaken a number of policy initiativesover the past few years (310). Through the 1976 Med-ical Device Amendments, it expanded the Food andDrug Administration’s (FDA’s) role in assessing med-ical products for safety and effectiveness. In 1978, itestablished the National Center for Health Care Tech-nology (NCHCT) with a mandate to conduct medicaltechnology assessments. Technology assessment hasbeen defined as a “comprehensive form of policy re-search that examines the . . . social consequences oftechnology” (7,269). Technology assessments mustconsider a wide range of outcomes of a technology,including safety, efficacy, cost effectiveness, and socialimpact. These outcomes are judged by considering var-ious forms of information about a technology. Thisinformation is typically derived from multiple studiesthat vary in their methodological adequacy and ap-propriateness to assess the technology.

——● Authors’ acknowIe&ments: The authors would Iike to thank Drs. Fred

Bryant, John McSweeny, William Yeaton, and the many anonymous re-viewers for their helpful comments on previous drafts. The authors also thankMarga Van Goethem and Jean Holther for the many hours of conscientioustyping.

The authors accept full responsibility for any remaining infelicities of proseand inaccuracies of thought. The views expressed in this paper are those ofthe authors and do not necessarily represent the views of OTA.

Despite their importance, methodological featuresof the research studies used to develop a technology

assessment are often given only minimal attention.Methods that have very different functions and ap-plicability, such as controlled clinical trials and con-sensus development, are often lumped together andviewed as alternatives to one another (see 266). Sim-ilarly, randomized clinical trials (RCTs) are often seenas a unitary method, although they represent a diverseset of procedures. In addition, while the usefulness ofa technology assessment rests on the ability to integrateresearch evidence, little attention is paid to researchsynthesis activities. There are no clear-cut standardsfor the quality of evidence that should be considerednor for the ways in which discrepant informationshould be consolidated. Recent Government confer-ences on methods for assessing medical technologyhave not changed the situation (e.g., 3,87).

The pressures for diffusion of medical innovationsrequire valid statements of efficacy, safety, and socialimpact (266). These, in turn, necessitate appropriatemethods for assessment. Proper research methods al-low one to state with confidence that observed effectsare actually due to the medical innovation—i.e., well-designed and carefully conducted evaluative studiesfor technology assessments will produce valid and re-liable results. As the remainder of this appendix willdemonstrate, the failure to conduct proper studiesoften results in serious criticism of both the validityof the research and the validity of the technologyassessments based on this research. A framework fordetermining validity that can be used to interpret theresults of individual studies and to synthesize the find-ings from many studies will be presented.

The purpose of this appendix is to review some prin-ciples for interpreting and integrating the results ofevaluative studies that underlie the assessment ofmedical technologies and to indicate their place in ageneral strategy for medical technology assessments.The remainder of this appendix is organized in threesections. The section immediately below discusses theinterpretation of individual evaluative studies ofmedical technology, It introduces validity concepts anddescribes the relationship between the design of re-search studies and the usefulness of the informationgenerated. Three broad categories of designs are dis-cussed: 1) RCTs; 2) controlled clinical trials lackingrandomization, also known as quasi-experiments (45);

127


and 3) uncontrolled studies known as nonexperimen-tal investigations or case studies. Some typical designsare described, and the problems they pose in inter-preting the evidence from studies assessing health caretechnology are presented. The second section belowexamines methods for synthesizing the results frommany studies. These include formal quantitative pro-cedures (e.g., meta-analysis) and group decisionmak-ing techniques (e.g., consensus conferences). Thevalidity problems in using these procedures are dis-cussed. The final section briefly describes a strategyfor integrating these assessment methods with the in-novation process.

Research Analysis: Interpreting theResults of Individual Studies

A thorough technology assessment is viewed as in-cluding 10 elements (269), one of the most importantelements in a technology assessment is the “evaluationof potential impacts, ” which encompasses “technicalfeasibility” (i.e., effectiveness), safety, ethics, andeconomic considerations. If technology assessments areto be useful, their evaluation component must be con-ducted in a systematic manner that employs acceptablescientific methods, especially research design (60,144).Proper research design is of utmost importance if theobserved changes in a patient population are to be cor-rectly attributed to the technology being assessedrather than to some extraneous factors. As the follow-ing discussion will show, it is often these other unre-lated factors that cloud the interpretation of techno-logical impact and undermine the validity of the tech-nology assessment.

Validity

Validity involves the careful analysis of research todetermine its adequacy or scientific soundness. Theanalysis of research requires an understanding of thestrengths and weaknesses of the methods used to gen-erate scientific evidence. The problems in determin-ing the validity of the findings in research studies havebeen of continual interest in medicine. Recently, themedical journal Lancet (170,171) carried a series deal-ing with research design issues in “assessing clinicaltrials. ” The articles discussed problems that can under-mine the validity of clinical trials, especially those deal-ing with medical innovations. A useful conceptualframework for examining issues of validity has beendeveloped by Cook and Campbell (68). These meth-odologists organize validity problems into four cate-gories: 1) internal validity, 2) statistical conclusionvalidity, 3) external validity, and 4) construct validi-

ty. These four categories provide a useful way of un-derstanding the implication of design issues for medicaltechnology assessment studies.

INTERNAL VALIDITY

Internal validity refers to whether the observed ef-fects of a medical innovation are truly due to thetechnology and not to some other factors. Internalvalidity, therefore, is the most important componentof validity. An important part of any technology as-sessment asks questions such as: Would patients haveimproved even if they did not receive the innovation?or, Do they really improve more with the innovativeprocedure than with the traditional approach? Anevaluative study that can adequately answer thesequestions is called an internally valid evaluation. Froma scientific perspective, internal validity involves theassignment of causality to the innovation for the ob-served benefits or risks.

A key issue in the internal validity of an assessmentis the “control” of factors extraneous to the innova-tion. When random assignment of patients to treat-ment and control groups fails or is not employed, anumber of plausible alternative explanations can beoffered. These so-called “threats to validity” include,among others, alternative hypotheses based on selec-tion and statistical regression (5,68). Selection or selec-tion bias occurs when patients are assigned to receivea treatment because of particular characteristics (e.g.,better prognosis), while statistical regression ariseswhen patients are chosen because of their extremevalue on a laboratory test or other measure relevantto the treatment. Many of these validity threats aredefined, described, and discussed in the following sec-tions. Their usefulness in interpreting the evidencefrom technology assessment studies will be demon-strated.

A properly conducted RCT is internally valid. Evenwhen studies are advertised as RCTs, however, oneshould carefully examine their methods or proceduresto determine if the randomization process was properlyconducted. A recent RCT published in the Journal ofthe American Medical Association by Hoehler, et al.(1.90), illustrates the problem. To assess the effec-tiveness of a rotational spinal manipulation for backpain, the authors report, 95 subjects were admitted tothe trial and were “randomly assigned to either the ex-perimental or the control group.” From this briefdescription of the randomization procedure, onewould expect that 45 to 50 subjects would be assignedto each condition. Instead, the initial table reveals thatthere were 56 in the experimental, spinal manipula-tion condition and 39 in the control group. This ap-

Appendix C—Assessment of Medical Technology: Methodical Considerations ● 129. —.

pears to be quite divergent from what a randomizedprocess would produce. (The probability of this dif-ference occurring is greater than the “1 in 20” levelassociated with chance. ) Although there may be goodreasons for this discrepancy besides chance, theauthors are mute on this point. One is left with thesuspicion that other factors (e.g., severity of pain) mayhave influenced patient assignment to conditions and.that these selection factors may be responsible for theobserved results. These factors would pose a threat tointernal validity due to differential patient selectioninto the two groups.

STATISTICAL CONCLUSION VALIDITY

There are many threats to the validity of a technol-ogy assessment study. Threats related to the analysisof the data are particularly important, and Cook andCampbell have called these threats to statistical con-clusion validity. This category of validity focuses onthe appropriateness of statistical tests and their abili-ty (or power) to determine whether or not observedeffects are due to chance. Many, otherwise internallyvalid, studies in health have used too few subjects (see153,171) to detect anything but the largest effects. Stat-isticians call this a Type II error—the acceptance ofa finding of no difference (in effectiveness) when it isfalse. It is possible that some useful technological in-novations have been discarded due to faulty statisticalprocedures. For example, a recent study (264) on theeffectiveness of timolol in reducing mortality after aheart attack noted that one reason most other studiesof these beta-blockers have found little or no effect wasthat they contained too few patients “to exclude thepossibility that a beneficial effect was being over-looked. ”

EXTERNAL VALIDITY

External validity concerns the generalizability of theobserved effects to other patient populations, settings,or conditions. That is, would the treatment be bene-ficial in other settings or are its effects specific to thepresent situation? The concept of external validity iscaptured in OTA’s definition of “efficacy” (266), thelikelihood of benefit under optimal circumstances to“individuals in a defined population . . . .“ The im-portance of external validity considerations can befound in an example drawn from the first National In-stitutes of Health (NIH) (96) consensus developmentconference on the efficacy of mammography in thedetection of breast cancer. The panel concluded thatthe technology was only beneficial for women over50 and might be harmful for others due to the risksof repeated exposure to radiation. These conclusions,

based largely on one study (341), indicated dramaticdifferences in effectiveness from one subpopulation ofwomen to another.

CONSTRUCT VALIDITY

The last type of validity deals with conceptualissues. It depends on the adequacy of the theory thatone has about what makes the innovation effective andthe adequacy of the measures of the observed effects(or variables) derived from the theory. The recently

concluded debate on the efficacy of radical mastec-tomy demonstrated the role of theory (147). Once itwas shown that cancer was disseminated through theblood stream, the basis of the Halsted radical surgery

was called into question. Construct validity also refersto improper measurement of outcomes as well as im-proper control of the technology. The latter can oftenbe confused or contaminated by other changes thatmay cause the observed effects.

Outcome Measures.—One of the major problemsin assessing medical technology is the absence of goodoutcome measures of the constructs considered impor-tant. For example, a researcher in behavioral medicineattending a conference on the social impact of cor-onary artery bypass graft (CABG) surgery noted,“There is no consensus about how you define andmeasure quality of life” (284). As a consequence,technology assessment studies often focus on a varie-ty of process variables (e.g., admissions, length of stay,etc. ) that may, or may not, be indicative of the de-livery of services and are not concerned with the over-all impact of a technology on patient health. Often,the absence of such observations can be traced to thelack of a specific, well-defined treatment procedure.

Even where there are outcome measures or endpoints, these may be “soft”* or subjective. Relief ofangina in CABG surgery is a case in point. Both pa-tients’ and physicians’ expectations concerning thebenefits of surgery (see 297) may influence judgmentsof relief. Such expectations are the rule for the tech-nological advances in modern medicine. As discussed,it is essential to eliminate from technology assessmentstudies the potential bias produced by these expecta-tions of efficacy (i.e., placebo effects). Good measuresof the impact of innovative treatments are needed,Often, the debate on the efficacy of medical technol-ogies swirls about very few objective outcome meas-ures (e.g., survival in CABG surgery, cesarean sectionin fetal monitoring).

‘Paul Meier, University of Chicago, personal communication, December1960.


Design Categories

Several types of designs have been used in studiesevaluating or assessing medical technologies. Thesegenerally fall into the three categories noted above:1) RCTs, 2) quasi-experiments or controlled trials, and3) uncontrolled case studies. These designs representthe principal methodological approaches to researchstudies of medical technology assessment and varywith respect to the validity of the evidence they pro-duce. In this section, the advantages and disadvantagesof each design category are examined with respect tovalidity. The major concern in this discussion is withthe choice of an appropriate control or comparisongroup and the effect this has on the validity of thefindings.

RANDOMIZED CLINICAL TRIALS

The “true” experiment or RCT is the preferred designfor producing unambiguous assessments of a medicaltechnology (see 60,187). The essential ingredient in anRCT is randomization: Patients or other experimen-tal units are randomly assigned to experimental (treat-ment) or control conditions. Although some (76) arguethat only posttreatment measurement of patients is re-quired in an RCT, most health researchers use bothpretreatment and posttreatment measures. This pro-vides a check on the initial or baseline equivalence ofthe groups and an accurate (or unbiased) estimate ofthe amount of change produced by the innovation.The basic question asked in a true experiment iswhether effects observed in the experimental (ortreated) group are also observed in the control (or un-treated) group. If the answer is essentially “no,” theeffects may be safely attributed to the technology.

RCTs are in reality a family of designs that vary insize and complexity. The number of treatment condi-tions can vary (e.g., dosage levels) as can the size ofthe population and the theoretical significance of thestudy. Small randomized trials are often performedearly in the development of a technology to demon-strate or test the efficacy of the treatment’s innovativeelements. Such studies typically involve only a singleinvestigator observing a few subjects—either animalsor humans—at a single site. At another level, large-scale, multicenter trials are often conducted to establishthe efficacy or safety of a developed technology. SuchRCTs are usually necessary to provide the appropriatenumber and type of patients to assess the technologyas quickly as possible. Moreover, the diversity of sitesand subjects can provide useful data on the externalvalidity of the innovation, These multicenter trials arenot immune from problems. They add difficulties inorganizational complexity and hence limit the re-

searcher’s ability to assess the technology under “idealconditions.” Indeed, much of the debate over theVeterans Administration’s (VA’s) multicenter RCT ofCABG surgery centered on such problems (254,255),There were wide differences both in types of patientsselected and in the operative mortality among the sites.Comer (66) discusses several strategies for successfullymaintaining the integrity of the randomization process(e.g., a centralized procedure with few implementers).

Blinding.—Other attributes of RCTs are also impor-tant to note. In order to reduce the bias in physicianand patient expectations, physicians and patientsshould both be unaware of or “blind” to the treatmentthe patient is receiving. This is called a “double-blindstudy” and is frequently used in assessing drugs. Insome cases, such control is not possible. For example,today it would be ethically impossible to give somepatients sham surgery to assess the efficacy of CABGsurgery or even to give them the much simpler inter-nal mammary artery ligation surgery that was provenineffective using such a control group (20). And evenif it were possible, only the patients would not knowwhich “treatment” they had received (i.e., the studywould be a single-blind study). As noted above, theinability to blind patients and physicians contributesto construct validity problems in interpreting thesurgery’s effect on the relief of angina.

When researchers and patients are not blind to thetreatment being delivered, it is possible that their ex-pectations can affect (or be confounded with) the out-comes. To avoid this, RCTs often use a placebo (i. e.,a procedure that appears identical to the innovationbut has no therapeutic benefit). A good example ofa relatively uncomplicated RCT employing a placebois provided by The Coronary Drug Project (72) thatassessed the effectiveness of a drug using this tech-nique. Placebo control is useful in establishing con-struct validity but is hard to employ with most non-drug innovations.

The Hoehler, et al. (190), study of spinal manipula-tion for relief of back pain (noted above) is an excep-tion in that it employed a placebo treatment for anassessment of a “technique. ” The control group pa-tients received a “soft-tissue massage of the lum-brosacral areas.” The authors assumed that this wasa valid placebo because their previous research showedthat “patients with no knowledge of spinal manipula-tion probably cannot distinguish that therapy fromsoft-tissue massage. ” They found no significant dif-ference at discharge as both groups were substantial-ly improved. In fact, the observed “dramatic” effectsof a number of innovations (e.g., gastric freezing andinternal mammary artery ligation) were later shownby well-designed RCTs to be due to a “placebo effect.”

Appendix C—Assessment of Medical Technology: Methodical Considerations . 131

Chalmers, et al. (54), maintain that research in-vestigators must also be blind to the randomizationprocess and to the interim results while the trial is inprogress. In the former situation, bias could affect pa-tient assignment; in the latter, it could also affect pa-tient withdrawals. In either case, the validity of thestudy is jeopardized.

One should be cautious, however, in assuming thatall RCTs are necessarily exemplary and immune tothreats to their validity. Research reports often con-ceal major flaws in the conduct of the RCT. The re-cent critique of the Anturane Reinfarction Trial byFDA (363) provides a cogent illustration of the prob-lems that can occur. The FDA audit found major errorsin coding outcomes and classifying patients that under-mine the credibility of the study. For example, errorsmade in the assignment of cause of death systematical-ly favored finding a benefit for sulfinpyrazone (An-turane) in reducing mortality following myocardial in-farction. Moreover, the classification scheme itself wasfound to be lacking in meaning (i.e., in construct va-lidity).

Statistical conclusion validity is also important inassessing RCTs. Most of the RCTs on coronary bypasssurgery have data analysis problems stemming fromserious attrition (or experimental mortality) in themedically treated condition, with patients crossingover into the surgery group. The various analytic ap-proaches for handling this problem have been inade-quate (400), including those based on initial patientassignment or “intention-to-treat” (289). A reexamina-tion of the crossover problem indicates that such in-appropriate statistical analyses may result in a TypeII error. If the worst medical cases are switching (asis indicated), then the mean outcome for their groupis being inflated. For example, a simple algebraic cal-culation indicates that the observed amount of cross-over (i. e., one-sixth) by the worst medical patientswould increase the mean by at least one-fifth of astandard deviation (i. e., 0.2 SD) or 20 percent. Sincesurvival data usually have a negatively skewed dis-tribution, the increase in the mean could be more.Thus, it is possible that the crossovers in the coronarybypass RCTs conceal a surgically significant differencelarger than 25 percent between the two groups.

Conclusions.—Although it is often true that large-scale randomized experiments are more expensive toconduct and require more planning than nonexperi-mental designs (see 222), that is not always the caseand they should not be rejected out of hand. Reviewsof health research practices indicate that the use of in-expensive nonrandomized designs often produces cost-ly errors, since faulty results can lead to incorrect con-clusions and inappropriate policy decisions (42,158,

335). Gastric freezing provides a classic example (143).Hundreds of devices were purchased by physiciansbased on the evidence from poorly designed, nonran-domized studies using few patients. In many cases, thegreater confidence in the results of an assessment thatan RCT permits greatly outweighs any difficulties inits implementation.

CONTROLLED CLINICAL TRIALS

Despite the advantages of randomized experiments,they are often difficult to implement in settings suchas hospital clinics and physicians’ offices. McKinlay2

has pointed out that RCTs are especially difficult toconduct for existing technologies that are already wide-ly diffused. Unfortunately, widespread diffusion hasbeen a frequent occurrence in assessing medical in-novations (see 253), In such situations, administratorsare usually reluctant to make the changes in policiesand procedures needed to conduct a randomized ex-periment. Another important obstacle to conductingRCTs that is common in health evaluations is the apriori conviction of medical personnel that specific pa-tients are best suited for the innovative treatment beingevaluated. In this case, staff will resist and possiblyeven subvert the randomization process. For example,the assessment of high-oxygen environments as a causeof retrolental fibroplasia in premature infants was im-peded by well-intentioned nurses (346). In one study,nurses raised the oxygen level for the experimentalgroup babies in the belief that the low-oxygen envi-ronments were harmful. In another study, it was nec-essary to implement the treatment only partially, untilevidence of the harmful effects of oxygen were moreapparent. The corruptive behaviors derived from pre-conceived attitudes pose an additional barrier to con-ducting an assessment study in an applied field setting.

Sometimes researchers find that conditions prohibitRCTs. This problem can occur for a variety of reasons:politics, as noted below in the Salk Vaccine Trial; cor-ruption of the design through attrition or other im-plementation problems; ethical prohibitions where pa-tients or physicians have been persuaded of the efficacyof a treatment (see 171); or cost considerations wherefunds for a long-term local study are unavailable,Sometimes, unfortunately, nonrandomized studies areconducted because of a naive belief in the ability ofstatistical techniques to correct for the biases intro-duced by selection.

When randomized experiments are not feasible, in-vestigators often use one of several quasi-experimental

‘J. B. McKinlay, “From ‘Promising Report’ to ‘Standard Procedure’: SevenStages in the Career of a Medical Innovation, ” Mlbank A4em. Fund Q. 59:374,1981.


designs (see 45). Quasi-experiments involve the use ofself-selection procedures in the assignment of patientsto either treatment or control conditions. These designsdo not permit the rigorous controls provided by RCTs.Even good quasi-experiments allow some competingexplanations for observed treatment effects. In par-ticular, two quasi-experimental designs—the cohortdesign and the time-series design—are commonly used.The validity of these designs is discussed below.

Cohort Design.—The cohort study or nonequivalentcontrol group design (NECGD) is the quasi-experimentthat results when random assignment of subjects to thetreatment and control conditions is not employed (seetable C-1). Because random assignment is not used,the “treatment” and “control” groups are “non-equivalent” and may differ in systematic ways. In thediscussion that follows, the term “comparison group”is used instead of “control group” when that situationobtains.

Roos, et al. (319), employed a cohort or NECGDto determine the effectiveness of tonsillectomy with orwithout adenoidectomy. Using claims and patient reg-istration data provided by the Manitoba Health Serv-ices Commission, the investigators were able to createtwo comparison groups to assess the impact of thesesurgeries on subsequent episodes of respiratory illness.The first, and larger, group consisted of operated andnonoperated persons under the age of 14 covered dur-ing a 3-year period, whose records indicated evidenceof tonsillar illness. For the experimental (operated)group there had to be data available for 1 year beforeand 1 year after their surgery. The records of the com-parison group had to indicate that they remained un-operated during this period.

A number of threats to the internal validity of thisstudy were examined by Roos, et al. (319). Since bothtreatment and comparison groups were similar in ageand sex, it was felt that maturation (i. e., changes inhealth with age) was not a threat to validity. More-over, by using concurrent controls, history (i.e., theeffect of temporal events such as new health practices)was also eliminated as a threat. However, ‘local’history (68) (i.e., dealing with familial or physician fac-

Table C-l . — N o n e q u i v a l e n t C o n t r o l G r o u p

o r C o h o r t D e s i g n

P r e t e s t P o s t t e s t

Treatment group . . . . . . . . . . . . 0 x o– R

Comparison group . . . . . . . . . . 0 00 = O b s e r v a t i o n o r m e a s u r e m e n tX = The application of the treatment or technology

– R = Absence of randomization

SOURCE: Adapted from T. D. Cook, and D. T. Campbell, Quasi-Experimentation:Design and Ana/ysis of Research in F/e/d Setting, 1979.

tors such as predisposition toward surgery) may havediffered among the two groups. To reduce the in-fluence of this potential threat, a second comparisongroup, composed of the siblings of those operated on,was also used.

Statistical conclusion validity (i.e., the correctnessof the data analysis) was also strengthened by usingtwo different analytic approaches as well as subsidiaryanalyses to eliminate effects due to statistical regres-sion. This latter threat was examined by stratifying thetwo groups according to the number of preoperativeepisodes of respiratory illness. If regression was caus-ing or influencing the results, there would be greaterchanges in the persons with the most preoperative epi-sodes (i.e., the extreme scores). In all cases, the resultswere the same—i.e., the operated group showed (sta-tistically significant) fewer postoperative cases ofrespiratory illness. Additional analyses to control forthe severity of the illness (by examining specific diag-nostic categories) yielded similar results.

The findings of this study are not meant to be de-finitive with respect to the effectiveness of tonsillec-tomy. This procedure has become the focus of somedebate with the advent of antibiotics (389), and RCTsare currently being conducted. However, the study isan instructive methodological example that illustratesthe assessment of actual practice or the “effectiveness”of an innovation.

Matching.—One common form of the cohort designinvolves examining naturally occurring patient popula-tions (as in the above example on tonsillectomy) todetermine whether they differ on important charac-teristics and then statistically adjusting for these dif-ferences. These procedures are referred to as matchingor retrospective matching. Two examples from the lit-erature on CABG surgery illustrate the approach andits problems.

McNeer and his associates (240) examined the datadrawn from 781 consecutive patients treated for cor-onary artery disease at Duke University Medical Cen-ter between 1969 and 1973. Of these patients, 402 weretreated medically and 379 had bypass surgery. Patientswere compared on 89 baseline variables. The authorsbelieved that “therapeutic decisions tend to be ran-dom.” They found the two groups to be “remarkablysimilar” and the results to be unchanged when in-dividual variables were corrected or statistically ad-justed for initial differences. However, as Ross (326)noted in his review of this study, there was a syste-matic pattern of differences among significant variablessuch that the “surgical cohort would have a betterprognosis irrespective of the form of therapy . . . .“For example, surgical patients had (statistically signifi-cant) more positive exercise tests, higher ejection frac-tion, and smaller heart size. The separate analyses and

Appendix C—Assessment of Medical Technology: Methodical Considerations ● 133—— ——.——.———.. .—

adjustments do not correct for this systematic bias, andthe conclusions are therefore rendered suspect. It ispossible that the outcomes merely reflected the preex-isting differences among the two groups.

One common form of this design involves the directmatching of patients receiving different therapies ortreatment. A recent example of this method is the studyconducted by Hammermeister, De Rouen, and Dodge(181) to assess the efficacy of CABG. Data from theSeattle Heart Watch angiography registry were usedto form 287 matched pairs of surgical and medical pa-tients. The patients were matched on seven variables(e.g., ejection fraction, arrhythmia, number of stenoticarteries). An analysis of the actuarial survival ratesresulted in a statistically significant finding indicatingdecreased mortality for patients treated surgically.When the data were analyzed by the amount of cor-onary disease (i.e., one-, two-, or three-vessel disease),improved survival due to surgery was detected onlyin the subgroup of 97 pairs with two-vessel disease.

Campbell and his associates (43,44) have graphicallydemonstrated the problems posed by a matching de-sign. In particular, they note that statistical regressionto the mean (usually abbreviated as “regression”) isa major threat to the internal validity of the resultsin such designs. The basic regression phenomenon canbe easily illustrated. Assuming that the two groups (inthis case, medical and surgical patients) differ on somerelevant unmeasured variables, it is possible that theymay be drawn from populations that differ in theirhealth status (see fig. C-1). Given that surgeons arelikely to select the best candidates for this procedure,the assumption seems warranted. The resulting match-ing procedure would then pair medical patients abovetheir group’s mean with surgical patients below theirgroup’s mean. Given the imperfect (or unreliable)measures used, the two groups will regress to their re-spective means due to this statistical artifact. Thereason is that the extreme scores of the matched pa-tients also include an extreme “score” on the “error

F i g u r e C - 1 .—An Example of Statistical RegressionResulting From Matching

Health status

component” or unreliable part of the measure repre-senting the many unmeasured variables. By chancealone, this unreliable component will be less extremethe next time the measure is taken. This can cause orcontribute to the finding of a statistically significantdifference as the two groups regress to different means.

The report on the Duke registry by McNeer, et al.(240), clearly fits this picture. The surgical patients inthat study were drawn from a “healthier” population,As Hammermeister, et al. (181), acknowledged, “thereare probably additional unmeasured or undescribedvariables of prognostic significance” in such data.Although these investigators are skeptical that this canalter the results, accumulated evidence indicates thatregression can produce spurious statistical findings.There are no foolproof statistical remedies to this prob-lem, but there are some recently developed analytictechniques that can partially adjust for measurementerror (206). These approaches may be useful in situa-tions where there are multiple measures of health statusand a conceptual model specifying the presumed rela-tionships among the variables involved. This techniquewould improve the statistical conclusion validity prob-lems associated with this design.

The problems in matching indicate the difficulty inovercoming differences resulting from selection in anonrandomized study design. The inability of statis-tical techniques to remove or adjust away these dif-ferences is graphically illustrated by the results of arecently reported study of the effects of drugs on cor-onary heart disease (73). Significant differences werefound in the 5-year mortality rate for adherers (15 per-cent) and nonadherers (28 percent) in the placebo con-trol group. A multivariate statistical analysis employ-ing 40 baseline variables was performed to adjust forthe differences in adherence. The adjusted mortalityrates were only 16.4 and 25.8 percent, respectively.The baseline characteristics accounted for only a smallamount of the initial difference. The authors noted thatthere must be unmeasured variables such as alcoholconsumption and personality characteristics that canaccount for this difference.

Retrospective Case-Control Study .—Perhaps themost difficult variant of the cohort design is found inthe field of epidemiology where retrospective case-control studies are frequently used to establish causalprocesses. This design consists of a group of peoplewith a disease (i.e., the cases) who are compared withanother group without the disease (i. e., the controls)to determine if they differ in their exposure to a pre-sumed causal agent. The major problem in this designis in the selection of the comparison (or control) group.This is the major threat to the validity of this familyof designs, because it is not possible to adjust for in-itial differences or to ensure that the treatment and

—— —

134 Strategies for Medical Technology Assessment

control groups are equivalent. The problem faced bythe epidemiologist-researcher is considerable, becausethe retrospective nature of the design implies no con-trol of the treatment.

A recent dispute over the role of estrogen therapyfor postmenopausal women as a cause of endometrialcancer illustrates the problems encountered in usingthis research design to assess technologies. The majorpoint of contention among researchers (191,196) con-cerned the appropriateness of the control group. Thetraditional approach in this area had been to selectwomen with other forms of gynecological cancer.Studies using this selection procedure have found aconsistently high association between endometrialcancer and estrogen use.

This method of selecting controls has been criticizedfor not correcting a bias among the target cases thatfavors the obtained result. Specifically, it has beenclaimed that estrogen is associated with uterine bleed-ing and that this condition normally leads to carefuloverrepresenting in the population of confirmed en-dometrial cancer patients. To counteract this poten-tial selection bias in choosing cases, Horwitz and Fein-stein (191) recommend the use of women being treatedfor uterine diseases by either dilation and curettage orhysterectomy. These women, they argue, will includemany referred because of vaginal bleeding. The useof such a population to create both treatment casesand controls will adjust for the bias resulting from in-creased surveillance and detection. Using both selec-tion procedures, Horwitz and Feinstein demonstrateda reduction in the likelihood of estrogen causing cancerfrom about 11 to a factor of about 2.

Critics of this alternative selection approach claimthat there is little or no detection bias since most casesof endometrial cancer are eventually diagnosed (196).They maintain that the alternative controls used byHorwitz and Feinstein are biased because they exhibitmany benign conditions not normally detected. More-over, estrogen may cause some of these other uterinediseases. Consequently, estrogen would be overrep-resented in the controls. As Cole (61) has stated, pa-tients undergoing the same diagnostic procedure as thecases can be “an inappropriate control group” sincethe same causal agent may be responsible for their ill-nesses.

In conclusion, it is important to note that the resultsgenerated by this design are essentially correlationaland do not lead to unequivocal causal inferences. Hor-witz and Feinstein3 located 17 medical “topics” wheremultiple case-control studies reached differing conclu-sions, Selection bias (i.e., “avoidance of constrained——. —-—

‘R. 1. Horwitz, and A. R. Feinstein, “Methodologic Standards and Con-tradictory Results in Case-Control Research, ” Amer. J. Med. 66:556, 1979.

controls”) was the most frequent methodologic prob-lem involved in the 17 disputes. The two approachesfor constructing a control group discussed above canbe viewed as providing a range of estimates for therelationship being examined. Because of the internalvalidity problems associated with this design, the useof different control groups to bracket the range ofrelative risk estimates should be considered. Thiswould also improve construct validity in those in-stances where the effects of the technology are not wellunderstood. Multiple case-control studies can also playa useful role in generating or confirming candidates(or potential causes) for unanticipated negative find-ings (e.g., toxic shock syndrome). In these instances,this epidemiologic approach is on the methodologicfrontline of medical technology assessment. Often,where the event is rare and the number of cases issmall, it is the only available method for making anassessment —e.g., of the role of aspirin in Reye’s Syn-drome. As with the Horwitz and Feinstein critiques,multiple studies using different controls were necessarybefore the association of aspirin to the disease was con-sidered established.

Historical Controls.—Innovations often diffuse sorapidly and completely that the potential for untreatedcontrols is greatly reduced or eliminated (see 253,266).In such a situation, researchers typically are forced touse a variant of the NECGD or cohort design that em-ploys historical control groups—i.e., patients treatedprior to the innovation. The important change in thedesign is a temporal one; patients in the comparisongroup are no longer treated concurrently with the ex-perimental group, Some problems with the historicalcontrol group design are illustrated by a recent articlediscussing the use of adjuvant chemotherapy for treat-ing osteogenic sarcoma (215).

Following the development of this treatment in theearly 1970’s, researchers began to experiment withways to improve its apparent effectiveness. One ap-proach was to treat patients with the drugs before theircancer had metastasized. Historical controls drawnfrom patient records dating from the 1960’s were usedin this research, and the results were provocative.Nearly half the patients treated lived 2 years withouta recurrence of the disease, compared to only 20 per-cent of patients in 1960. Unfortunately, the change intherapy from 1960 to 1970 was also accompanied byother changes in diagnosis, treatment, and patients.The use of the computed angiographic tomography(CAT) scanner in the 1970’s provided a much moresensitive test for detecting patients who did not havemetastasis. At the same time, surgeons began remov-ing metastasis in the lungs. At the Mayo Clinic, whereboth of these techniques were employed withoutchemotherapy, the survival rates equaled those of pa-

Appendix C—Assessment of Medical Technology: Methodical Considerations ● 135

tients treated with the drugs. In addition, the patientmix probably changed over time so that those withthe worst prognosis no longer constituted the majori-ty of those treated. These criticisms of the researchdesign and recent findings of a small controlled trialhave convinced the National Cancer Institute to sup-port a multicenter RCT to assess the efficacy of ad-juvant chemotherapy for osteogenic sarcoma.

This design demonstrates the importance of historyas a plausible rival hypothesis in interpreting researchresults. It also points out that innovations in medicaltechnology are not discrete events, but are often ac-companied by other changes in the organization anddelivery of medical practice that can affect constructvalidity. For example, surgeons note that there weremajor changes in the procedure for CABG surgery inthe mid-1970’s (e.g., cold-blood technique) and at-tribute to these changes responsibility for the declinein operative mortality. But, as this discussion hasshown, the decline could also be due to a correspond-ing change in patient mix as more low-risk patientswere convinced of the benefits of this innovation.

Wortman, Reichardt, and St. Pierre (402) have alsosuggested multiple measurements as a method ofstrengthening the basic NECGD. They recommend“double pretests” to estimate the change in baselinebehavior of subjects in the absence of any treatment(by allowing each person to serve as his or her own“control”). The double pretest considerably strengthensthe basic NECGD and should be employed wheneverthere is time to conduct two pretests prior to treatment.It is feasible when there is time to conduct two pretestsprior to treatment. It is feasible when there is somelag between patient application and acceptance in atreatment program, as sometimes occurs in oversub-scribed programs with long waiting lists. In situationswhere treatment is or must be made immediately avail-able, the use of a double pretest would probably notbe consistent with professional ethics.

Time-Series Design.—Often, data relevant to theassessment of a medical technology are collected at reg-ular intervals over an extended period. Data archivessuch as the one used in the Manitoba evaluation oftonsillectomy can provide periodic information on thefrequency and outcome of an innovation. If this is thecase, a time-series design can be used. This design con-sists of multiple observations prior to and subsequentto the initiation of a treatment or other type of in-tervention (see table C-3 top row). Analysis of a timeseries involves checking for changes in either the levelor slope of the series after the intervention.

Using this design to study the impact of a hospitalmerger on a number of cost indicators, Whittaker (39I)was able to demonstrate that, contrary to prior belief,

Table C-2.—Relationship of Methods and PolicyIssues to the Innovation Process

Level ofdevelopment Method/validity Policy issueNew Needs assessment Social need

Technical feasibility/construct validity

Emerging Research design/ Efficacy, safety,internal validity social impactCost-benefit analysisSecondary analysis/statistical conclusionvalidity

Existing Postmarketing Effectiveness, safetys u r v e i l l a n c e

D a t a s y n t h e s i se x t e r n a l v a l i d i t yN e e d s “ r e a s s e s s m e n t ”C o s t - e f f e c t i v e n e s sanalysis

SOURCE: National Institutes of Health, “Criteria for Identification of CandidateTechnologies for Consensus Development,” memo, Feb. 23, 1978

the cost-per-stay increased after the merger as did totalexpenses per patient day. Employing this quasi-exper-imental design and the sophisticated statistical analysisprocedures that have recently been developed for it,Whittaker demonstrated that a complex “organiza-tional” innovation had uniformly “unfavorable”impacts.

The interrupted time-series design may at first seemto be an attractive assessment methodology that coin-cides with a number of convincing innovations—e.g.,renal dialysis and the cardiac pacemaker represent suc-cessful medical technologies that appear to fit thisdesign. However, upon reflection, it is clear that otherinformation was available and used in the assessmentof these innovations—i.e., physicians knew what hap-pened to patients who did not receive the innovation—they invariably died. In such cases where the prognosisor time course of a disease is well documented, thetechnology evaluator has the benefit of a comparisonseries: a multiple time-series design (see table C-3). Thecomparison series helps to eliminate a number ofthreats to validity (e.g., history and maturation) andto reduce the plausibility of others. Time-series datacan provide useful and inexpensive monitoring of aninnovation and can even furnish evidence of causal ef-fects. Thus, they could be used in the postmarketingsurveillance of medical innovations.

Statistical analysis of time-series data is still a rari-ty in the assessment of medical technology. Although

Table C-3.—Multiple Time. Series Design


there have been a few exceptions such as Albritton’s4

study of the 1966 Federal program for measles im-munization, most researchers have been content to pre-sent their data in graphic form (see 342). The effectsof interventions are often dramatic, and visual judg-ments of statistical and medical significance may beadequate in many cases, However, statisticians (see330) have long warned that graphic representations ofdata can often be misleading. This warning has beenspecifically repeated with respect to time-series anal-ysis (175,202). These authors demonstrate that visualand statistical analysis of time-series data often leadto opposite conclusions. More detailed descriptions ofinterrupted time-series analysis and examples of ap-plications may be found in Glass, Willson, and Gott-man (167), and McCleary and Hay (237). Recent tu-torial articles such as the evaluation of a RegionalizedPerinatal Care program in North Carolina (161) pro-vide examples of the growing use of this design inassessing medical interventions.

Conclusions.—Although the cohort or NECGD isoften easier to use than an RCT, it suffers severalweaknesses in the form of threats to validity. The mostserious threat are selection differences. Because sub-jects are not randomly assigned to treatment and com-parison conditions, pretest or baseline differencesamong the groups are quite likely. These initial dif-ferences are then confounded with changes due totreatment observed at the posttest. A number of ana-lytic approaches have been suggested to deal with thisproblem. For example, the Cox regression techniquehas been used to analyze the survival data from cohortstudies (see 181). However, the analysis rests on theassumption of proportional hazards, that both groupshave the same risk of illness, and this is unlikely tobe true where the groups are nonequivalent. The re-sults from such nonrandomized experiments thus re-main extremely equivocal, particularly when the ex-perimental and comparison subjects differ significantlyin terms of important pretreatment characteristics.

Because there is no agreed upon analytical solutionto the problem of baseline selection differences, prob-ably the best that researchers can do currently is touse several different methods of analysis (70). If theresults from the various methods are congruent, eval-uators may state their conclusions with appropriatecaution. If different methods lead to different results,the situation is more confusing, and the technologyassessment will have to be more tentative in its con-clusions. Although this design may be appealing, itposes such severe problems in analysis (i.e., it hasdoubtful statistical conclusion validity) that extremecare is warranted (402).

4R. B. Albritton, “Cost-Benefits of Measles Eradication: Effects of a FederalIntervention, ” Policy Analysis 4:1, 1978.

In sum, do nonequivalent controls, particularly withmatched groups, provide useful information for a tech-nology assessment? Our general answer is that theydo not. The methodological problems resulting fromthese designs are often of such serious concern as toundermine the credibility of the findings. Only whenthe competing explanations or rival hypotheses (i.e.,important threats to validity) can be demonstrated tobe implausible or can be ruled out through other sub-sidiary data should such studies be considered seriouslyin a technology assessment. Statistical solutions, inparticular, should be viewed with skepticism despitethe impressive impenetrability of their algebra.

As this discussion has shown, there is justifiable con-cern about the credibility of the evidence produced bycontrolled nonrandomized studies. For many expertsand informed practitioners, the potential existence ofsuch methodological problems is sufficient to castdoubt on the findings. These concerns in determiningthe efficacy of medical innovations are not new.Meier’s (243) discussion of the Salk polio vaccine trialof 1954 indicates that the original quasi-experimentaldesign was upgraded to an RCT in certain States be-cause of these concerns. The original design called forsecond-grade children to receive the vaccine with first-and third-graders as comparison groups. The dif-ference in the size of the effect observed from thesetwo designs illustrates the problem with estimates ofefficacy obtained from cohort quasi-experiments. Thedifferences in the incidence of polio cases was 40 (per100,000) for the RCT, while it was only 27 for thealternating grade cohort quasi-experiment. The non-randomized results thus underestimated the efficacyof the vaccine by nearly 50 percent.

The multiple time-series quasi-experiment is muchstronger on internal validity than the cohort design.The research by Sherman (342) and his associates (343)demonstrates the potential utility of the time-seriesdesign in the evaluation of health programs at the in-dividual patient or program level of analysis. The time-series design is relatively unobtrusive; it rarely requiresthe changes in operating policy that a randomized ex-periment often does. In addition, time-series analysesmay be used to assess innovations that have alreadybeen in operation for a considerable length of time (see403). Since many observations are required to performtime-series analysis, it is most appropriate for agen-cies that collect data at regular intervals. Hospital andinsurance reimbursement or claims records would bemost useful, These could be used to provide informa-tion on cost, utilization, and health outcome.

UNCONTROLLED DESIGNS

The most common form of evaluative study formedical technology assessments employs a nonex-


perimental or uncontrolled design (401). These so-called case studies do not include any comparisongroups at all and usually report judgments by physi-cians about the extent to which each patient improved.

The first studies on gastric freezing (143) were almostall of this type. The results of the early studies werelargely on the self-reports of a few patients subjectedto the procedure. The basic case study may be de-scribed as a “posttest only” design, since only onemeasurement of the status of the subjects is used. Aslight elaboration of the “posttest only” case study isthe one-group pretest-posttest design, which includesa measure of the status of the patients prior to, as wellas after, treatment. The use of two assessments allowsthe researcher to estimate changes in the patients overthe course of treatment, as well as their final status.

The problem in using nonexperimental evidence toassess a medical innovation is illustrated by a newtechnology to facilitate the management of diabetes—home blood glucose monitoring. This is a fairly recentinnovation that shows promise of helping diabeticsmonitor their blood glucose levels more accuratelythan before, thereby allowing them to participate intheir treatment by changes in diet and exercise (362).To obtain information on blood glucose level, the pa-tient pricks a finger and applies the blood to a reagentor chemstrip. Glucose level can then be determinedeither directly or by reading a reflectance meter. Thistechnique is viewed as a replacement for urine testing,although it costs three to four times as much.

There is a great deal of enthusiasm about bloodglucose monitoring, and it is being introduced in anumber of diabetes outpatient clinics. However, thereis little evidence as to its effectiveness. Only ninestudies of this innovation could be found, and noneof them used a control group. Furthermore, many ofthe studies simultaneously introduced other regimenswith the blood glucose monitoring procedure, therebyraising construct validity questions. The other regi-mens introduced included exercise, group therapy, andspray injection of insulin. Any of these techniquescould have produced the beneficial effects reported.Moreover, most of the studies had very few patients;five had 17 or fewer subjects. Thus, selection of highlymotivated patients, for example, could produce over-ly optimistic results.

Although nonexperimental studies can provide in-formation concerning the technical feasibility of a newmedical technology, they are far from definitive. Theycan also provide useful “qualitative” information (286)concerning the acceptability of the technology to pa-tients (e.g., their willingness to draw repeated bloodsamples from their finger), factors affecting compliance(e.g., the interpretability of the chemstrip), and related

behavioral issues that may hamper its utility. How-ever, these studies should not be viewed as providingadequate information concerning efficacy and safety.Without a valid comparison group, it is not possibleto determine whether the benefits are due to patientself-selection or to other factors. Nor is it possible totell whether the innovation is superior to the urine test-ing methods now commonly used.

The major difficulty with nonexperimental designsis that they are subject to practically all of the threatsto internal validity described above. It is inappropriateto interpret such studies as indicating that observedchanges in patients are due to the innovation. Unfor-tunately, such interpretation is a common occurrence.Physicians, lacking training in research methods, canmistakenly perceive such preliminary pilot studies asbeing definitive. This can result in premature diffusion.The situation is often exacerbated by the exaggeratedclaims made by the developers of the technology.

Research Synthesis: Integrating theResults of Multiple Studies

The preceding section emphasized a set of principlesunderlying the design and interpretation of individualstudies to assess the effects of medical technologies.The assessment of medical technology, however, is aprocess that involves more than the consideration ofa specific research study (see 266,269). In order to con-duct a technology assessment, multiple sets of evi-dence, where available and relevant, must be con-sidered and synthesized. Although little attention hasbeen given to methods for synthesizing evidence aboutthe effects of technology (266,398), there do now ex-ist formal techniques to integrate the findings from dif-ferent studies and to develop generalizations based ontheir results. The methodological and conceptual issuesinvolved in the conduct of such analyses are consideredin the discussion below as part of the technology as-sessment process.

The synthesis of research data is often both con-troversial and complex. Controversy arises because theresults of studies about a particular technology may

vary and/or be interpreted differently by differentassessors. Synthesis is complex because medical tech-nologies may have different clinical outcomes depend-ing on who uses them or when they are used. Estab-lishing the efficacy and safety of a technology on thebasis of research evidence is typically a lengthy proc-ess. These assessments (i. e., safety, etc. ) dependbasically on the amount and quality of the researchevidence and the analyst’s ability to deal with theavailable information (i. e., ability to determine the

98- 1L+4 O - 82 - 10


validity of the evidence and to combine the varioustypes of information appropriately).

This section focuses on problems of synthesizing in-dependent research studies relevant to a technologyassessment and describes some formal procedures thatenable systematic integration of research results. In ad-dition to describing quantitative methods, it considersa number of methods for synthesizing information thatrely on group decisionmaking approaches to technol-ogy assessment. These methods are often used to re-solve controversies about research evidence and todevelop guidelines for employing particular medicaltechnologies.

Although information from a variety of sourcesmust be considered as part of a technology assessment(e.g., costs, social impact, etc.), research data concern-ing efficacy and safety form the central component.These outcomes are essential for determining social im-pact (see 295). Methods for synthesizing research in-formation and using it in decisionmaking are criticalto the outcome of an assessment. One purpose of thefollowing discussion is to suggest what types of dataare useful in the development of technology assess-ments and how they should be treated. The sectionis organized in four parts: 1) the application of thevalidity concepts introduced in the previous sectionto the synthesis of multiple research studies; 2) cur-rent approaches and problems to synthesizing researchevidence; 3) quantitative research synthesis and in-tegration methods; and 4) formal, group decisionmak-ing methods for synthesizing research evidence.

Validity

The previous section of this appendix emphasizedinference problems inherent in the interpretation of in-dividual studies of medical technology. From a meth-odological perspective, true experiments, particular-ly RCTs, reduce problems of equivocality of inference,as compared to other research strategies. A singleRCT, however, cannot resolve all questions about atechnology, and technology assessments cannot relysolely on their availability. If randomized studies arenot available, decisions will have to be made abouthow to treat the validity problems inherent in othertypes of research. For example, the reduction orelimination of threats to internal validity by an RCTdoes not automatically avoid problems due to low ex-ternal, construct, or statistical conclusion validity. Inparticular, external validity often must be establishedby examining evidence from multiple studies. Thus,validity considerations are as relevant to the problemsof aggregating and synthesizing the results of manystudies as they are to interpreting a single study. In

the following discussion, the validity frameworktended to indicate its use in integrating evidencemultiple studies.

INTERNAL VALIDITY

is ex-from

Internal validity problems are central to the syn-thesis of findings from multiple studies. Because of thelimited availability of RCTs, other evidence that canreduce the number of plausible alternative explanationsfor findings should be considered. However, the validi-ty of nonrandomized studies must be carefully ex-amined. If all, or most of the evidence about a par-ticular technology was generated through similar, andperhaps consistently flawed research designs, the ad-vantage of multiple sets of data may be lost. If theavailable literature includes a large number of studieswith low internal validity, then a simple aggregationof the results may yield a conclusion open to a varie-ty of alternative interpretations, especially whensimilar validity problems affect each study.

The existence of a few studies using randomized con-trol group designs, on the other hand, does not guar-antee high internal validity. Again, the alternative ex-planations must be considered to determine if they canbe eliminated. For example, the previous section notedthat the RCTs assessing the efficacy of CABG surgerywere consistently flawed by differential patient attri-tion (or experimental mortality). Only through theavailability of other evidence provided by additionalcontrol groups or improved analyses can the remain-ing threats to internal validity be eliminated (see 71).The principal problem in data aggregation is to iden-tify such validity problems and to develop a strategyfor aggregating the results of studies that differ in theirinternal validity.

In most cases, it is likely that experimental, quasi-experimental, and nonexperimental data will be avail-able. The problem, then, is the appropriate choice ofboth the evidence and the amount of emphasis itshould be given. One possibility (166) is to aggregatestudies that are high in internal validity separatelyfrom more “poorly controlled” ones.

Gilbert, McPeek, and Mosteller (159,160) provideevidence of the importance of this strategy for medicaltechnology assessment. These investigators comparedthe results of randomized and nonrandomized clinicaltrials of a series of medical innovations. They foundthat positive results were more likely to be obtainedby an uncontrolled research study than by an RCT.RCTs tended to yield much less favorable conclusionsabout effectiveness. For example, among 53 studies ofportacaval shunts, they found only 6 well-controlledtrials. Of these controlled trials, three were associated


with negative conclusions about the treatment andthree yielded moderately positive conclusions. Thiscompares with 32 uncontrolled studies, where 24 werevery positive, 7 were moderately positive, and 1 wasnegative. In general, Gilbert and colleagues found thatthe poorer the methodological quality (i.e., the lowerinternal validity), the more likely that a treatmentwould appear to be effective. The implication is thatconflicting claims surrounding medical innovationsmay merely reflect differences in the validity of theresearch designs.

STATISTICAL CONCLUSION VALIDITY

In evaluating a set of studies, it is necessary to con-sider whether serious threats to statistical conclusionvalidity exist in individual studies and whether thesethreats prevent developing conclusions about the tech-nology under study. Berk and Chalmers (26) examinedthe adequacy of the statistical analyses in a researchsynthesis of studies dealing with the cost effectivenessof ambulatory care (see below). They reviewed thosestudies reporting no difference in clinical outcome todetermine whether there was sufficient statisticalpower to detect a 25-percent difference (if one existed).Of the 23 randomized trials, 16 had sufficient power.Seven RCTs plus all the nonrandomized controlledtrials were classified as having “indeterminant clinicaloutcomes since selection bias may influence the out-come and obviate statistically valid comparisons whencontrols are not selected at random. ”

EXTERNAL VALIDITY

External validity is essential in assessing data frommultiple studies. The more widely a treatment has beentested, the easier it should be to establish the degreeto which results are generalizable to various popula-tions and settings. Studies high on internal validity,such as RCTs, may often yield differing and apparentlyconflicting results, because different patients, settings,or procedures are used. It is crucial that these studiesbe aggregated or stratified according to external validi-ty factors. The differences can often be dramatic. TheNIH consensus conference on CABG surgery (96)found the surgery effective for patients with left-maincoronary artery disease, but not for patients withsingle- or double-vessel disease. Similarly, radicalmastectomy, once the universally recommended pro-cedure for breast cancer, is no longer endorsed by ex-perts (96) for women whose disease is detected early(i.e., Stage I and II).

CONSTRUCT VALIDITY

Construct validity is also a serious concern in syn-thesizing the results from many studies. As Pillemerand Light (292) have noted, it may be that differencesacross studies are due to the use of treatments that haveonly been labeled similarly. For example, surgeons atthe consensus conference on CABG surgery, notedabove, dismissed most of the studies raising concernsabout the procedure’s safety (i.e., high operative mor-tality), because the studies were conducted before amajor change, called the “cold-blood technique, ” wasadopted in the mid-1970’s. The modification of a newtechnology can greatly affect its performance. Thesechanges, which are often unreported, mean that anevaluation is, in fact, assessing a family of technol-ogies, or a “moving target” (380). The rapid develop-ment that characterizes the early stages of techno-logical innovation can lead to errors in data aggrega-tion, because unreported, new components of thetreatment may have been incorporated into variousassessments. Often these labeling problems are moreinsidious in that other unnoticed technological changesco-occur with the innovation (e.g., improved diagnosisin osteogenic sarcoma).

Outcome Measures.-The major problem confront-ing those desiring more systematic methods for syn-thesizing the results from many research studies hasbeen the inability to combine many different measuresof efficacy and safety. One must ensure that com-parable measures of the appropriate construct havebeen employed. For example, Berk and Chalmers (26)report a systematic review of the efficacy of am-bulatory care as a cost containment measure to reduceinpatient expenditures. They found 134 relevant ar-ticles. Studies lacking either construct or statistical con-clusion validity were eliminated. Of the 109 actualstudies reported, 31 were eliminated because economicoutcomes were not discussed. In the remaining 78 in-vestigations, they found an appropriate measure ofcosts in only four studies! Thus, the improper measure-ment of a construct can significantly reduce the validityand usefulness of many studies for synthesis.

Problems With Traditional Synthesis Procedures

The traditional approach to synthesis is the literaturereview, Almost all technology assessments begin withsuch a research summary. Unfortunately, these re-views tend to be asystematic and subjective. Reviewersselect the evidence they believe to be most relevant andtypically organize their presentation around the dem-

.


onstration of a particular hypothesis. Although tech-nology assessments, such as those developed by theformer NCHCT, were based on summaries assembledand reviewed by several experts, there are still anumber of problems in relying on this approach toderive the implications of research and to resolvecontroversies.

METHODOLOGY

A central problem in literature reviews is how todeal with methodological issues. As noted above, therelatively few RCTs generally available (401) presentan important obstacle to the reviewer. Well-controlledresearch studies are probably the best way to produceunequivocal evidence, However, the weight of otherevidence may sometimes hinder their use.

This problem is illustrated by the controversy overelectronic fetal monitoring (EFM). Several recentreviews of the efficacy and cost effectiveness of EFM(e.g., 17,367) have indicated that significant risks areassociated with monitoring (in particular, an increasein cesarean section rate) and that it is not a beneficialdiagnostic tool for many of the patients with whomit is being used. Significantly, reviewers who are skep-tical of the use of EFM are primarily researchers;reviewers who are clinicians have come to a differentconclusion and have strongly supported the broad useof EFM (see, e.g., 189). Researchers appear to disre-gard much of the published literature because it con-sists of reports of uncontrolled research. Wortman(401) notes that 23 of the 24 poorly controlled studiessupporting EFM--there were no well-controlled studiessupporting it—employed historical controls. From theperspective of a research methodologist, the lack ofinternal validity indicates that there is no valid basisfor comparing monitored to unmonitored births. Clini-cians, in contrast, appear to be swayed by the largenumber of case studies that describe successful applica-tions/assessments of EFM. Since the rate of false pos-itives leading to cesarean section is relatively low, thisliterature probably is most consistent with their ownexperience.

Even when RCTs are available and the weight ofthe evidence is not as discrepant as in the EFM situa-tion, they may not fully answer questions about thetechnology. Tonsillectomy is a case in point. A sub-stantial literature exists about the safety and efficacyof tonsillectomies, and experimental, quasi-experi-mental, and nonexperimental research is available.Cochrane (60) reports three different clinical trials ontonsillectomies conducted in England during the 1960’s,but he contends that none of the trials resolved thepolicy controversy over the appropriate use of ton-sillectomy. According to Cochrane, the available

RCTs exhibit two methodological problems: 1) thetreatment was compared with no or inadequate med-ical treatment (instead of an alternate treatment); and2) the patients’ parents were not blind to the condi-tions of the experiment, so those whose children wereon the waiting list may have exaggerated their chil-dren’s symptoms.

Wennberg, Bunker, and Barnes (389) note that alarge-scale clinical trial is currently being conducted,but that the trial, in itself, will not resolve the con-troversy. This is because the current RCT does not in-clude a sample of the full population of children forwhom tonsillectomy is recommended. In essence, sev-eral internal and external validity problems preventthese available and pending RCTs from being unequiv-ocal tests.

TIMELINESS

Some (e.g., 34) believe that clinicians, over time,will be able to determine which medical treatments areuseful and which are not. The implication is that meth-odological considerations are not central. Others (e.g.,389) have suggested that this approach is ineffectiveand that many common medical practices are inade-quately evaluated and perhaps worthless or unsafe. Aquestion exists as to whether systematic reviews ofresearch evidence can influence medical practice.

Several studies have examined the use of researchby clinicians. Fineberg, Gabel, and Sosman (145), forexample, reviewed the use of scientific papers by anes-thesiologists. They found that there is a significant lagbetween research discoveries and their publication.Their view is that scientific papers affect actual prac-tice slowly. This would seem especially true of reviewsthat attempt not only to summarize but also to drawimplications from the literature. In part, this is becauseit takes considerable time for a published literature onany medical technology to develop. In several now-classic cases (e.g., gastric freezing), literature reviewswere only published years after a procedure was aban-doned because it was ineffective or unsafe (see 143,245).

In the gastric freezing example noted above, eithera more timely RCT (i.e., earlier) and/or more syste-matic attention to the available nonexperimental datamight have hastened the abandonment of the pro-cedure. These two evaluative processes are, in fact,related. Thus, if an RCT is not conducted during theinitial investigational stage of a developing technology,then it is even more important that systematic atten-tion be given to whatever data are generated, sincethese data may indicate whether or not an RCT isneeded. It should also be clear that an RCT may not“solve” the technology problem, and, in many cases,

Appendix C—Assessment of Medical Technology: Methodical Considerations . 141

research synthesis may stimulateditional data that are needed.

CONCLUSIONS

The evaluation of safety and

the generation of ad-

efficacy evidence tounderstand the effects of particular medical technol-ogies is complex. Complexity is related to the presenceof bias and methodological problems, such as the lackof appropriate control groups in research reports andliterature reviews. Research evidence may exist for anymedical technology, but it may be difficult or impossi-ble to synthesize these data without carefully consider-ing the validity of the individual studies. Developingresearch that can be used for a technology assessmentis obviously difficult, but is only the first step. Syn-thesis strategies are clearly necessary as part of thisprocess to deal effectively with the results of the manystudies bearing on a technology.

Systematic Procedures for Data Synthesis

A major implication of the previous discussion isthat the need for policy-relevant information oftenoutstrips the capabilities to provide it. The develop-ment of conclusive evidence about a technology atpresent seems to be a relatively slow process. Thisdiscovery process is probably better geared to the oc-currence of “breakthroughs,” those rare single studiesor programs of research that resolve a controversy,than to dealing with elaborate arrays of potentiallyconflicting or inconsistent information. Procedures areneeded that enable the accumulated insight gainedfrom research to be usable within the technology as-sessment process.

The problems and benefits in systematically organiz-ing and integrating research findings are discussedbelow. The procedures described, although not rep-resenting a panacea for all the problems identified, sug-gest how the process of research synthesis can be morerigorous. Some elementary qualitative procedures, aswell as sophisticated statistical techniques, for conduct-ing research synthesis are described below. The goalis to outline the range of systematic methods that maybe employed and to contrast them with more tradi-tional techniques.

VOTING METHOD

A simple form of synthesis has been called the votingmethod (226). This technique essentially involves or-ganizing a body of literature according to some pre-specified set of criteria. Usually, vote counting involvesthe selection of a particular sample of outcome studies,coding some aspects of their design and/or conceptual

framework, and classifying the observed outcome(s)according to whether they are favorable, neutral, orunfavorable (i.e., “taking a vote”). The Gilbert,McPeek, and Mosteller (160) study, referred to above,is an example of this type of synthesis. Sampling theliterature to determine the rate of successful innova-tion in anesthesia and surgery, their analysis indicatedthat about half the innovations assessed by RCTs weresuccessful when compared to a “standard” treatment.

A frequent use of the voting method is to demon-strate differences obtained by various methodologicalapproaches. For instance, Gifford and Feinstein (157)critiqued studies of anticoagulant therapy for acutemyocardial infarction (MI). They examined all avail-able literature on acute MI that reported control groupstudies of acute MI treatments. For each of 32 studieslocated, they coded the degree to which the diagnosticcriteria for MI were clear, whether randomized con-trol groups or other methodological criteria were em-ployed, and summarized their findings in several con-tingency tables. The results of the vote count indicatedthat anticoagulant therapy was superior to no treat-ment more often in reports that did not observe meth-odological standards than in those that did.

The strength of vote-counting analyses lies in: 1) theprecise identification of the populations of studies tobe sampled, and 2) the coding of substantive and meth-odological aspects of the study according to clearlydefined procedures. More widespread use of the tech-nique could probably aid in determining which specificpatient populations and/or conditions could be effec-tively treated by a medical technology. The votingmethod helps to avoid the problems of reviews thatonly selectively describe research or pay attention onlyto some aspects of the study. In addition, such analysesmay be particularly useful in identifying relationshipsbetween methods and outcomes.

Krol (216) cites three problems with the voting meth-od: sample size, effect size, and Simpson’s paradox.Large studies are likely to produce statistically moresignificant results than those with small numbers ofsubjects due to differences in statistical power. Thus,a finding of no difference among treatment and con-trol conditions will be correlated with small samplesize. In fact, Hedges and Olkin (186) have demon-strated that the voting method itself generally lacksstatistical power. A second problem is the all-or-nonenature of the method, Some findings may show small,marginal effects and others large ones, but they wouldcount the same. Consider the case where effect size iscorrelated with outcome—large, positive effects andsmall, negative ones. The voting method would yieldno difference when, in fact, there was an overall pos-itive effect. Simpson’s paradox is a more subtle statis-


tical point in which it is possible, under certain condi-tions, to reach different conclusions by aggregatingdata from each study rather than by counting eachstudy separately. The paradox results from unbalancedcell frequencies. Finally, Light and Smith (226) havenoted these and some additional problems with themethod. The most important is that vote counting mayoversimplify the results of studies and cause one tooverlook more subtle, but important, relationships(especially interactions among variables).

META-ANALYSIS

A second synthesis technique, called “meta-anal-ysis,” has been developed by Glass (164,165). Meta-analysis or the “analysis of analyses” is a rigorousstatistical approach to research synthesis. Meta-anal-ysis utilizes the actual results of studies and permitsthe determination, across a set of studies, of themagnitude-of-treatment impact. Most statistical anal-yses, as summarized in research reports, ignore boththe size and direction of effects and yield only a globalprobability of a “significant” difference. Meta-analysesare useful for assessing treatments where a large num-ber of studies exist and where findings across studiesseem to have great variability. As used by Glass, suchanalyses require that comparison groups be available(i.e., either randomized or quasi-experimental groups)and that the original research reports contain appro-priate statistical information such as the group meansand standard deviations. Glass (164) describes someindirect procedures for deriving the effect size from theinferential statistics reported in a study (i.e., t-test, F,etc. ).

Effect sizes (ES) are calculated by determining thedifference between the mean of the treatment group(T) and the mean of the comparison group (C), dividedby the standard deviation of the comparison group(SDc). Thus,

SDC

This procedure converts the average effect of each out-come measure into a common scale (i.e., standard de-viations) that can be compared to results of otherstudies. If a treatment has no effect, then there wouldbe a zero effect size; if the treatment is effective (i.e.,better than the current alternative), the effect size ispositive; and, if the treatment is inefficacious, the ef-fect size is negative. By making some assumptionsabout the skewness of experimental and control groupscores within each study, and the distribution of ef-fect sizes across a large number of studies (i.e., thatthey are normally distributed), effect sizes can be con-verted into percentile ranks and inferences can be madeabout the overall effects of a medical technology.

One of the best recent health technology examplesof a meta-analysis is Smith, Glass, and Miller’s review(354) of the outcome studies of psychotherapy treat-ments (see also, 353). Smith and colleagues searchedthe published literature, including abstracts, and in-cluded within their analysis all available control groupstudies of the effectiveness of any form of psychother-apy. Drug studies were analyzed separately, whilethose studies that did not involve the use of profes-sional therapists (operationally defined as psycholo-gists, psychiatrists, and social workers) were elim-inated from the analysis. The investigators coded anextensive number of variables for each study, includingmethodological criteria such as the nature of the pa-tient assignment to condition (e.g., random v. match-ing), experimental mortality, and other threats to in-ternal validity. Effect size scores were calculated foreach principal dependent measure. The analysts alsodeveloped a code for validity of the outcome measures.

Smith, Glass, and Miller’s (354) findings indicatedthat, on the average, the difference between scores ofthe groups receiving psychotherapy and scores of thecontrol groups was 0.85 standard deviation units. As-suming the normal distribution of effect size scores,this average standard score indicates that a typical per-son who receives psychotherapy is better off than 80percent of the people who do not. Smith and col-leagues also conducted a number of analyses to deter-mine whether the methodology of the study affectedresults and whether different therapies (or other fac-tors) were differentially efficacious. They found fewreliable methodological differences. It appeared thatoutcomes were not related to the use of randomizedcontrol groups. This finding should, however, be tem-pered by the knowledge that all of their sample studiesused comparison groups and were generally high ininternal validity. When this is not the case (i. e., wherequasi-experiments are included), then the outcome canvary with the methodology (i. e., research design).Wortman and Yeaton5 have shown this to be the casefor the studies on CABG.

There has been some criticism of Smith and Glass’(353) approach based on their “lumping together” ofa large number of what some consider incomparabletreatments and outcomes (e.g., see 137). The strengthof the effect size technique, however, is that it pro-vides a common metric that permits analysis of thedifferences (methodological and substantive). Smithand colleagues’ classification variables for each studywere fairly comprehensive and yielded a systematiccomparison of studies on the basis of their conceptualand methodological designs. What is problematic

‘P. M. Wortman, and W. H. Yeaton, “Synthesis of Results in ControlledTrials of Coronary Artery Bypass Graft Surgery” (Ann Arbor, Mich: Institutefor Social Research, 1982) (report submitted for publication).


about such meta-analysis, however, is that the findingsare heavily dependent on a number of decisions thatare not always made explicit. These include the studiesselected/rejected from the literature, variables in-eluded/excluded, and their construct validity. It is notpossible to ascertain biases resulting from Smith andcolleagues’ sampling decision nor whether only certaintypes of studies, therapies, or variables are assessedusing control group designs (273). A broader analysisof psychotherapy research might yield different con-clusions than those drawn by these investigators.

OTHER SYNTHESIS TECHNIQUES

A number of other methods exist for statisticallycombining the results of independent studies (see69,292,324). The effect size method described aboveactually incorporates several procedures. The most im-portant of these methods is the comparison of treat-ments to detect interactions between characteristics ofa study and outcome (i. e., external validity issues). Asnoted in the earlier discussion of the voting method,some of these procedures can be employed when ef-fect scores are not computed. Additional statisticalmethods combine probability values from variousstudies and adjust outcome scores according to therelevance of the data.

Rosenthal (324) describes a number of proceduresfor combining probabilities. These range from addingobserved probability (p) levels across different studiesto adding weighted standardized (z) scores. Thesemethods also include the testing of mean probabilityvalues. Essentially, using such procedures allows oneto indicate whether significant effects are obtainedacross a set of studies. The problem in using probabili-ty values is one of statistical conclusion validity. Thenumber of subjects per study influences the statisticalpower to detect whether significant overall differencesare present.

DuMouchel and Harris (131) discuss another inter-esting quantitative method for synthesizing the resultsof experiments done with human and animal species.This method, a sophisticated application of Bayes’theorem, provides estimates of carcinogenic risk fromvarious substances derived from the results of epide-miological studies.

IMPORTANCE OF SYSTEMATIC DATA SYNTHESIS

Earlier, it was noted that technology assessment isessentiality a synthesis process that involves the reviewand integration of research findings. There are a num-ber of specific benefits that result from employing for-mal procedures for data synthesis (see 292). The firstadvantage is that formal syntheses help to identify con-

tradictions in the literature by systematically organiz-ing studies according to specified classification factors.It becomes possible to segregate differential outcomesaccording to treatment characteristics and/or meth-odological approaches. The analysis of different find-ings when controlled and uncontrolled studies areemployed (see 160,400) is a good example of this aspectof meta-analysis.

A second benefit of meta-analysis has to do withthe use of effect size scores. Not only do such scoresprovide insight as to the worth of the treatment, asin the Smith, Glass, and Miller (354) psychotherapyexample, but they also provide a benchmark for laterresearch. Thus, for example, a meta-analysis con-ducted by Posavac (294) of 23 controlled studies ofpatient education programs found a 0.75 average ef-fect size. Posavac indicates that this should providea standard against which new patient education pro-grams can be assessed. If the effect sizes of new pro-grams are only 0.20 (and similar dependent measuresare employed), this would probably indicate that theprograms are not particularly effective, at least for theproblem or population for whom they were designed.

Another advantage of quantitative synthesis meth-ods are that they serve to control for certain statisticalconclusion validity problems (e.g., power) that somecommentators have reported as severe in the medicalliterature (e.g., 141,172,337). It can be assumed thatthe widespread use of meta-analysis and other quan-titative approaches to synthesis would improve statis-tical reporting practices by calling attention to differentinvestigators’ use of data. In addition, errors in anal-yses, such as the use of multiple independent inferen-tial tests without appropriate error rate control or in-correct inferences because of a lack of power, wouldbe compensated for by most meta-analytic procedures.Although errors in data collection and, perhaps, incomputation of means and standard deviations wouldnot be corrected by these synthesis methods, the sys-tematic analysis of multiple studies should render theeffect of such errors less consequential. The attentionto systematic considerations of the “weight” ofevidence across research studies should have a generalsalutary effect.

Finally, it should be noted that, although these pro-cedures seem most appropriate for evaluating moremature technologies that have accumulated a con-siderable body of research, they are often applicableto less developed technologies. In some cases, whereonly meager evidence is available from a small set ofstudies, it may be that a review of specific componentsfrom some other portion of the literature may suggestthe effectiveness of the new technology. Thus, physio-logical evidence may be considered with other clinical,


experimental data as in the case of radical mastectomy(see 147) noted earlier.

Group Decision Methods

Although the application of formal statistical pro-cedures for the integration of data from individualstudies should improve the ability to conduct technol-ogy assessments, the use of such methods does not en-tirely resolve policy controversies. Such analyses can-not go beyond the available data on a particular prob-lem, nor can they substitute for informed judgment.In the discussion below, some recently suggested pro-cedures for resolving conflicts across research studiesand for developing assessments of particular technol-ogies are described. These informal methods includea new approach to decisionmaking sponsored by NIH,referred to as consensus development, and a numberof other decisionmaking techniques (e.g., Delphi) thathave been employed in assessments of medical tech-nology.

NIH CONSENSUS DEVELOPMENT

In response to congressional pressure to assist in thetransfer of technology, NIH initiated its consensusdevelopment program in 1977 (310). Perry and Kal-berer (287) recently described the consensus develop-ment program at NIH. Its goal is to bring togethervarious concerned parties (e.g., physicians, consumers,bioethicists) in order to seek agreement or “consensus”on the safety, efficacy, and appropriate conditions foruse of various medical procedures. Judgments aboutthe technology under consideration are intended to bebased on the scientific evidence of its effectiveness aswell as on information about its social, ethical, eco-nomic, and legal impacts. The consensus developmentprocess is designed to produce a written recommen-dation, called a “consensus statement, ” that can be ac-cepted by clinicians and researchers. The statement issupposed to identify both what is known and notknown about the technology.

Topics for NIH consensus development are chosenbecause of their current or potential importance (e.g.,in terms of cost, number of patients affected). SinceSeptember 1977, NIH has held more than 30 consen-sus conferences at which the evidence and implicationsof a wide variety of technologies have been considered.Topics have ranged from bee sting kits to CABG sur-gery. The technologies include both emerging, as wellas currently used, technologies that either have notbeen carefully evaluated for safety and efficacy or arecontroversial. Recently, there has been a trend towardmore mature technologies (see next major section

below) for which there is more scientific evidence con-cerning effectiveness.

Over the past few years, the conferences have gen-erally followed a similar format. A panel of neutralexperts is selected by NIH to hear presentations by theleading medical researchers addressing a prespecifiedset of questions about the technology. The presenta-tions, usually summarizing the latest research findings,are made over a 2-day period during which both panel-ists and audience members discuss the research find-ings. On the evening of the second day, the panel issequestered to draft a statement responding to thequestions. Usually, they deliberate through the night,writing as many as four drafts of the consensus state-ment. In some rare cases, minority reports are devel-oped to indicate disagreement with the majority rec-ommendations. The next morning the statement is readto the audience for their comments and criticisms. Theconference concludes with a press conference. Thepanel then disperses with the final task of revising thestatement. The consensus statements are widely dis-seminated by NIH through direct mail to thousandsof organizations and individuals and by publicationin leading medical journals such as the New EnglandJournal of Medicine and the Journal of the AmericanMedical Association.

From a methodological perspective, two aspects ofthe consensus development process are of concern:1) its sensitivity to the limitations of the researchevidence, and 2) the extent to which a comprehensiveand systematic review of the research literature is con-sidered. There is little published evidence concerningthese issues. An examination of panelists participatingin previous consensus conferences (96,97,98) indicatesthat there has been no consistent policy to include amethodologist—either a biostatistician or epidemi-ologist. On few panels were such persons included.This means that in most cases there was no informedperson who could indicate the methodological limita-tions of a study. The problems to which meth-odological ignorance can lead have already beendescribed.

The consensus conference on CABG surgery was anexception (95). Two biostatisticians are listed asmembers of the panel, and their influence on the con-sensus statement is evident. The methodological limita-tions of the research literature with respect to a keyquestion are discussed at length (see 95). A numberof these methodological problems have been notedabove: attrition due to crossovers, use of historicalcontrols, statistical analyses of registry (i. e., quasi-experimental) studies, and the like.

Despite this indication of methodological detail,there is apparently no formal policy to provide syste-


matic reviews of the research literature. For example,the weight of the evidence on the efficacy of the cor-onary bypass procedure as presented in the publishedconsensus statement (95) was evidently derived fromtwo large, multicenter RCTs: the somewhat controver-sial VA study (255) and the ongoing European trial(136). Our examination of the literature revealed thatthere are at least 30 studies, of which 9 are RCTs.Given the emphasis on external validity issues (i.e.,identifying the patients for whom the surgery is ben-eficial), the limitation of the discussion to two studieswas clearly unwarranted.

This problem has occurred in other consensus con-ferences as well. In a recent letter to the Journal of theAmerican Association, Jones (203) noted that one ofthe conclusions from the conference on adjuvantchemotherapy of breast cancer was “based on in-complete information. ” He pointed out that the resultsof only five studies were presented. while there were“at least nine major studies” containing “convincingevidence” on the effectiveness of chemotherapy inpostmenopausal women. (The consensus statementclaimed effectiveness for only “a select group of breastcancer patients.”)

On the other hand, in most consensus conferences,the attention to these methodological concerns hasbeen reversed. Most consensus statements reveal lit-tle discussion of methodological issues and limitationsof the studies even where this might be appropriate.However, extensive background materials are oftenmade available to the panel. These included a com-puterized bibliography of the literature and reprintsof the articles.

The consensus conferences are coordinated by NIH’sOffice for Medical Applications of Research (OMAR).Although the topics are selected by the relevant in-stitutes, OMAR makes the final decision about thesuitability of the topic, panel composition, and theproposed format for a consensus conference. Over thepast 2 years under OMAR’s direction, the conferenceshave developed in a number of ways. The use of afixed format has already been noted. Other approachesinvolving adversary (i.e., nonneutral) panels and taskforces have been almost entirely abandoned. More-over, the questions that have been posed to the con-ferences have been addressed strictly to those issueson which there is enough factual evidence to reachagreement. This has resulted in the omission of con-troversial issues. For example, in the recently publishedstatement from the Reye’s Syndrome consensus con-ference (67) questions about the role of salicylates (i.e.,aspirin) were deliberately omitted because OMAR feltlittle was known about it (although the limitations ofthe studies establishing this association were briefly

discussed). An editorial on the coronary bypass con-sensus statement in the New England Journal ofMedicine (308) complained that it and other consen-sus statements “represent the lowest common denom-inator of a debate—the only points on which the ex-perts can wholeheartedly agree.” This reflects the cur-rent orientation of OMAR away from “state-of-the-art” conferences. One methodological consequence isthat gaps in knowledge and needs for further researchmay not be as readily identified.

FORMAL GROUP DECISION METHODS

In addition to the NIH consensus development proc-ess, a number of systematic procedures for develop-ing consensus based on behavioral science principles(see, e.g., 163) have been developed. The goal of theseprocedures is to aid groups composed of individualswith different information and perspectives to develop

group judgments that best take account of the posi-tions of the individual members. In the discussionbelow, two methods—Delphi and nominal group tech-nique (NGT)—are presented. These techniques il-lustrate the potential and limitations of these methodsfor technology assessment.

Delphi Technique. —Delphi (78) is probably theoldest structured model for involving groups in deci-sionmaking processes and has been used widely inhealth care. The Delphi technique uses a series of ques-tionnaries (or individual interviews), each followed byanonymous feedback summarizing all the participants’responses. Although Delphi was originally developedby the Rand Corp. to synthesize expert opinions onnational defense problems, it has been extended tomedical problems (232,246,250,318,336).

A unique feature of the Delphi technique is that per-sons selected to participate in the process generally

have no direct contact with one another. Instead, par-ticipants are provided with a summary of the ques-tionnaire responses, usually by mail. Personality orstatus variables, thus, have little chance to exert in-fluence on a member’s opinion, as they might in face-to-face meetings such as the NIH consensus develop-ment conferences, By using anonymous feedback, eachexpert has an equal chance of influencing other par-ticipants (41). The technique is also viewed as pro-viding a framework within which to approach theproblem in a focused manner. Finally, and perhapsmost importantly, the technique provides a limitedtime frame in which to achieve consensus (41,135).There are a fixed number of iterations, usually three,in the questionnaire feedback process.

Delphi has been used to estimate the probability ofan epidemic occurring. Information about morbidity


and mortality rates for both the total population anda high-risk population were sought in an investigationreported by Schoenbaum, McNeil, and Kavet (336).The investigators employed a modified version of theDelphi technique using two separate groups of par-ticipants. The first group consisted of five experts oninfluenza epidemiology and virology. Subsequentquestionnaires fed back anonymous responses of theparticipants to the previous questionnaire. The secondgroup consisted of 10 experts in immunization, infec-tious diseases, and preventive medicine. Their subse-quent questionnaires were accompanied by summariesof responses compiled from previous questionnaires.The iterative process was continued until medianestimates for each group varied by less than 10 per-cent from the previous questionnaire’s responses. Sinceresults of the Delphi process indicated that the proba-bility of a full-scale epidemic was minimal, subsequenteconomic analyses revealed that it would not be ben-eficial to attempt to vaccinate the total population.They concluded that efforts should be directed at im-munizing the high-risk population.

The Delphi technique has been criticized as beinglittle better than the “seat-of-the-pants” method cur-rently employed by policymakers, and as being amethod which bases “knowledge” on an informal setof opinions rather than on formal decision analysis(332). Others (10) maintain that it is as subject to thesame total error found in most predictions. The proc-ess is also time and group dependent, since the resultsare based on the information available to a specificgroup of experts at a specific point in time. It shouldbe repeated as data change with time. It also appearsless well-suited than face-to-face group meetings as aprocess for resolving minimally controversial issues(318) or for synthesizing the state of the art in a givenfield (163). Nonetheless, the technique’s relevance forgathering predictive information seems clear (77). TheDelphi technique may also have use in resolving highlycontroversial issues likely to be distorted when par-ticipants interact personally with one another.

Nominal Group Technique.—In another structuredgroup process, members engage in limited interaction.Typically, all participants may be seated at a commontable and asked to write their views on each of a num-ber of issues posed by the leader of the meeting. Eachview is recorded on a separate card, and talking is pro-hibited. The cards are collected, and their contents arelisted for all to see without any indication of who isthe author of each. The group then discusses theseitems, often choosing the ones that interest them most.Delbecq, Van de Ven, and Gustafson (82) call this the“nominal group” technique (NGT) because the in-dividuals at the table (at the outset) are a group in

name only. The (silent) presence of others whilewriting the cards creates social facilitation whichstimulates participants to do well. Subsequent discus-sion dwells on the ideas proposed without any likeli-hood of distraction by attitudes toward those who didthe proposing,

Thornell (368) has recently reported a study com-paring the Delphi technique with the NGT. Physicianswere randomly assigned to one of three Delphi or NGTpanels to develop procedures for handling four hypo-thetical emergency medical services cases. In order todetermine the reliability of the decisions, panelists werecontacted individually 6 months later and asked to castan anonymous vote on the procedures originally dis-cussed. The degree of consensus achieved was the samefor both techniques. The most striking finding, how-ever, concerned the reliability of decisions over time.There were “very extensive” changes in the NGT vote6 months later, suggesting that it is “a less than reliabletechnique for reaching a consensus. ” In conclusion,although the physicians reported that they liked theNGT much more than Delphi, group norms and pres-sures were developed with the NGT that produced un-stable or false consensual agreement.

Relationship of Assessment Methods toStages of Innovation

In considering various methodological approachesto medical technology assessment, there are tworelated issues that must be examined, The first is howto deal with the limited funds available for conduct-ing technology assessments. The second is when orwhere to intervene in the innovation process. In orderto allocate scarce methodological resources, it isnecessary to understand some essential properties oftechnological innovation in medicine.

There are many excellent examples of medical in-novations in both the private and public sectors (e.g.,143,252,259). A very recent one—the portable insulininfusion pump—illustrates a number of generic issuesin the innovation processes. The case study approachis limited (as was noted above); thus, this example ismeant only to be descriptive rather than definitive.

Although the discovery of insulin as a “cure” fordiabetes was a major breakthrough, subsequent ex-perience with treatment by subcutaneous injection hasrevealed that it does not eliminate morbidity or mor-tality. Currently, diabetes ranks third among majordiseases as a cause of death in the United States (309).Moreover, it is associated with a large number of crip-pling and debilitating conditions. For example, it is theleading cause of blindness. It also leads to myocardialinfarcts, strokes, and other serious conditions.

Appendix C—Assessment of Medical Technology: Methodical Considerations 147

Recently, there has been much discussion of and re-search on the possibility of using a portable insulin in-fusion pump to administer and control diabetes (290).Several investigators have demonstrated that suchdevices control not only blood glucose levels but othermetabolizes as well (291). Although the exact cause(s)of the various pathologies associated with diabetes arestill not understood, the results are nevertheless viewedas significant (133). However, these studies involve fewpatients—seven and eight, respectively—and shouldserve only as vivid case studies of the ability of theseportable devices to achieve rapid and “strict control”over abnormalities associated with diabetes.

Although the underlying processes of diabetic-re-lated diseases are unknown, it is believed thatmicrovascular injuries (i.e., diabetic microangiopathy)result from the inadequate control obtained by con-ventional methods, primarily injection. There is nowsome provocative evidence (133,290) that the strictcontrol obtained with the infusion pump can preventand perhaps reverse these complications. Thus, thereexists a physiological basis or “hypothesis” for thepotential efficacy of this device. Such a physiologicalexplanation is often essential for generating interest ina medical technology. When coupled with powerfuldemonstrations of potential efficacy such as werenoted above, a technology possesses the essential in-gredients for rapid diffusion.

Two other considerations also figure into the proc-ess. The first concerns the safety of the device; the sec-ond its availability. As noted above, diabetes is amajor threat to human life and well-being. Bunker,Hinckley, and McDermott (40) observed in their re-view of a number of surgical innovations that underthese conditions “efficacy is apt to be considered self--evident.” It also appears that safety is seen as nearlynegligible in such life-threatening situations. Wherethere is no alternative treatment and death is the like-ly outcome, patients and their physicians are moti-vated to try any promising innovation (382). Undersuch circumstances, innovations are likely to diffuseand diffuse rapidly. All that is required is sufficientavailability or supply of the device. The literature onthe infusion pump reveals that there are many manu-facturers. It can thus be predicted that this technologyis on the theshold of diffusion. Despite the many un-answered questions concerning the long-term effec-tiveness and acceptability of the pump, despite re-searchers’ claims that it is “an experimental procedurewhich is still far from being a safe treatment routine, ”and despite doubts about its effectiveness (350), theingredients for the rapid diffusion of this technologyare all in place.

Type of Technology

Throughout the preceding sections of this appendix,there has been an implicit assumption that the methodsdescribed are appropriate for all medical technologies.Is that assumption true? As shown in the precedingdiscussion, it applies to drugs and surgery, but whatabout devices, especially those involved in diagnosis?

OTA (266) has described five criteria for assessingdiagnostic technologies, one of which is impact on “pa-tient outcome. ” This criterion has been the emphasisof the methods described in this appendix. Thus, diag-nostic devices do not differ in their appropriatenessfor the methods for technology assessment discussedabove. They only differ in the number of other criteriathat can be used in their assessment (e.g., accuracy)and in the range of health outcomes they affect.

For example, two of the criteria OTA describes dealwith the quality of the information the device pro-vides. This involves established concepts and measuressuch as specificity and sensitivity of a diagnostic test(see 241). The other criteria deal with the organiza-tion and delivery of health services. These are impor-tant secondary impacts that should be considered forall technology assessments after the primary deter-mination of efficacy and safety have been made. Bantaand McNeil (15) provide an instructive example ofthese assessment criteria applied to the CAT scanner.They acknowledge that it is difficult to study healthoutcomes for this type of technology and also difficultto conduct randomized studies of it. As a consequence,secondary impacts involving nonrandomized studiesusing other criteria may be necessary in the short run.As previously noted in this appendix, such technologyassessments require extreme care and cautious inter-pretation.

In addition to diagnostic, preventive, and ther-apeutic technologies, OTA (269) considers “organiza-tional” innovations as a major category. Many innova-tions in health are primarily organizational in theirmedical function. For example, intensive care units(ICUs) represent a largely organizational change aimedat containing costs by centralizing patient care. Healthplanners and administrators, in particular, often regardICUs primarily as an organizational change and notas a well-defined treatment with specified impacts. AsRussell (331) notes, it has been “difficult to design aconvincing test of intensive care’s effectiveness. ” Theconfusion between organizational change and healthimpact has also characterized the movement towardProfessional Standards Review Organizations, healthsystems agencies, and many other major Federal healthinitiatives. There clearly is a need for planned innova-


tion where the rationale underlying change and its in-tended impact(s) are specified. The research designsdiscussed in this appendix are also applicable to theseorganizational innovations. However, much more at-tention needs to be given to the implementation proc-esses or operation, and to the integrity of the innova-tion (340). Thus, it would be important to determinewhether emergency medical services have been prop-erly installed before assessing their effectiveness.

A Preliminary Strategy for Assessment

Our discussion of innovation raises the question ofits relationship to the various methodologies describedin this appendix. A number of researchers, includingWilliamson (396) and McKinlay,’ have described mod-els or stages in the innovation process that can be usedto relate issues in validity and design to the develop-ment of a medical technology. According to a recentNIH conception similar to Williamson’s (see table C-2),a medical innovation goes through three stages of de-velopment. At the earliest level (i.e., “new”), there isthe perception of need such as a cure for a disease ora better way of diagnosing it and a preliminary assess-ment of the technical feasibility of the idea underly-ing the innovation (i.e., construct validity). The tech-nology then becomes a reality, usually in an early form(i.e., “emerging”) that can be assessed for its efficacy,safety, and social impact (e.g., quality of life). At thispoint, research design and validity issues (i.e., inter-nal and statistical conclusion validity) as well as costconsiderations are important. Once satisfactory evi-dence is obtained at this level, the innovation devel-ops to an “existing” level where the emphasis is on itsacceptability or external validity. Widespread diffu-sion of the innovation should occur at this point, andthe relevant policy issues concern the cost effectivenessof the technology and the continued observation orpostmarketing surveillance of the technology forunintended negative side-effects (388). Given the in-ability to predict the future impact of technologies—especially low-frequency, unanticipated negative side-effects such as toxic shock syndrome—continued sur-veillance using epidemiological (i.e., case-control) andrelated methods will be necessary.

The large number of potential technologies to assessand the pressures to develop and diffuse them quick-ly ensure that some stages of development will not bescrutinized with the appropriate methods. Many of theabove examples (e.g., CABG surgery, gastric freezing)illustrate this point. In fact, it is the overdiffusion of

.—‘J. B. McKinlay, “From ‘Promising Report’ to ‘Standard Procedure’: Seven

Stages in the Career of a Medical Innovation, ” A4ihank Mere. Fund Q. .s9:374,1981.

young technologies and their associated costs that haveled to the need for strategies to deal with the problemsof technology assessment. The model described above,coupled the methods presented, provides the basis forsuch a strategy. There remains a need to order the tech-nologies according to their priority for systematic,thorough assessment.

According to the model, technologies in the firststage of development do not need to be assessed. Sincemany, if not most, medical innovations will not pro-gress beyond this point, the burden of assessment willbe considerably reduced. Technologies maturing be-yond this level can be ordered by the potential benefitsand harm they pose. This ordering could be deter-mined simply by calculating the product of the benefitor risk the technology poses to either decreased or in-creased mortality multiplied by the amount of use en-visioned for the technology. For example, CABG sur-gery may pose a 4 percent risk of death for the 100,000patients operated on last year. This would result in4,000 deaths. Another decision rule could involve cost.Obviously, medical technologies could be ordered byboth of these rules. The choice among the various pos-sible ordering procedures is one that falls in the policydomain and is beyond the scope of this discussion.

Conclusions

A brief examination of innovation in medical tech-nology reveals that it is a dynamic, temporal processthat requires considerable flexibility in the meth-odology used. Different approaches are relevant at dif-ferent stages of technological development. Moreover,policy-relevant evidence may not be available whenneeded, either because of the pressures for diffusionor the low priority for assessment initially assigned tothe innovation. The Dalkon Shield, an intrauterinedevice, is a recent, unfortunate example of prematurediffusion. Furthermore, no matter how thorough theassessment of a medical technology, there is alwaysthe possibility that unanticipated negative side-effectswill be discovered at a later date when use of the tech-nology is more widespread (e.g., X-ray treatment forfacial acne). At such times, a decision to reexaminethe technology will have to be considered along withthe choice of an appropriate methodology for accom-plishing this. Such postdiffusion technology assess-ments are much more difficult to accomplish. Therecently initiated RCT to assess chemotherapy as atreatment for osteogenic carcinoma is an example ofthis process of surveillance and reassessment.

Given the scarcity of resources, it is unlikely thatthere will be much increase in the number of large-scaleRCTs. Most of these require Federal support, and con-siderable funds are already allocated for such technol-


ogy assessments. For example, Levy and Sondik (222)report that in 1976 about one-eighth of the total budgetfor NIH’s National Heart, Lung, and Blood Institute—over $50 million—was devoted to major clinical trials.However, for many innovations, small-scale, localRCTs are probably feasible. Unfortunately, these arenot often conducted. One can only speculate as to thereasons for this. Physicians often lack the meth-odological training to conduct such studies or the con-viction that single-site studies are useful. The implica-tion for medical technology assessment is that therewill be an increased reliance on studies using othermethods of evaluation unless some new policy initia-tive (see 307) is taken. As noted in this appendix, theseother evaluative designs are most vulnerable to chal-lenge and are often seriously flawed. Where suchquasi-experimental approaches are employed, replica-tion and “triangulation” (71)—the use of multiple linesof evidence to eliminate or reduce-the salient threatsto validity—should be encouraged.

When should one conduct a large-scale RCT? Levyand Sondik (222) describe a complex multiphase, mul-

tigroup decision process based on four broad decisioncriteria: knowledge, methodology, resources, andethics. Methodological considerations, involving

power, significance level, effect size, and the like, areused to estimate the number of subjects and the lengthof the study. These factors determine the cost of thestudy and hence its feasibility. In sum, Levy andSondik outline a complex group decision process thatprovides a type of cost-benefit analysis for conducting

an RCT. The emerging methodology of decision anal-ysis (386) would be useful in selecting medical innova-tions for such high-quality technology assessments.

In conclusion, the dynamic nature of medical inno-vation requires constant monitoring. This can be ac-complished either through postmarketing surveillance(as noted above) or by careful, systematic reviews ofthe accumulating literature dealing with the innova-tion. Thus, medical technology assessment must notbe viewed as a one-time event. As the model describedin table C-2 indicates, evaluative studies for technology

assessments should be considered at all stages of de-velopment, particularly during the second stage.

Appendix D.— Medical Technologies and Innovation

Introduction

In some respects, the innovation process for medicaltechnologies parallels that for other technologies. Al-though there are many variations, the basic processis as follows. An innovation is conceptualized by rec-ognizing both technical feasibility and potential de-mand. If a decision is made to pursue the innovativeidea, problem-solving activity follows, drawing fromavailable information and further research and devel-opment (R&D) activities. If a solution to the problemis found, it may be the one originally sought, or a solu-tion to a modification of the original problem. Thefinal stage before widespread utilization of an innova-tion is its introduction into the market.

It is at this point that the innovation process formedical care technologies differs from that for mostother technologies. Drugs must meet premarket ap-proval requirements for efficacy and safety. Medicaldevices, depending on their classification, must eithermeet general controls, adhere to performance stand-ards, or meet premarket approval requirements forefficacy and safety. New medical and surgical proce-dures, though not subject to the same regulatory re-quirements as drugs and devices, are increasingly sub-ject to more systematic applications of clinical testingto evaluate their efficacy and safety; and decisions topay for their use are also increasingly being subjectedto more systematic analyses by private and publichealth insurers.

Definitions of Innovation

The basic criterion for an innovation is “newness,”or “differing in significant ways” from previous prod-ucts or programs (213). In its most limited definition,an innovation is an invention that is regarded as novel,independent of its adoption or nonadoption (405). Butin other definitions, inventions are not considered in-novations unless the adopting system perceives themas such—i.e., innovation involves the process of con-ceptualizing a new idea, finding the solution to theproblem, and using a new item of economic or socialvalue (256).

These different concepts of innovation impinge onthe question of whether regulatory and reimbursementpolicies inhibit the innovation process. One might find,for example, that regulation reduces the number ofnew patents. Using the invention concept of innova-tion, one might then conclude that innovation has beenhindered. Patents, however, offer little insight into thevalue of inventions. Even innovations that haveachieved widespread use are not necessarily beneficial

(252). From that standpoint, inventions that do notshow social utility as well as economic worth are notinnovations. Thus, it could be argued that inventionsthat do not meet regulatory or reimbursement criteria(representing collective judgments on social utility aswell as economic worth) are not innovations.

Research on Innovation

There are four principal approaches of research forunderstanding the innovation process: 1) statisticalstudies, 2) contextual comparisons, 3) critical incidentstudies, and 4) case studies.

National level statistical studies concerning innova-tion might include the contribution to gross nationalproduct, rates of diffusion, effects of legislation, etc.Such studies suffer principally from a lack of differen-tiation. Innovations vary enormously in terms of com-plexity, radicalness, compatibility, etc., and organiza-tions and industries vary in size, technology, history,culture, etc. However, most statistical studies general-ize on the basis of an assumed homogeneity.

Contextual comparisons are made by selecting andcomparing organizations that are similar along severaldimensions (e.g., size, technology, product range) butdiffer in terms of success at innovating (or some similardimension). From contextual comparisons, it maybepossible to extract a list of factors common to the suc-cessful innovators but not to the others. Given largesamples, the regularity with which some factors ap-pear confirms their importance in the innovation proc-ess. Contextual comparisons cannot account for all thelocal sources of variation, however, and remain at ageneral and somewhat superficial level.

Critical incident studies deal with individual recol-lections about important stages in the development ofvarious innovations. The problem here is one of sub-jective emphasis and bias, rich in detail but not neces-sarily giving the whole story. Critical incident studiesalso tend to deal with major innovations only and toprovide little information about incremental changes,the total contributions of which may equal or exceedthe contribution of single radical innovations.

Case studies are a frequently used approach in medi-cal technology assessments. An attempt is made to getclose to the process for a long period of time from aninvolved but neutral viewpoint. Case studies representan attempt to understand the dynamics of a processwhich is naturally changing in character and contentall of the time, something very few of the other typesof studies consider. But case studies lack a developedmethodology, and, although they may be able to ac-

150

Appendix D—Medical Technologies and Innovation . 151

count for the behavior of one specific organization,there is no basis for generalizing beyond that to others.

The limitations of these four basic approaches are:1) they either provide general data which are limitedin applicability to specific circumstances; or 2) theygive a highly specific account of one organization orinvention, identifying most of the factors which influ-ence the innovation process but with no direct generalapplicability (27).

Since such studies attempt to place a rational, pre-dictive framework on creativity, it is not surprisingthat they provide an enormous amount of descriptivedetail but do little in the way of establishing cause-and-effect relationships. Innovation seems to be theresult of many interrelated factors and not of any par-ticular factor. Nevertheless, it is useful to review theavailable research findings to help gage what effectsregulatory programs and changing governmental reim-bursement policies can or might have on the inno-vation process.

The next section of this appendix summarizes whatis known about the factors that affect the innovationprocess for drugs, devices, and medical and surgicalprocedures. The second section discusses regulatorymechanisms and medical care reimbursement policiesand draws inferences concerning their possible effectson the innovation process.

Factors That Affect theInnovation Process

Characteristics of Successful Innovations

When successful innovations are examined for theirkey characteristics, certain recurring factors are com-monly found in all industrial areas. Their relative im-portance varies from industry to industry and even be-tween specific innovations in one industrial area, buttogether these factors provide a composite picture ofthe conditions under which the innovation processthrives.

Personnel of five types contribute to successful in-novations (313). “Idea-exploiters” (as opposed to “idea-havers”) not only think up new ideas but also do some-thing about them. “Entrepreneurs” (or “product cham-pions”) advocate and push for change and innovation.“Program managers” (or “business innovators”) han-dle the supportive functions of planning, scheduling,business, and finance related to the developmental ac-tivities of their technical colleagues. “Gatekeepers” (or“special communicators”) are the links who bring in-formation from outside sources, joining technical,market, and manufacturing sources of information to

the potential users of the information. Finally, “spon-sors” or (“coaches”) are senior people not carrying outthe research or advocating the innovation, but pro-viding junior people with the resources necessary tomove technological advances forward in the organiza-tion.

Motivating forces for the initiation of innovative ac-tivity are roughly divided into “technology-push” and“market-pull” theories. The former reflect the beliefthat pushing technology through basic research willeventually result in significant technological develop-ment. The latter reflect the belief that the market,through recognition of a need and creation of a de-mand for new products, is the dominant factor in pro-ducing successful innovations.

The general industrial literature supports the theorythat market-pull is the primary influence. From 60 to80 percent of important innovations across the indus-trial spectrum have been related to market demands(375). In a study from West Germany, 70 percent ofsuccessful innovations originated from market-pull and80 percent of failures began with technology-push(156). However, it is apparent from this literature thatit is not an either/or situation between technology-push and market-pull.

Comroe and Dripps (64) argue that in the area ofbiomedical technology, technology-push is a more im-portant factor. In studying the 10 most important clin-ical advances against cardiovascular and pulmonarydiseases from 1945 to 1975, these investigators re-viewed 529 publications considered to be the key re-search articles leading to these advances. They con-cluded that 41 percent of the key articles “reportedwork that, at the time it was done, had no relationwhatever to the disease that it later helped to prevent,diagnose, treat, or alleviate.”

As for sources of effective technical solutions: “Inmost industries, no single firm commands a majorityof the resources available for research, nor can anyone firm respond to more than a portion of the needsor problems requiring original solution. It is not sur-prising, therefore, to find that most of the ideas suc-cessfully developed and implemented by any firmcame from outside that firm” (375). Moreover, the pre-dominant route of information is personal experienceand contacts, not the scientific literature.

An effective technical solution may be an originalinnovation or one adopted or adapted for a particularproblem. About 20 to 30 percent of significant innova-tions are adopted or adapted ones (219,256) and, asmight be expected, a new technology has a greater pro-pensity to be adopted or adapted for a new use whenit has passed through the initial and developmentalstages into the late maturity stage (376).


Finally, the user or the manufacturer may be thesource of the solution, and studies have shown thatin many industries (e.g., computers, specializedmachinery, scientific equipment), a user came up withthe solution, which was then adopted and turned intoa product by the manufacturer. Roberts (313) believesthat in the medical devices industry, the manufacturer’srole is primarily one of adoption and broad-based dis-tribution.

Channels for exploitation is the stage that precedeswidespread diffusion. In medicine, the typical mecha-nism is the clinical trial, and the evidence concerningwhether clinical trials function effectively to transferresearch results into clinical practice is conflicting (223,404). In most industrial fields, the nonprofit sector con-tributes infrequently to innovation. In biomedical in-novation, however, universities, medical schools, andhospitals are crucial. Yet few linkages exist in thebiomedical area between academia and industry to putinnovations into widespread use through commercialmarketing. Some linkages may result from a recentchange in the patent laws (Public Law 96-517) toenhance commercial exploitation of inventions devel-oped with Federal assistance. And in genetic engineer-ing, universities, medical schools, and hospitals arenow forming business relationships with industry, re-ceiving substantial amounts of research funds in ex-change for exclusive licenses to market the anticipatedinnovations.

In the liecycle of a technology, major technolog-ical changes occur in the early stages, but incrementaltechnological changes usually dominate the later stages(376). In other words, product innovation dominatesin the early stage, with little change in manufacturingprocess; but as the technology progresses, there is arapid decline in product emphasis and dramatic in-crease in process orientation, Finally, small companiescontribute most to innovation in the early stages ofa technological field, but large companies dominateby the time the field matures. Roberts (313) has ob-served this pattern in genetic engineering, a new fieldwhere small companies are the dominant contributors.

Innovation Process for Drugs and Devices

Although there is little information specifically re-lated to economic factors that affect innovation in thedrug and medical devices industries, the following gen-eral observations are probably applicable. Innovationis one way to compete in the market and is part ofsome companies’ overall strategy. Basic questions arethe extent to which a company pursues long-term v.short-term strategies, and the extent to which com-petitive success over the long run depends on a com-pany’s commitment to technological superiority.

Mansfield, et al. (235), define three probabilities forassessing the importance of different factors at differentstages of the innovation process: 1) the probability ofsuccessfully completing the technical problem-solvingstage; 2) the probability of successfully completing thecommercialization stage, given that the technical prob-lem-solving stage has been completed; 3) the probabili-ty of economic success, given commercialization. (Eco-nomic success means that the project will yield a rateof return which is equal to or in excess of that availablefrom alternative investments.) The product of thesethree probabilities is the probability that a projectwhich is initiated will be an economic success.

The aforementioned probabilities are affected byboth external and internal factors. Externally, a highrate of inflation means high interest rates. These, inturn, make it more expensive to raise capital for long-term investments, and produce large fluctuations inprices, thereby garbling the relative price signals whichproducers use to determine the kinds of productionprocesses that would minimize future costs. These un-certainties might turn corporate strategy toward ashort-term focus.

Corporate strategists also have to choose betweenlong-term technological breakthroughs and short-term,quick-payback product and process improvements. Amajor innovation of great technical novelty may nothave a well-defined market potential. In contrast, amodest product improvement may have a highly pre-dictable market. The major innovation may have amuch greater profit potential, but the risk of failureis also much greater. As mentioned above, one studyfound that 70 percent of successful innovations origi-nated from market-pull and 80 percent of failuresbegan with technology-push (156). Projects that origi-nate with R&D personnel are more likely to be tech-nically challenging than projects that originate withmarketing or other company personnel. They alsohave a lower probability of successful commercializa-tion, because R&D personnel are likely to haveless understanding of market potential. But once pastthe commercialization stage, projects originating withR&D personnel have a higher probability of economicsuccess (235), presumably because they are the mostlikely to have the combination of technical and eco-nomic factors necessary for ultimate success in themarket. Whether the greater probability of economicsuccess can offset the lower probabilities for technicalcompletion and commercialization is a matter of judg-ment. Many projects are not initiated or carriedthrough to completion, because they are judged eitherto have insufficient market potential or to have risksthat are too great.

There is evidence that corporate strategy in theUnited States has turned increasingly to a short-term

.

Appendix D—Medical Technologies and Innovation ● 153

focus, and opinions have been expressed that this isthe primary source of our current problems concern-ing innovation, declining productivity growth, andbalance of trade with other countries. Recent invest-ments have been skewed toward equipment and rela-tively short-term projects and away from structuresand relatively long-term investments (231), and an in-creasing portion of industrial R&D is directed towardrelatively short-term developmental work and less to-ward long-term fundamental research (257). In thisvein, some critics have accused corporate managersof relying too much on near-term market considera-tions in selecting R&D projects. These critics caustical-ly recall that “the initial market estimate for comput-ers in 1945 projected total worldwide sales of only 10units” (183).

These critics also contend that current managementpractices in the United States lead to focusing on short-term, low-risk projects. For example, the decentraliza-tion of organizational structures requires a greater de-pendence on short-term financial measurements, suchas return on investments, for evaluating the perform-ance of individual managers and groups. In addition,compensation plans for company executives rewardshortsighted behavior. h-i a survey of 174 companies,79 percent rewarded executives for short-term per-formance, and only 42 percent offered “long-term” in-centives, which were defined as anything over 1 year.In another survey, year-end bonuses were larger thanlong-term incentive awards; while bonuses amountedto about 50 percent of salary, the median long-termaward was only 34 percent of salary (298).

Finally, there is an increasing proportion of corpo-rate presidents with legal or accounting as opposed totechnical backgrounds. Critics maintain that this trendreflects a shallow concept of the professional manageras “an individual having no special expertise in anyparticular industry or technology who nevertheless canstep into an unfamiliar company and run it successfullythrough strict application of financial controls, port-folio concepts, and a market-driven strategy” (183).Such critics contend that although technological issuesmust be an integral part of broader strategic issues,they cannot be handled by the same methods appliedto finance and marketing.

Table D-1 .—Concentration of Innovational

Regardless of the industrial sector, most small manu-facturers do not engage in formal R&D. For firms un-dertaking R&D, innovational effort tends to increasemore than proportionately with firm size up to somepoint that varies by industrial sector. Innovations pro-duced mainly by large firms are typically those in capi-tal-intensive industries. The exceptions are in aero-space, shipbuilding, and pharmaceuticals, where capi-tal intensity is low but development costs for newproducts are very high. These findings are illustratedin the United Kingdom. Between 1945 and 1970, smallmanufacturers produced none of the 44 innovationsproduced by all U.K. pharmaceutical firms, but smallmanufacturers’ share of net pharmaceutical output in1963 was 12 percent (151,327).

The U.S. drug industry is also characterized by highand rising development costs for new products and astrong shift toward greater concentration of new prod-ucts in the very largest of the approximately 600 phar-maceutical firms. Since the late 1950’s, the number offirms producing a new chemical entity has declined,and the development of new chemical entities has beenincreasingly concentrated in the top four and eightlargest firms (see table D-1). In other words, innovativeoutputs have been concentrated in the 20 largest of the600 drug firms, and most of this concentration isamong the top four to eight innovators.

While the four largest firms’ share of innovative out-put remained stable from the late 1950’s through theearly 1960’s, then accelerated sharply, their share oftotal prescription drug sales remained fairly constant(see table D-2). Taken together, these findings indicatethat the increasing concentration of new chemical en-tity output in fewer firms has accrued to large firms,mostly at the expense of the smaller firms, in the top20 innovators. But the four largest firms, despite a neardoubling of their share of innovative output, have hadessentially the same share of total prescription drugsales during this period. Most of the large drug firmsare dependent on a few drugs for much of their in-come. For example, the three leading products of thecompanies listed in table D-3 accounted for 22 to 84percent of their total U.S. pharmaceutical sales in 1979.

These observations probably reflect the followingscenario: The vast majority of the 600 U.S. drug firms

Output in the U.S. Drug Industry, 1957-71

Total number of new Number of firms Innovational output of concentration ratios

Period chemical entities (NCEs) h a v i n g a n N C E 4 - f i r m 8 - f i r m 2 0 - f i r m

1957-61 . . . . . . . . . . . . . . . . . . . . 233 51 0.462 0.712 0.9311962 -66. . . . . . . . . . . . . . . . . . . . 93 34 0.546 0.789 0.9761967 -71 . . . . . . . . . . . . . . . . . . . . 76 23 0.610 0.815 0.978SOURCE H. G Grabowski, et al , “The Effects of Regulatory Policy on the Incentwes To Innovate” An International Comparatwe Analysis, ” 1976

98-144 9 - 82 - 11


Table D-2.—Percentage of Innovational Output andTotal Sales Accounted for by the Four Largest

U.S. Drug Firms, 1957-71

Four largest firms’ Four largest firms’share of share of total

Period innovational output drug sales

1957-61 ....., 24.0 26.51962-66 . . . . . . 25.0 24.01967-71 . . . . . . 48.7 25.1S O U R C E : H. G . G r a b o w s k i , e t a l . , “ T h e E f f e c t s o f R e g u l a t o r y P o l i c y o n t h e

Incent ives To Innovate: An Internat ional Comparat ive Analysis,” 1976.

Table D-3.—Percentage Total U.S. PharmaceuticalSales Accounted for by Three Leading Products,

Selected Corporations, Selected Yearsa

1970 1975 1979

Abbot. . . . . . . . . . . . . . . . . . .American Home Products:

Ayerst . . . . . . . . . . . . . . . .Wyeth . . . . . . . . . . . . . . . .

Bristol-Meyers:Bristol . . . . . . . . . . . . . . . .Mead-Johnson . . . .

Burroughs-Wellcome. . . . . .Ciba. . . . . . . . . . . . . . . . . . . .Lederle . . . . . . . . . . . . . . . . .Lilly . . . . . . . . . . . . . . . . . . . .Merck . . . . . . . . . . . . . . . . . .Pfizer. . . . . . . . . . . . . . . . . . .Robins. . . . . . . . . . . . . . . . . .Roche . . . . . . . . . . . . . . . . . .Searle . . . . . . . . . . . . . . . . . .Shering . . . . . . . . . . . . . . . . .Smith Kline . . . . . . . . . . . . . .Squibb. . . . . . . . . . . . . . . . . .Upjohn. . . . . . . . . . . . . . . . . .Warner-Lambert:

Warner. . . . . . . . .Parke-Davis. . . . . . . .

NA = not availableaU S sales.

36

6437

6940NA474846355243804542442847

5325

33

7444

463856NA3160446545804948423150

NA27

28

8443

283751553243446546704440662356

NA22

SOURCE Charles River Associates, Inc , “The Effects of PatentTerm Restora.tlon on the Pharmaceutical Industry, ” a report to OTA, May 4, 1981

are small manufacturers producing primarily genericdrugs for limited markets, but also other patenteddrugs. After patents expire, generics erode some of themarket captured by large innovator drug firms andthese firms regain their share of total sales through theintroduction of new drugs.

The U.S. medical devices industry has experiencedsubstantial growth since World War II. Industry salesin 1977 were $8.1 billion—five times the amount in1958 (corrected for inflation). Growth has been pre-dominantly in the number of firms rather than in theirsize. The U.S. medical devices industry is composedof several thousand firms—many specialized smallfirms which together have a small share of the market

and a few large firms with high market shares. Thereare high entry and exit rates in the industry, mostlyamong small firms (8). Profitability is higher than aver-age in the economy.

Dominance by large companies suggests the pres-ence of economies of scale, while the persistence ofmany small companies suggests that economies of scaledo not apply to specialized areas. Possibly, however,the large firms really represent the industry; i.e., ratherthan representing the differentiation of the industryinto small and large functions, the large number ofsmall firms may represent a high-birth, high-mortality,and high-turnover sector of the industry (122). ArthurYoung & Co.’s survey of the industry, for example,did not differentiate between bankruptcy and acquisi-tion in its observation of the high-turnover rates forsmall firms. However, D’Arbeloff (79) comments thathigh-turnover rates may reflect a high-risk, high-profitatmosphere for small firms.

In general, small firms fill a special niche in the med-ical devices market, and their growth into larger firmsis hindered by conditions such as advertising re-quirements, links with distribution channels, and theneed for new capital expenditures (355), Thus, the in-dustrial pattern is that of limited internal growth, withacquisition or establishment of smaller companiesbeing the primary method of expansion. Small plantsare opened to manufacture new products following in-vention and development, while large plants areopened by large companies to take advantage of loweroperating costs. These large companies tend to be ex-tremely diversified as a whole, yet there is little prod-uct diversification within their medical devices plants(8).

Recently, the distribution of medical devices hasshifted from small regional and local suppliers to majornational dealers. National dealers are often subsidiariesof large manufacturers or are acquirers of small man-ufacturing firms. The advantage of larger firms is thatthey are better positioned to provide special buyer ed-ucation through their larger, better trained staff (355).The inability of potential manufacturers to gain accessto these networks is an additional barrier to growthof the small firms entering the medical devices fieldand probably accentuates their acquisition by largermanufacturers.

The U.S. medical devices industry is somewhat insu-lated from price competition by the high level of third-party reimbursement, and price competition is not assignificant a force in mitigating price increases as it isin other industries. Nevertheless, there is a high degreeof product differentiation, and the industry appearsto be competitive at various levels even though themarket for the most part is price insensitive (8). In

—


other words, a policy of product differentiation andsales promotion may increase a firm’s net revenuesabove the competitive level (288). Profitability meas-ures confirm this viewpoint, indicating a slightly higherprofitability in the devices industry than existsthroughout the economy. This may explain the ob-served trend of expansion through acquisition (prod-uct differentiation), coupled with major national deal-ers (sales promotion) either being subsidiaries of largemanufacturers or being acquirers of small firms. Prod-uct differentiation, distribution, and perhaps the levelof new capital investments also appear to act as inhibi-tors on the growth of small firms (8) and contributeto such firms’ failure or acquisition by larger firms.

Innovation Process for Medicaland Surgical Procedures

The invention, development, and diffusion of medi-cal and surgical procedures may generally be describedby the model of the innovation process developed forproducts and their manufacturing processes. New pro-cedures usually involve some drug and/or device, andinnovations in medical and surgical procedures can beviewed as user-generated innovations, where a previ-ous innovation is adopted or adapted (modified) foranother purpose. Regardless of how medical and surgi-cal procedures fit into the model of the innovationprocess, however, a focus on procedures separate fromthe drugs and devices that are used in them is neces-sary, because physicians, as users, are both generators(technology-push) and purchasers (demand-pull) of in-novations. Thus, it is crucial to get at least a notionof how they perform these dual roles. But there areno standard determinants of when or how proceduresbecome medically acceptable (197) and few criteria forwhen they become obsolete.

There are three separate literature sources for ana-lyzing the dissemination of information in medicine.The first comprises sociological research on the diffu-sion of innovations in social systems (208,317); the sec-ond is literature concerned with the effects of commu-nication variables on attitudes and behavior (239); andthe third is the scattered, nontheoretical literature inmedicine, consisting of descriptive studies of the dis-semination and adoption of different medical innova-tions (62,145,233,331).

The medical literature on the dissemination andadoption of innovations is weighted toward studies ofsingle medical technologies which are diagnostic ortherapeutic in purpose. There is a large literature onhow physicians learn about and adopt new drugs anda growing literature on specific devices or techniques,but little is known about communication about or the

adoption of complex medical procedures which maynot involve drugs or hardware (e.g., psychotherapy).

In practice, however, the crucial distinction is be-tween communication which informs physicians aboutnovel technologies and that which influences physi-cians to act (405). Even though the most importantsource of new knowledge about improvements in med-ical technologies is the professional literature, physi-cians cite professional colleagues more often as sourcesthey turn to when contemplating actual implementa-tion of new procedures (145,233,234).

The importance of informal communication both inthe process of scientific discovery and in the diffusionof technological innovations seems to be a feature notonly in medicine but in all fields of technological dis-covery and diffusion (213). Moreover, it may be thatthere is a prestige hierarchy in which those at the topare “trend setters” (49). If this is so, widespread adop-tion of an innovation could be enhanced by convinc-ing influential organizations to adopt it first, then let-ting prestige-seeking organizations imitate them (213).

Physicians of greater prestige do tend to hear aboutinnovations sooner than others (62), and they are alsomentioned by their fellow professionals as influentialsources of information on the medical practice of oth-ers. However, the adoption process when the adopt-ing unit is an organization (e.g., hospital) is substan-tially different from the process when the adopting unitis an individual (e.g., physician in solo practice) (178,405), and these processes differ by the level of complex-ity of the organization. Outside forces such as third-party reimbursement or regulatory practices may alsoaffect how quickly the individuals in the medical com-munity learn about or adopt a technology.

The following general scenario may help make thesetheoretical and empirical findings more concrete.Medical and surgical procedures usually begin as user-generated (e.g., physician) innovations. In medicine,an innovative procedure may be in the form of adopt-ing an existing drug for a new purpose or changingthe mixture of drugs and their dosages to adapt themto a different medical problem. In surgery, it may bein the form of a modification of an existing technique(usually in accompaniment with modifications of thedevices being used) for application to a new use. Intreatment areas that do not depend on drugs or devices(e.g., psychotherapy) or in which drugs and devicesare used but are not crucial to the innovation (e.g.,primary care), it maybe an innovative interpretationof the existing knowledge (e.g., the multiple schoolsof psychotherapy which have sprung up, the “familyphysician”).

Increasingly, innovations in procedures arise in aca-demic or academic-associated centers, where physical


and professional resources are readily available; aresearch, innovation-seeking atmosphere is encour-aged; and contacts with others in the field extend notonly nationally, but also globally. Innovators in suchsettings know how to present the innovations in amanner that will be technically acceptable, and theyalso have the prestige which gives them access to pro-fessional meetings and journals to publicize their re-sults. Their presentations and publications not onlydiffuse the innovation to a wider audience, but moreimportantly, begin to legitimize it. Depending on theclaimed innovation’s nature, usually defined in termsof how the innovation will revolutionize or at leastsubstantially affect the related area of medical or surgi-cal practice, other academic centers will begin to pur-sue it, too.

At this point, several Government agencies may en-ter the picture. The National Institutes of Health (NIH)may provide support for the innovator and researchersin other health centers in the form of randomized clin-ical trials (RCTs), most likely conducted in some ofthe clinical research centers funded by NIH. A newuse for a drug, invention of a new device, or modifica-tion of an existing device requires the Food and DrugAdministration’s (FDA’s) approval. Increasingly, in-vestigational new drug or device uses approvedby FDA for limited testing are given to the same centerswhich NIH supports as clinical research centers (or atleast to the health institutions in which these designatedcenters are located), Sooner or later, the Health CareFinancing Administration (HCFA) may receive a re-quest for reimbursement of the new procedure and willgive great weight to the NIH clinical trials for evidenceof safety and efficacy. Meanwhile, FDA must makea determination of safety and efficacy for market clear-ance of the drug or device under review. FDA willoften have to make its decision long before NIHreaches a decision and terminates funding for the clin-ical trials. The reason is that FDA must act in a time-ly manner and reach its conclusion on minimal evi-dence, while NIH has no similar regulatory responsi-bilities and is more interested in the cumulative evi-dence. FDA’s decision, moreover, especially in the caseof devices, may rest on the narrow question of the ef-ficacy and safety of the device in a particular setting,not of the entire procedure in general use. But releaseof the device to the general market, once premarketapproval is given, also tends to speed up the diffusionof the procedure which NIH may be studying. Thisresult, in turn, places more pressure on HCFA to reim-burse for the procedure.

Most of these points are illustrated in the brief casestudies in appendix E on: 1) gastric freezing for thetreatment of ulcer, 2) hemodialysis for the treatment

of schizophrenia, 3) percutaneous transluminal coro-nary angioplasty, 4) maternal serum alpha-fetoprotein,and 5) hemodialysis and kidney transplantation.

Funding of the basic research which advances medi-cal care comes primarily from NIH, with smaller butimportant amounts from private foundations (223).The central role which basic research plays in the proc-ess of medical innovation (64) is the justification forthe substantial public and private moneys invested.

In the development and diffusion phases of medicalinnovation, initial findings are translated into clinicalprocedures. These phases are central to the innova-tion process, but there is relatively little formal fund-ing. The National Center for Health Services Research(NCHSR) was originally called the National Center forHealth Services Research and Development, but its en-abling law, when finally passed in 1974, specificallyforbade the center to fund development. Althoughabout half of NCHSR’s grant awards have been forprojects classified as demonstrations, little has been de-voted to new medical and surgical procedures.

The primary focus of NIH is research, and there ap-pears to be no systematic or comprehensive policy ofNIH support for development. Figures to documentthe size of NIH’s investment in development are notavailable. Although NIH grants and contracts havebeen given to support development in a number ofareas (e. g., the artificial heart program, cancer screen-ing, cancer chemotherapy, and, in recent years, hemo-dialysis), the amount invested in development prob-ably constitutes a relatively small portion of the cur-rent $3.8 billion NIH budget.

For developmental costs of procedures used in theprevention or treatment of individual diseases, privatefoundations have provided important support. A not-able example is the generous funding by the HartfordFoundation of Dr. Belding Scribners hemodialysis pro-gram in Seattle in the early 1960’s. Other examples in-clude grants by the American Cancer Society for can-cer screening and treatment programs and by the JulesStein Foundation for the development of radial kera-toplasty (a type of surgery on the eye).

Although there are no explicit data on which to baseestimates, the developmental costs of medical innova-tion are without doubt very large. By and large, thecosts of the developmental phase of early clinical ap-plication have been paid by patients, usually throughstandard medical insurance policies.

Even for procedures that have been clearly desig-nated as experimental, reimbursement has often beenprovided. Thus, for example, when total hip replace-ment was first introduced into this country in 1971,it came under the aegis of FDA because of the use ofthe acrylic, methylmethacrylate, in the operating room

Appendix D—Medical Technologies and Innovation . 157

construction of the new joint. Despite the artificialhip’s clear designation as an experimental device byFDA, the total hip procedure was reimbursed from theoutset as an acceptable surgical procedure.

Heart surgery has similarly been reimbursed fromthe outset through standard medical insurance policies.The single exception was the introduction of hearttransplant surgery at Stanford in 1969. Other institu-tions performing heart transplants have simplycharged standard fees. Coronary artery bypass graft(CABG), when introduced in 1969, was considered byits innovators to be standard therapy, despite repeatedcalls for randomized clinical trials (RCTs) of the newoperation (52,358). Clinical charges for CABG werepaid via standard policies from the outset and contin-ued to be paid even when, several years later, CABGwas finally subjected to RCTs.

Benson Roe, a cardiac surgeon at the University ofCalifornia School of Medicine in San Francisco, hasrecently described the historical justification for the“extraordinary” fees in cardiac surgery (316):

Historically, of course, there was justification for ex-traordinary fees in cardiac surgery. The developmen-tal years of this field were indeed difficult, demandinginnovative talent and an enormous amount of time—re-quirements that many were unable to fulfill. The earlycardiac surgeon participated in the diagnostic studiesand preoperative preparation, planned and directed thetechnical details of the cardiopulmonary bypass, con-ducted the entire longer operation, and personally su-pervised every detail of postoperative care, often spend-ing late nights at the bedside.

As heart surgery has become, if not routine, at leasta great deal more safe and considerably simple, Roesuggests (316):

. . . one might expect the surgeon’s fee to have droppedconsiderably, but it has not. On the contrary, fees forcardiac surgery have escalated at a rate that far exceedsthe inflationary factor.Much the same pattern Roe has observed in the case

of cardiac surgery has been followed for other techno-logically complex surgical procedures, including intra-ocular lens implantation and microdissection in brainsurgery, as well as orthopedic joint replacements. Notonly are the enormous costs of medical and surgicaldevelopment absorbed by medical insurers (222)—andeventually by the public—but the charges for new pro-cedures, once standard, remain high.

Public Accountability

Regulatory actions and more informed reimburse-ment decisions are intended to help ensure that newand emerging technologies are efficacious, have accept-able risks, and are appropriately used (e.g., are costeffective). Private industry determines which drugs and

devices it will develop primarily on the basis of mar-ket-based criteria. To address perceived deficienciesof the market approach, governmental actions infuseadditional criteria based on social and political con-cerns.

These governmental actions have generally been reg-ulatory in nature, concentrating on the costs to ourhealth, safety, and environment—costs which, becausethey are diffuse, can best be addressed through collec-tive, governmental actions. Government’s role as apurchaser of technologies, of great significance inhealth care because of Government’s role as insurer,has also led to a need to make more informed judg-ments about the kinds of technologies used in healthcare. These judgments are needed not only to minimizereimbursing for the use of ineffective technologies, butalso to help decide which among the array of technol-ogies are the most appropriate. The regulatory proc-ess unquestionably slows diffusion of technologies intothe marketplace, and some technologies are filteredout. Slowing the diffusion of new technologies mayallow for more informed and timely decisions beforewidespread use.

Constraining the diffusion of new drugs or devicesbefore they are adequately assessed also affects theconditions under which new technologies are fostered.Meeting regulatory requirements for evidence of effi-cacy and safety increases industry’s costs, for exam-ple, by delaying industry’s return on capital investedin R&D activities. Factors such as these play a signifi-cant role in industry’s assessment of whether a newtechnology could be profitably marketed or in decidingwhich of several promising technologies to developfurther. But the full extent of a new technology’s capa-bilities is usually not known until it is put into use,and use can lead to improvements and, in some cases,further innovations.

The question of the effect on innovation from regu-latory and reimbursement policies is not simply oneof whether innovation is inhibited, but also whetherthe alterations in the innovation process are unin-tended and undesirable. Government support of R&Dhas long sought to alter the innovation process, mostnotably to accelerate the pace of innovation and topush it in certain directions. NIH is a prime exampleof both undirected and directed support for the devel-opment of new medical technologies, combining basicresearch within separate institutes targeted at specificdiseases,

As the recent experience of air quality control pro-grams demonstrates, market-modifying factors suchas regulation can also alter the direction that innova-tions take. The kinds of regulations put into effect canforce innovation along certain pathways, some ofwhich allow for more maneuvering (e. g., in contrast


to specifying the kinds of pollution control devices tobe installed to achieve air quality standards, the “bub-ble” concept of regulating air pollution sources, wherea maximum air pollution level is set, leaves it to pollu-tion sources to stay within those limits by whatevertechniques they can muster). Restraints on the market-ing end of the innovation pathway confront innovatorswith new conditions, and the hallmark of innovationis to generate new answers when conditions change.Although there will be industrial losers and winnersunder these regulatory rules, that does not necessar-ily mean that innovation has been hindered. It mayinstead have its direction altered, much as Governmentattempts to alter innovation at the R&D end for similarsocial purposes.

There is general agreement that competition amongmedical care providers is typically not based on price(331); under current reimbursement policies, there areincentives to adopt all available diagnostic tools andto pursue any therapy anticipated to have any value.This is particularly true for hospitals. Third-party cov-erage currently accounts for about 90 percent of ex-penditures for hospital care. As the price of technologyhas little effect on providers and patients under existinghealth insurance arrangements, a greater adoption oftechnology can be expected to occur under these ar-rangements than would occur under more price-com-petitive reimbursement arrangements.

At a simple level of comparison, recent changes incurrent regulatory and third-party reimbursement pol-icies can be thought of as approaching some middleground from opposite ends of the spectrum. Regula-tion purposefully slows down the innovation process,particularly at the early diffusion stage, and modifica-tions are now being sought (e.g., in premarket approv-al requirements for drugs) to ensure that this slowingof the innovation process is no more than necessaryto achieve the regulatory program’s objectives. Cur-rent reimbursement policies, on the other hand, ’ areseen as boosting the diffusion of new medical technol-ogies and modifying existing technologies beyond whatwould take place under more price competitive sys-tems, and reforms are being aimed at constraining theadoption process.

Because the purpose of regulation is to infuse socialcriteria into judgments of a new technology’s worth,conclusions based on the economic impact of regula-tory requirements must be reached with caution. Reg-ulation is expected to change the innovation process.The issues are whether the specific changes were in-tended and whether the benefits of regulations areworth the price paid in resulting alterations of the in-novation process.

Present reimbursement policies tend to reward theuse of technological innovations and discourage lesstechnologically oriented patient care activities (2).Thus, there is a need to infuse more price sensitivityinto the reimbursement system. Taken together withthe regulatory approach, changes to infuse price sen-sitivity would theoretically: 1) allow market entry ofinnovations which have met social criteria of worthi-ness, and 2) make it possible for those new technolo-gies which have passed the regulatory test to then com-pete with one another on a price basis. Curtailing ex-cessive demand by a more price-sensitive approach,however, means changing the conditions of the cur-rent medical technology innovation process. Again,the question here is whether such major changes in thedemand for new medical technologies will affect theinnovation process in unintended and undesirableways.

Regulation

The purpose of regulation is to guide the course oftechnical change in such a way that, over time, newtechnologies are responsive not only to the cost andperformance characteristics valued by the marketplace,but also to the social values that motivate regulation.Regulatory requirements become added conditions forsuccessful completion of the innovation process.

There are three possible approaches to regulation:1.

2.

3.

precluding technologies deemed socially-undesir-able by either banning or selectively restrictingtheir use;deflecting technologies by forcing their develop-ment or diffusion (e.g., through uniform require-ments) into technologies with performance char-acteristics deemed socially desirable; andusing market-like mechanisms (e. g., pollutionfees or marketable pollution rights) to encourageproducers to economize on the use of commonresources such as air and water (244,283).

Of these regulatory constraints, preclusion and de-flection are the methods currently used to regulate bio-medical technologies. Prescribing how a product is tobe made is preclusive, while specifying the qualitiesthe product must have is deflective. The difference isbetween standardizing the product (preclusion) andstandardizing its performance (deflection).

In addition to these purposeful constraints, regula-tion requires compliance outlays and introduces anumber of other factors which can indirectly constrainthe innovation process. Compliance outlays includesuch direct costs as efficacy and safety testing, legalfees, and employee time spent on regulatory matters.

Appendix D—Medical Technologies and Innovation 159

Greater R&D costs are usually associated with themore technically demanding regulations, such as thoseapplied to drugs. These are resources that could bespent in other areas (e.g., development and marketing)or could enhance profit margins.

Uncertainty discourages risk-taking and prolongsdecisionmaking, and regulation can introduce uncer-tainty over how to comply with the regulatory require-ments, which may constitute a “moving target. ” Forexample, additives are not allowed in foods if they arefound to cause cancer. But technical advances in de-tecting smaller and smaller concentrations of one sub-stance in another (in some cases, at the level of 1 in1 billion), coupled with the regulatory interpretationthat any amount detected is illegal, mean that com-plying with the law depends on the latest advances indetection methods, even if the best method of keep-ing the banned substance out of the food has loweredconcentrations below that detectable by the previousmost sensitive method of detection.

Delay is an inevitable result of certain types of regu-lation, as in those areas requiring premarket approval.Delay also occurs administratively —e.g., when short-ages of qualified personnel or turnover in personnelprevent prompt review of applications, or when a reg-ulatory reviewer is unsure of what decisions to makeand consults extensively within the agency beforereaching a decision. Litigation over an agency’s deci-sions and judicial review of these decisions impose fur-ther delays. These delays can be significant enough toaffect the expected economic return on an innovation,which might cause the petitioning company to aban-don the product and make investments elsewhere. De-lay may, in effect, extend the life of already approvedproducts, and, if costly, can impede the entry of smallbusinesses into the particular market. Delay can alsoreduce the effective patent life of a new product, af-fecting its return on investment.

Regulation can also have other effects on innova-tion. It can affect the psychology of officials of privatefirms in conscious ways (e.g., when officials make de-cisions with an eye toward the likely reactions of theregulating agency) and unconscious ways (e.g., be-cause officials have been accustomed to having to meetregulatory requirements). Furthermore, disclosure ofdata in support of an application for a new productapproval can help another manufacturer compete withthe original manufacturer.

REGULATION OF DRUGS AND MEDICAL DEVICES

The responsibility for Federal regulation of drugsand medical devices rests with FDA. FDA’s regulatorymodes are: 1) the establishing of standards, 2) the pre-

market notification process, 3) the premarket approvalprocess, and 4) policing.

Policing typically occurs in lawsuits by FDA againstviolative products or firms. This mode is employed,for example, in the regulation of the labeling of medicaldevices. Establishing standards is a way of prescrib-ing requirements for products or processes. For exam-ple, regulations governing “good laboratory practices”specify the mandatory, or in some cases the recom-mended, characteristics of the well-designed, proper-ly conducted preclinical study. The premarket notifica-tion process gives FDA the opportunity to veto a firm’splans before they can be implemented. The 1976 Med-ical Device Amendments require that a firm intendingto distribute a device for the first time notify FDA 90days in advance to permit the agency to determinewhether the device requires premarket testing and eval-uation.

The premarket approval process is used by FDA toregulate drugs and certain devices. In the case ofprescription drugs, a manufacturer must conduct testsfor efficacy and safety on the drug, submit the datato FDA and obtain its approval before the drug canbe marketed (244). FDA becomes officially involvedin the development process for a new drug when itssponsor files a “notice of claimed investigational ex-emption for a new drug” (IND) for permission to testit in humans. There are three phases in the clinical in-vestigation, and each phase must have been precededby specified animal tests. (Animal test requirementsfor contraceptives are more stringent than the re-quirements set forth below for other drugs).

Phase I studies are investigations of a new drug’sclinical pharmacology to determine levels of tolerance(toxicity), followed by early dose-ranging studies forsafety (and, in some cases, efficacy) in selected pa-tients. The total number of both healthy volunteersand patients, which varies with the drug, ranges from20 to 50. If the drug is found to be safe, the manufac-turer can proceed to the next phase of testing. PhaseI studies must be preceded by 2- to 4-week studies intwo animal species.

Phase II studies are designed to demonstrate effec-tiveness and relative safety of a new drug and are car-ried out on 100 to 200 patients under controlled condi-tions. If the drug’s therapeutic value is demonstratedand there are no serious toxic effects, the manufacturercan proceed to the next phase. Phase II studies mustbe preceded by 90-day studies in two animal species.

Phase 111 studies are expanded controlled and uncon-trolled clinical trials, involving 500 to 3,000 patientsin usual medical care settings (clinics, private practice,hospitals). At least two well-controlled clinical trials,accompanied by complete case records for each pa-


tient, are usually required by FDA for approval of a“new drug application” (NDA).

If these clinical trials are successful, the drug’s spon-sor may file an NDA. An NDA is a request for FDA’spermission to market the drug. Chronic animal tox-icity studies (l-year dog, 18-month mouse, and 2-yearrat studies) must be completed by the time of NDAsubmission. If the FDA review finds the effectivenessand toxicity data acceptable, the application is ap-proved. Since 1962, FDA has reviewed over 13,500 ap-plications for INDs and has approved about 1,000NDAs (154).

The Medical Device Amendments of 1976 to theFood, Drug, and Cosmetic Act greatly expandedFDA’s role in regulating medical devices. Prior to the1976 amendments, FDA had classified devices such assoft contact lenses, pregnancy test kits, intrauterinedevices, nylon sutures, and hemostats as “drugs” (359).The U.S. Supreme Court ruled in 1969 that this movewas justified since Congress intended the public to beprotected from unsafe and ineffective devices (299).

The Medical Device Amendments of 1976 estab-lished a three-tiered system of controls on medicaldevices. Class I devices are subject to general controlsonly; Class II devices must meet performance stand-ards; and Class III devices must have premarket ap-proval.

Class I devices are subject primarily to the Food,Drug, and Cosmetic Act’s basic prohibition againstmisbranding and adulteration. Class I controls applyto accuracy in labeling and the sanitation and physicalintegrity of low-risk medical devices. All devices mustmeet these minimum standards. FDA also has the pow-er to ban any device, regardless of classification, whichpresents a substantial deception or an unreasonableand substantial risk of illness or injury that is not cor-rectable by labeling.

Class II controls are placed on devices for whichgeneral controls alone are judged insufficient, butabout which sufficient information exists or could bedeveloped to establish performance standards for thedevice. Under the 1976 amendments, existing volun-tary standards could be used, but legal counsel advisedthat such actions would violate due process, as the“voluntary” standard might become essentially “man-datory” with the FDA stamp of approval, circumvent-ing the opportunity for public comment and discus-sion (247).

Class 111 controls are comparable to the premarketapproval process for drugs. These controls are appliedwhen general controls or performance standards maynot provide reasonable assurance of the safety and ef-ficacy of a device which is life-sustaining, life-support-ing, implanted, or presents a potential unreasonable

risk of illness or injury, or when performance stand-ards cannot be developed. Any device which was clas-sified as a “drug” before the amendments is auto-matically assigned to Class III unless reclassified. Anydevice developed after the enactment of the amend-ments which is not judged by FDA to be “substantial-ly equivalent” to a preamendment device in Class I orClass II will also be assigned to Class III and requirea premarket approval application. In the first 4 yearsafter implementation of the 1976 amendments, about98 percent of the listed devices in the 10,540 premarketnotifications received were declared “substantiallyequivalent” to a preamendment Class I or Class IIdevice (260).

The Medical Device Amendments also allow FDAto permit developing and marketing approval of aClass III device under a “product development pro-tocol, ” where FDA and the manufacturer agree in ad-vance on a plan for the development, testing, and re-lease of the device. This approach has not been im-plemented.

The 1976 amendments require any distributor of adevice intended to be marketed for the first time tofile a notice with FDA at least 90 days in advance topermit the agency to decide whether the device needspremarket approval to assure safety and efficacy. FDApermits earlier distribution if it concludes and notifiesthe distributor that premarket approval is not required.If the 90 days pass without comment from FDA, mar-keting can begin. In 1981, FDA estimated that 2,300premarket notifications would be reviewed.

Industry often uses FDA approval to advantage inits marketing strategy. All results of clinical investiga-tions will ultimately be included in a package insert,product data sheet, or physicians’ brochure, which areFDA-approved generators of promotional claims(300).

MEDICAL AND SURGICAL PROCEDURES

Except insofar as State laws require that medical andsurgical procedures be performed by physicians andthat hospitals have certain facilities if they are to carryout certain procedures, medical and surgical proce-dures are essentially unregulated. State licensing stat-utes that define who can and cannot practice medicine(dentistry, etc.) preclude other technical personnelfrom performing many such procedures, Laws that re-strict the performance of procedures to licensed facil-ities such as hospitals deflect from these settings inno-vative organizational arrangements such as home birthdelivery and outpatient surgery.

Regulation of the practice of medicine is a Statefunction carried out by State medical licensing boards.


However, State medical licensing boards primarily reg-ulate entry into the practice of medicine and do littleto monitor the continued competence of licensed physi-cians beyond assuring that they meet requirements forcontinuing medical education. However, a Federal pro-gram, the Professional Standards Review Organiza-tions (PSROs), was enacted in 1972 to review medicalcare delivered to persons eligible for Medicare or Med-icaid coverage. (As this program’s functions relatemore to Federal reimbursement for medical services,it will be described in the section below on reimburse-ment ).

REGULATION OF CAPITAL INVESTMENTS

Three related Federal programs have been enactedin an attempt to regulate capital investment: 1) sec-tion 1122 review, 2) State certificate-of-need (CON)laws, and 3) the National Health Planning and Re-sources Development Act.

Since 1972, section 1122 of the Social Security Acthas required the Medicare and Medicaid programs towithhold funding for depreciation, interest, and returnon equity capital for certain investments found incon-sistent with planning objectives by a health planningagency. The provision applies to investments of morethan some specified amount (initially $100,000) andcovers changes in beds and services that are providedby certain health care facilities, such as ambulatorysurgical facilities. Health maintenance organizations(HMOs) are included, but private physicians’ officesare explicitly exempted. In 1977, 37 States had con-tracted with the Department of Health and HumanServices* to conduct section 1122 reviews.

The effect of section 1122 review is controversial.Since the statute excludes operating expenses and phy-sicians’ services, only a small percentage of a provider’stotal revenue may be at risk of scrutiny or control.For example, the operating expenses of computed to-mography (CT) scanners account for as much as 50to 75 percent of the technical expenses (279).

State CON laws, in effect, constitute a franchisingprocess for potential adopters of expensive medicaltechnologies. Enacted by 35 States by 1977, these lawsrequire prior approval by the State of investmentsabove a certain threshold (now usually $150,000 ormore). Local health systems agencies have responsibil-ity for areawide planning and initial CON review. Al-though the laws vary, most apply to hospitals andnursing homes. Like section 1122, most CON laws ex-empt private physicians. Sanctions include denial ofoperating licenses, court injunctions, and fines.

● Then the Department of Health, Education, and Welfare.

The National Health Planning and Resources Devel-opment Act of 1974 required States to pass CON lawsby 1983 as a condition of future Federal funding underthe Public Health Service Act, the Community Men-tal Health Centers Act, and the Alcohol Abuse andAlcoholism Act. The 1974 planning act generally ap-plied to the same facilities covered by section 1122review. However, the 1979 planning act amendmentsexempted HMOs from having to secure a CON for in-patient investments because of a belief in HMOs’ effi-ciency.

In an early study of CON laws, Salkever and Bice(333,334) reported reduced hospital expenditures onbeds, but unchanged overall hospital investment.Faced with greater control over beds, hospitals mayhave channeled their investments to other technol-ogies. Furthermore, as Ginsburg (162) had found earli-er, occupancy was positively associated with bed ex-pansion, although occupancy rates had no apparenteffect on total hospital investment.

Cromwell, et al. (75), investigated the effect of CONlaws on the adoption of specific technologies. CONappeared to reduce adoption rates for expensive, wide-ly adopted technologies—namely, X-rays and cobaltand radium therapies—but did not affect other tech-nologies examined.

The existence of planning legislation was not corre-lated with interstate differences in the adoption of theCT scanner (392). In fact, impending legislation mayhave spurred adoption as providers rushed to placeorders before the law applied to CT scanners. Suchan effect may have occurred in California, whose 1976law exempted equipment already ordered (12,19).

Reimbursement Policy

In contrast to regulation, which is often seen as hav-ing constraining effects, the growth in third-party cov-erage of medical care is seen as a major cause of theexcessive adoption and use of many medical technol-ogies (142,331). It is important to keep in mind, how-ever, that just as regulation is but one influence on in-novation, reimbursement policy is but one contributorto the overall tendency to adopt and use medical tech-nologies at excessive levels. Other factors include com-petition among hospitals to achieve quality and pres-tige to attract patients and physicians, public demandfor sophisticated technologies, increasing specializa-tion within medicine, physicians’ desires to do as muchas possible for their patients, uncertainties related towhat constitutes appropriate use, and the defensiveoverutilization of medical tests and procedures becauseof the threat of malpractice suits.


There are two basic forms of payment mechanismsin the U.S. medical care delivery system: cost-basedand charge-based (306). Government programs, pri-marily Medicare and Medicaid, were developed to“buy into” what was then perceived as a market pric-ing system. When the statutes were enacted in 1965,the legislation established the principle that the Gov-ernment purchaser would pay institutional providersthe costs of services to patients. Physicians were to bepaid their “usual, customary, and reasonable” fees.The assumption was that Government was buying atthe margin and would not affect the average costs ofthe system,

The 1972 amendments to the Social security Act re-flected a growing understanding that purchases ofmedical services were sufficiently large to affect pur-chase price and costs. Consequently, limits were placedon the amount which would be paid by Medicare toboth institutional providers and physicians. Ratherthan being related to efficiency, these cost limitsreflected rates of increase in charges over time.

PSROs were enacted into law in 1972 and consistof areawide groupings of practicing physicians respon-sible for reviewing care delivered to persons eligiblefor Medicare or Medicaid coverage, They help assurethat services provided and paid for by Federal benefici-ary programs are medically necessary and of a quali-ty that meets locally determined professional stand-ards, and that they are provided at the most economi-cal level consistent with quality of care. PSROs areseparate, independent, nonprofit organizations locatedin a number of designated geographic areas of thecountry. They are physician-dominated organizations;upward of 50 percent of all practicing physicians inthis country nominally belong to the PSRO in theirarea, although usually only a small fraction of thesemembers participate regularly in PSRO activities.

For a variety of organizational and legislation rea-sons, PSROs have first concentrated on reviewing in-patient care provided in short-stay hospitals. One oftheir hospital-review activities is traditional utilizationreview intended to reduce unnecessary hospitalization.A second review activity is profile analysis, by whichPSROs retrospectively review patient care data (aggre-gated by, for instance, provider or physician character-istics) to highlight patterns of care. Such analyses allowPSROs to identify problems in the use of services andto set objectives for changing the use of services. Athird major type of hospital review activity is the“medical care evaluation” study which focuses moreon quality of care than on cost containment.

Some PSROs, especially those with long experiencein hospital utilization review, have moved beyondthese activities to take on utilization or quality of care

review in other facilities or medical settings. The majortopics of such studies are ancillary services (virtuallyall services except for room and board, and nursing,dietary, or physician services in the hospital), long-term care review, and ambulatory care review. Allthree types of studies have been done in demonstra-tion projects during the late 1970’s and have been car-ried on since then by some PSROs, often as “specialinitiative” studies. At one time or another, as manyas one-quarter to one-third of all PSROs had engagedin ancillary services or long-term care review; ambula-tory care is, so far, a less well-developed field.

Several PSROs (or separately incorporated analogs)do utilization review for private firms on a contractbasis. Perhaps as many as one-quarter of PSROs wereengaged in such review as of 1980, and they coveredpatients whose care was financed by private insurancecompanies, self-insured corporations, the CivilianHealth and Medical Program of the Uniformed Serv-ices, labor unions, and municipal governments.

In addition, several PSROs over the past few yearshave engaged in cooperative research projects, in-cluding projects related or analogous to technology

assessment. One example of such a project is an ongo-ing RCT to evaluate different educational interventionsintended to reduce the use of an outmoded obstetricpractice (X-ray pelvimetry) in hospitals duringdeliveries. Other PSROs have collaborated in studiesof variations of hospital use for several conditions suchas myocardial infarction (heart attack) or gall blad-der surgery.

Although the PSRO program pursues many objec-tives and tasks, the most visible have been thoserelated to utilization and costs of hospital care. The8 years of the program have not produced the desiredreductions in hospital stays or, especially, in the costsof Federal health programs such as Medicare. Improve-ments in quality of care, although less well docu-mented than the effects on costs, suggest that PSROactivities have ranged broadly across diagnoses andservices. Currently, the Reagan administration isdeemphasizing PSROs by defunding those thought tobe ineffective and by consolidating areas.

There are two widely used mechanisms to set reim-bursement levels in the “private” sector of the medicalcare market. One mechanism is the cost-based BlueCross/Blue Shield reimbursement system. In manyways, this system is similar to the Medicare program.Hospitals are reimbursed the “reasonable” cost of pro-viding care to patients, and physicians are paid “rea-sonable” fees, The second mechanism is payment forbilled charges. This approach is used by some BlueCross/Blue Shield plans and in all contracts establishedbetween patients and other insurers. Under this ap-


preach, all or some of the charges of hospitals andother medical providers are paid through insurers, un-less there are copayments and deductibles which arepaid by the patient. There are also patients not coveredby Government or other insurers who are responsiblefor their own bills. Billed charges are more like a mar-ket mechanism, except that demand is not directly af-fected by the income or wealth of the patient. A thirdpayment mechanism, not yet very widespread, is cavi-tation, whereby a fixed amount is paid for each pa-tient per time period, regardless of the health servicesprovided. The cavitation method generally involvesthe integration of financing and services, thus placingthe provider of care at financial risk.

INFLUENCE OF REIMBURSEMENT ON THEDEVELOPMENT, ADOPTION, AND DIFFUSIONOF MEDICAL TECHNOLOGY

When coverage has been offered from the outset fornew and experimental medical and surgical proce-dures, a high level of reimbursement has been justifiedon the basis of the special skills and large amount ofprofessional time required, and perhaps on the basisof increased risk. But, when such procedures have be-come routine, requiring less time and skill and posinglesser risks, fees for the procedure have usually in-creased rather than fallen (316).

Several examples have been provided by Blue Shieldof California (40). Phakoemulsification of the crystal-line lens, introduced as an alternative to lens extrac-tion for cataract, is—once learned—shorter and nomore complex than standard lens extraction, yet sur-geons initially attempted to charge 25 to 30 percentmore for the new procedure than they charged for theolder one. The Blue Shield Medical Policy Commit-tee disallowed the increase. Another example is theflexible fiberoptic endoscope. This new instrument iseasier to use than the standard rigid instrument, yetphysicians introducing the new procedure attemptedto charge 25 percent more. Similarly, orthopedic sur-geons who introduced arthroscopic menisectomy fortorn knee cartilage wished to charge the full fee forthe standard open arthrotomy and an additional feefor arthroscopy. In this instance, Blue Shield of Cali-fornia agreed to pay the full arthrotomy fee and anadditional 50 pecent of the arthroscopy fee. The ra-tionale for Blue Shield’s concession was that carryingout the simpler procedure might eliminate the need formany days of hospitalization and laboratory tests,with a considerable net savings in total charges.

Allowing a simpler procedure to be billed as a morecomplex procedure results in questionable increases inphysicians’ fees. In the example just cited, the largedifference in allowable charges when an operative pro-

cedure is added to a diagnostic procedure offers astrong invitation to remove some tissue during arthros-copy. During the diagnostic examination of the knee,a small piece of redundant synovial membrane maybe seen—a finding of no great import. Removing apiece of this tissue makes the procedure a “synovec-tomy,” for which the customary charge is $1,300,rather than simply a diagnostic arthroscopy, for whichthe customary charge is $500. The above scenario pre-sents a situation that may be reasonably justifiedmedically, but, even interpreted generously, there isa clear fiscal invitation to perform a procedure thatis more, rather than less, complex.

There also is a much more serious consequence ofthe manner in which charges are submitted for experi-mental procedures. With increasing scrutiny by third-party payers of bills submitted for new procedures andwith more than occasional denial of payment for suchbills, there is a strong incentive for physicians to re-quest payment for a standard procedure rather thanthe new one. This is also encouraged by the fact thatnew procedures often do not have a procedure codenumber, by which most bills are processed. Requestingpayment for a standard procedure may simply reflectan honest effort to use whatever code number seemsmost nearly to approximate the procedure actually per-formed. Whatever the motives, the net result is thatthe identity of the new procedure may be concealed,and the fact that an experiment has been carried outmay not emerge.

In bills submitted to Blue Shield of California, thereis an approximately 15-percent error rate in the codingof all procedures (39). It is estimated by the medicaldirector that 1 percent of the errors involve the useof existing codes for procedures to which new codeshave not been assigned.

Because it is difficult to define exactly what consti-tutes “accepted medical practice, ” the new proceduresthat have the best chance of being reimbursed are theones which deviate the least from existing procedureswhich are already being reimbursed. The Federal Gov-ernment, for example, has traditionally favored cover-age of new technologies perceived to be modificationsof existing interventions (270). The incentives, there-fore, are toward the development of parallel proce-dures or extensions of existing technologies.

For procedures that deviate substantially from ac-cepted medical practice, the reimbursement systemmay require considerable testing for safety, efficacy,and costs to determine if they offer sufficient contribu-tions to compensate for their deviation from standardmedical practice. These circumstances have several im-plications. First, when procedures remain outside thecoverage range, they may also suffer the fate of ano-nymity, neglect, lack of funding, or underutilization.


An obvious example is the traditional exclusion frommost insurance plans of much preventive medical care,most notably screening services. Second, the scrutinyof radical innovations rather than of incremental im-provements may be misplaced to the extent that thegrowth in medical expenditures is the primary reasonfor such scrutiny, The collective expense of small testsand procedures is arguably far greater than that of afew “big ticket” technologies (249). Third, if radicalinnovations have the most difficulty in receiving fav-orable coverage decisions, innovators might be in-clined to pursue less radical but more easily acceptedinnovations. This is a difficult hypothesis to test, asradical innovations have less chance of commercialsuccess than minor innovations; but once they pen-etrate the market, the magnitude of their commercialsuccess is greater than for minor innovations. Fourth,as discussed above, a technology-by-technology ap-proach to coverage decisions, with priorities deter-mined by how radically each technology differs fromexisting ones, may lead those seeking payment for theuse of new technologies to submit their claims for pay-ment under the guise of accepted procedures.

Under either cost reimbursement or charge payment,third-party payments generally are intended to coverthe full costs of new technologies, including purchase,maintenance, or operation of equipment; the leasingof equipment; the cost of drugs; or the facilities andequipment needed for a procedure (19). One wouldexpect that greater adoption of technologies wouldoccur under these relatively price-independent condi-tions than would occur under a more price-sensitivesystem. Cromwell, et al. ’s, interstate analysis (75)found that the percentage of revenues from third par-ties significantly and positively related to a hospital’sadoption of expensive technology. Russell (331) foundthat adoption of cobalt therapy and electroencephalog-raph occurred faster when the level of insurance cov-erage was higher and proceeded more rapidly as thatlevel grew. She also found that a greater contributionto hospital costs by Medicare was associated with in-creased adoption of cobalt therapy, intensive carebeds, and diagnostic radioisotopes. And Willems (392)concluded that open-heart surgery spread more quick-ly in areas with faster growth in insurance coverage.

Third-party reimbursement can also indirectly af-fect the adoption of technology by changing the avail-ability of financial capital to potential adopters. Aprominent example is the Medicare program, whichreimburses institutional providers for capital as wellas operating costs. Medicare payment for allowablecapital costs such as depreciation and interest providesa source of internally generated funds (28). Third-partycoverage, especially by Medicare and Medicaid, has

also reduced hospitals’ risks of bad debts, thereby im-proving their standing as credit risks to private lenders.Other changes in governmental programs, such as theHill-Burton program for funding medical facility con-struction and modernization, as well as various tax-exempt bond programs, have affected the source offinancial capital.

In addition to affecting the adoption of technologies,the extent of third-party coverage would be expectedto affect the use of technologies. Data on the use ofspecific technologies are generally lacking, however.Cromwell, et al. (75), found that many hospital tech-nologies are underutilized after being adopted. Non-profit hospitals in the Boston area were using auto-mated analyzers, patient monitors, and, in teachinghospitals, diagnostic X-rays, at only about half ofcapacity. Willems (392) considers such underuse aspresumptive evidence of the hospitals’ overinvestmentin new equipment.

It is not clear how this relatively price-independentadoption of medical technologies is used by medicalcare providers to compete with one another. As sum-marized by Banta, et al. (19):

Studies of hospitals have found no definite relation-ship between measures of competition and adoption.The situation is complex, because the characteristics ofthe market may relate not only to competitiveness, butalso to the availability and sharing of information andto local standards of practice. The evidence conflicts,depending on the characteristic used and the technologystudied. Russell (331) found that concentration of mar-ket power among a few large hospitals did not appearto influence the adoption of three common and twoprestige technologies, but that hospitals in more concen-trated markets were less likely to adopt open-heart sur-gery. Prior adoption in a locality reportedly speededthe adoption of intensive care units and electroencepha-lographs, but not diagnostic radioisotopes, open-heartsurgery, renal dialysis, cobalt therapy, and computers(75,33 I). In urban areas, greater adoption of radioiso-topes and electronic data processing occurred wherethere were many hospitals per capita, the hospitals wereof similar size, and they were close to other hospitals(212,301).

Different patterns have also been observed betweenadoption and the number of physicians per capita. Fac-ing a low physician-population ratio, hospitals maycompete for physicians through technology adoption.On the other hand, fewer physicians may exert lesspressure for adoption. The adoption of CT scannersand radioisotopes appeared unrelated to the physician-population ratio (301,392). However, greater adoptionof intensive care units, open-heart surgery, cobalttherapy, and renal dialysis occurred among States withhigher ratios (75). .Thus, even though current payment mechanisms for

medical care services can lead to excessive adoption


of medical technologies, there are still constraining fac-tors which make it clear that cost is not the only fac-tor which influences adoption.

Discussion and Conclusions

No one factor seems to distinguish successful fromunsuccessful innovations (252). The key events identi-fied in studies of successful innovations depend on thechoices of innovations for study. Failure to find acause-and-effect relationship between one variable andsuccess in innovations is not surprising, however, giv-en the multiple factors in the innovation process.

It is therefore also not surprising to find that the im-pacts of regulatory and medical care reimbursementpolicies on the innovation process are difficult to sepa-rate from the impacts of other factors, In the regula-tion of drugs, for example, the evidence points towardregulation as contributing to, but not as being the soleor primary determinant of, higher R&D costs, greaterconcentration of new drug development in fewer andlarger firms, and an orientation toward the epidemio-logically and commercially more important diseases.

The impacts of regulatory policies are not well un-derstood. The availability of data on R&D costs, num-ber and size of firms producing new products, thenumber of new products, etc., almost compel research-ers to focus on these parameters in their evaluativework. Quantitative rather than qualitative analyses arewhat most people expect in order to translate complexrelationships into simple terms such as “bottom line”numerical estimates of how regulation affects the inno-vation process. The result is a focus on easily identifi-able costs and a neglect of difficult-to-quantify bene-fits, and most of the controversy centers on whetherthese identified costs are due to regulation or other fac-tors such as, for example, existing trends in the drugindustry at the time of the 1962 drug amendments.

On the other hand, the studies that focus on costsmight be thought of as providing presumptive evidenceof the costs of regulation, thereby shifting the burdenof proof to regulation’s advocates to counteract theevidence with findings on the benefits of regulations.The problem with this approach is not only that healthimpacts cannot be measured adequately, but that evenif they could be, there are no unambiguous methodsto compare the costs and benefits (270).

Regulation is meant to alter the market forces con-trolling the innovation process, so it should come asno surprise that observable economic measures are al-tered. The fundamental question underlying the debateover the absolute or net costs (where benefits are con-sidered) is whether the achievement of the social pur-poses of regulation is worth the costs. With respect

to drugs, even when evaluation of the impact of regu-lation focuses on absolute costs, most critics of the cur-rent regulatory process call for marginal alterations,not radical changes. Such changes include more flexi-bility in efficacy and safety testing, speeding up of thepremarket approval process, and the use of postmar-keting surveillance systems.

For regulation of medical devices, there seems to beno major opposition to the law per se, only a wait-and-see attitude and differences of opinion as to howFDA is implementing certain provisions of the MedicalDevice Amendments of 1976. For example, FDA hasproposed six broad categories of devices which wouldbe subject to premarket testing: 1) invasive devicesintended to pierce the skin or mucous membranes;2) implantable or prosthetic devices; 3) energy-introducing devices; 4) medicinal gas devices; 5) de-vices, other than in vitro diagnostic products, that areintended for use in diagnosing of disease or monitor-ing physiological functions; and 6) in vitro diagnosticproducts intended to provide information which willbe used, interpreted, or analyzed by a health profes-sional.

Industry associations, such as the Health Industry

Manufacturers Association, urge continued case-by-case determinations and are opposed to these broadcategories, because they believe that there is no demon-strated need for the rule and point out that FDA hasnot conducted cost-impact studies (176).

Much as an IND does, “investigational device ex-emption” (IDE) regulations describe the requirementsfor clinical investigation and the responsibilities of themanufacturer, clinical investigator, and the institution-al review board. None of this documentation was pre-viously required, and industry has protested that themandated process would simply give rise to additionalcosts and delays as well as automatically trigger anFDA inspection of facilities, if one had not been pre-viously done, when an IDE was submitted. Therefore,FDA has been requiring IDEs only for devices whichrequire premarket approval (Class III devices) (359),

A proposed “mandatory experience reporting” rulewould require manufacturers, distributors, and im-porters to report to FDA any device which may havecaused injury or death, has a deficiency that couldresult in death or injury or give inaccurate diagnosticinformation, or is the subject of remedial action. Theproposed rule would require those covered to reportdevice-related deaths within 72 hours after receivinga complaint, injuries within 7 working days, andremedial action or communication with distributors,health care practitioners, or users within 2 workingdays (8). In response to FDA estimates of $20 perreport, the Health Industry Manufacturers Association


estimates that the entire industry would incur a totalannual cost in excess of $400 million (177).

The wide availability of medical insurance contrib-utes to overadoption of many new technologies, butother factors in the medical care delivery system havesignificant influences. Some of these other factors canadd incentives to overutilize new technologies, but ad-ditional factors seem to be keeping the rate of adop-tion of new technologies below the level expected ifcosts were the only or primary criterion influencingadoption. Thus, just as regulation has contributed toexisting trends in the drug industry, current reimburse-ment policies also have contributed to overadoptionof new technologies but cannot be credited as the deter-mining factor.

In addition to its contribution toward excessive de-mand for new technologies, current reimbursementpolicy has another significant effect on the innovationprocess. Radical innovations, which by their very defi-nition often fall outside generally accepted medicalpractice, tend not to be reimbursed and thus may beless likely to be developed. The current system discour-ages the identification of new procedures as such.

This appendix has described the effects of regula-tion and reimbursement policies on the innovationprocess as separate issues, but they clearly are inter-related. As new medical procedures develop, theyoften make use of new drugs and devices or use exist-ing ones in modified ways. In either case, the drugsand devices generally have to pass through the regula-tory process. Until they are approved, regulatory re-view acts as a constraint on the adoption and dissemi-nation of the procedures in which they are used.

Regulatory review is generally limited to the techni-cal questions of safety and efficacy, without considera-tion of the costs or relative values of the proposed drugor device once it reaches the market. In some cases,however, it goes beyond these questions. For exam-ple, in reviewing the injectable contraceptive DepoProvera, FDA used marketing as well as safety andefficacy criteria to deny approval. In that case, FDA denied approval not only because of its concerns overDepo Provera’s cancer-causing potential, but partlyon the basis that the patient population originallytargeted for Depo Provera had diminished substantial-ly as other methods of contraception and sterilization

had become increasingly available and accepted (193).Nevertheless, the more usual circumstance is such asthat found in the approval of the catheter used in per-cutaneous transluminal angioplasty (PTCA). In thatinstance, FDA released the catheter from investiga-tional device status and approved its marketing forPTCA while the procedure itself was still consideredby many to be experimental. Thus, while regulationof the accessories (i.e., drugs and devices) acts as a con-straint on the adoption of the medical and surgical pro-cedures in which they are used, once these accessoriesare released into the marketplace, they can act to stim-ulate use of procedures which are still experimental andnot accepted medical practice.

Does this observation point to a strategy for medicaltechnology assessment in which the criteria are similarfor both regulatory and reimbursement purposes?From the review of the regulatory process, it appearsthat the system for regulation of drugs and devicesmeets certain social goals. Although economic consid-erations are important, these considerations point to-ward specifying how the present regulatory processcan be improved, but not toward the infusion of eco-nomic measures into the regulatory criteria themselves.

The infusion of economic measures into the regula-tory criteria themselves may be arbitrary and counter-productive. Users of innovations are important con-tributors both in determining the full extent of an in-novation and in developing new innovations as spin-offs, the exact results of which can never be determinedbeforehand.

The current regulatory process for medical technol-ogies may need marginal changes, but the consensusseems to be that its social usefulness is worth the costswhich it places on an innovation process dominatedby a market approach. This conclusion is compatiblewith the common sense notion that society shouldfocus on the use of the tools and not on the toolsthemselves to keep the constraints on the innovationprocess at a minimum while also addressing the issuesof cost, quality, and appropriateness of medical care.

Current reimbursement policies both stimulate andconstrain the development of new medical technolo-gies. Possible modifications of these policies might wellbe examined for their potential.

Appendix E.— Case Studies of Medical Technologies

Introduction

The five case studies in this appendix are includedas illustrative examples of the innovation process, in-cluding the development and adoption, of selectedmedical technologies. Of particular note are the effectswhich Federal research funding and regulatory and re-imbursement policies have on the innovation process.

The case study on percutaneous transluminal cor-onary angioplasty (PTCA) was written by DavidSawi, and that on hemodialysis and kidney transplantsurgery by Katherine Jones. The other case studies ongastric freezing for the treatment of ulcers, hemo-dialysis for the treatment of schizophrenia, and mater-nal serum alpha-fetoprotein (MSAFP) were preparedby OTA staff.

Gastric Freezing

In the mid-1950’s, a surgical leader in the UnitedStates, Owen Wangensteen (of the University of Min-nesota Medical School), observed that iced saline solu-tion lavaged into the stomach slowed gastrointestinalbleeding and that animal experiments showed reduc-tion in stomach acid output following gastric cooling.From these observations, Wangensteen conceived ofthe idea of using gastric cooling for treatment of pep-tic ulcer disease (143).

In collaboration with a small refrigeration company,he developed a device to circulate alcohol at –15° Cthrough a nasogastric tube to a balloon inserted intothe stomach. After testing the device in dogs, he firsttried it on one patient, then a dozen others (143).

In 1962, he reported his results in the Journal of theAmerican Medical Association: no serious side effects,markedly reduced stomach acid output, immediate re-lief of ulcer pain, and radiographic evidence of ulcershealing (381). The 1962 report was extensively coveredin the popular media. Wangensteen also sought tospread his procedure through professional meetingsand publications, and the American College of Sur-geons prepared an instructional film on the technique(143).

By the end of 1963, 1,000 devices had been sold and10,000 to 15,000 procedures had been performed na-tionwide (143).

Beginning in 1963, however, the efficacy and safe-ty of gastric freezing began to be questioned. In 1964,there began to appear published reports concludingthat acid suppression was limited or unrelated to painrelief, symptomatic improvement was short-lived ordue to placebo effects, and important risks were pres-

ent. Variations in the technique were used and becamearguing points for valid use, but by 1966 the techniquewas rarely used. Furthermore, Wangensteen lost hissupport from the National Institutes of Health (NIH)for research in gastric freezing, although he stillthought the procedure worthwhile and did not believehis earlier reports to be inaccurate (143).

Hemodialysis for Treatmentof Schizophrenia

Schizophrenia, a disorder characterized by misinter-pretation and retreat from reality, delusions, hallucina-tions, ambivalence, inappropriate affect, and with-drawn, bizarre, or regressive behavior, is estimated toafflict 2 to 3 percent of the population. Because of itsrelatively high prevalence rate, onset in adolescenceand early adulthood, and lifelong chronicity, thisdisease imposes a major toll of disability and a greateconomic burden on our society. Although effectivetreatment for the disease could help large numbers ofpeople, ineffective new interventions pose the prospec-tive risk of wasting large sums of money. Schizo-phrenia’s unpredictable course in individual patients,coupled with the difficulty of assessing the health statusof patients, objectively enhances the risk that ineffec-tive or unproved modes of treatment will be adopted,perhaps even widely, in the management of schizo-phrenic patients (50).

In 1977, Wagemaker and Cade (379) created intenseinterest by claiming, in a report published in theAmerican Journal of Psychiatry, dramatic improve-ment in five physically healthy schizophrenic patientstreated with weekly dialyses for Up to 16 weeks. Sincethen, Wagemaker and Cade have continued this treat-ment. Wagemaker has dialyzed an additional 15 pa-tients, reporting 100 percent success in 7 women and3 good successes, 3 partial successes, and 2 failures in8 men (109).

Scattered reports in the literature between 1925 and1960 had claimed improvement in schizophrenic pa-tients given blood transfusions either from remittedschizophrenics or from healthy volunteers, but Feer,et al. (139), apparently were the first to use dialysis.These investigators reported in 1960 that three out offive schizophrenics improved after only one or twohemodialyses (139).

There is some evidence that hemodialysis may re-move a circulatory psychotogen, as Palmour and Ervin(285) reported a substance characterized as a beta-endorphin in the dialysate of the patients treated by

167


Wagemaker and Cade. A 100-fold decrease in the sub-stance, corresponding to improved clinical function-ing, was found in 14 of 16 treated patients. This claimhas not been duplicated and reported by others.

Because of the positive report by Wagemaker andCade, the Clinical Research Branch of the National In-stitute of Mental Health (NIMH) funded three researchprojects using a double-blind design. Dr. Wagemaker,of the University of Louisville (Kentucky), developedthe first project beginning in September 1978 throughhis “own initiative with considerable institute staff con-sultation to develop an acceptable protocol. ” Thisstudy is near completion but has not been reported.The other two projects are being conducted at theUniversity of Maryland (Baltimore) and the Universi-ty of Washington (Seattle), both starting in September1979. In addition, the NIMH Intramural Program con-ducted a small study, the results of which were pub-lished in Science in March 1981 (338). Of eight chronicschizophrenics, none of the patients improved duringactive dialysis, and four patients worsened.

In September 1980, the National Center for HealthCare Technology (NCHCT) issued a memorandum atthe request of the Health Care Financing Administra-tion (HCFA) for a recommendation regarding the useof hemodialysis in the treatment of schizophrenia.NCHCT concluded that the evidence on its safety andefficacy was inconclusive and recommended that theprocedure not be covered under Medicare (109).

NCHCT later commissioned an analysis to estimatethe economic effect of the decision not to reimbursefor the procedure. The study found that under the cen-tral estimate, by which 1 percent (4,000 patients) ofMedicare-age schizophrenics would receive dialysis in1984, annual costs would reach a peak of $15.4 millionin 1983, and the present value of total costs over a7-year life of this practice would total $29.3 million.High and low assumptions, under which a minimumof 0.25 percent (1,000 patients) or 5 percent (20,000patients) of the pool of schizophrenics are dialyzed,result in peak annual costs of $3,9 million to $82.1million and total costs of $7.7 million to $149.9million.

If similar fractions of the entire schizophrenic pop-ulation were to be treated, the total costs of treatingschizophrenia with dialysis in this larger pool wouldbe five times higher, or $39 million to $75O million.While funds for patients not treated under Medicarewould be provided through Blue Cross/Blue Shield,insurance companies, and private individuals, a por-tion of this cost would be assumed by the FederalGovernment as diminished tax revenues due to in-creased medical expenditures (50).

This case study characterizes NCHCT’s former rolein responding to HCFA’s Medicare coverage issues. It

also suggests the utility and need for a continuedsystematic coverage evaluation process within the Pub-lic Health Service. Expenditures for dialysis in theMedicare population alone could have created substan-tial cost burdens in the absence of a clear Federalpolicy. In view of the unproved effectiveness of thetreatment, NCHCT’s recommendation was based onmedical/scientific grounds. As this case illustrates,however, it is very difficult to separate cost implica-tions from such decisions.

Percutaneous Transluminal CoronaryAngioplasty*

by David Sawi, M.B.A.Kaiser Foundation Health Plan, Oakland, Calif.

Introduction

On January 16, 1964, Charles T. Dotter and M. P.Judkins performed the first percutaneous transluminalangioplasty (PTA) (6,27). The patient, an 83-year-oldwoman, was referred to the University of OregonMedical School Hospital with a disorder of the left leg.She had a 6-month history of pain and infection of theleft foot and toes, and a gangrenous appearance inthree toes had occurred within the previous 3 months.Angiographic examination revealed a 0.5 cm longatherosclerotic obstruction of the left superficialfemoral artery at the level of the adductor hiatus. Thepatient was considered unsuitable for vascular surgeryowing to her age, poor cardiac condition, and bad run-off. Because of her advanced gangrene, low thigh am-putation was advised, which the patient refused.

It was then decided to attempt catheter dilation.Treatment, using a coaxial double catheter, lasted ashort time and gave excellent results. The patient’s paindisappeared within hours, and the patient was ambu-latory within weeks. Repeated followup angigraphyconfirmed the patency of the treated artery. The pa-tient continued to walk without difficulty up to herdeath at the age of 86 years.

With the increase in lifespan over the past fewdecades, more patients now need surgical vascularreconstruction. This need increased the demand formore centers specializing in vascular surgery andequipped with intensive care facilities (30). Additional-ly, surgical measures were not always successful whenapplied to smaller arteries and faced limiting factorssuch as technical difficulty and operative trauma (6).

● NOTE: Reference citations for David Sawi’s case study on PTCA appearon p. 174.

Appendix E—Case Studies of Medical Technologies c 169

Description of Procedure

PTA is the noninvasive, mechanical treatment ofvascular obstructions with the use of catheters (5). Theprocedure is done by one of two methods, dependingon the nature of the lesion.

The first method, called transluminal dilation, iscorrecting a stenotic lesion by enlarging the diameterin the constricted lumen (or passage). For example, inDotter’s initial procedure (6), a tapered, radiopaque,Teflon dilating catheter of approximately 0.1-inchouter diameter was slipped over a coil-spring catheterguide of about 0.05-inch outer diameter. The catheterguide had been passed down the lumen until its tip hadtraversed the stenosis. The passage of the dilatingcatheter over the catheter guide enlarged the stenoticlesion by exerting outward pressure on the lumen. Thismethod was refined by the introduction in 1974 of acatheter with a distensible (balloon) tip, which, wheninflated, exerted outward pressure. on the lumen (12).

The second method, called transluminal recanaliza-tion, is used to correct a vascular occlusion by creatingan artificial lumen through an occluded segment. Thecatheter guide is passed through the occluded segment,and then the passage is enlarged by the introductionof the dilating catheter over the catheter guide (27).This approach is possible because the atheroma caus-ing the obstruction consists of a low-density fattymaterial which has the characteristic of inelastic com-pressibility (12,30).

The above procedures are often done in conjunc-tion with anticoagulants and platelet aggregation in-hibitors, which seem to improve outcome (30).

There are basically two classes of transluminalangioplasty: peripheral angioplasty (PTA) and cor-onary angioplasty (PTCA). The technique performedby Dotter and Judkins in 1964 (6) was peripheralangioplasty. In September 1977, Andreas Gruntzigperformed the first nonoperative transluminalangioplast y of coronary arteries in a human being (12).

Indications and Alternatives

Indications for PTA are disabling claudication (limp-ing, cramp-like pain due to inadequate blood supply),salvage effort prior to amputation, short stenosis ina large caliber, accessible artery, and little likelihoodof success with reconstructive surgery (31).

Indications for PTCA are more restrictive. The pa-tient should have a short history of angina] pain andshould be experiencing disabling angina. The obstruc-tion should be within a single vessel to minimize riskshould complications occur. Grüntzig estimates thatof patients with coronary heart disease, 3 to 5 percent

of the older medical population and 10 to 15 percentof the younger population are suitable for PTCA (12).

Alternatives to PTA appear varied. At times, PTAis done as an alternative to surgery, whereas in otherinstances, it is done when surgery is contraindicateddue to high risk and little chance of success (i.e., inelderly patients with prior heart problems) (5). Also,PTA can be an alternative to amputation.

PTCA is an alternative to coronary artery bypassgraft (CABG) surgery, but only for a small percentageof patients. As of 1979, it appeared that the 5-year sur-vival rate for PTCA was comparable to that of bypasssurgery. However, improved blood flow to ischemicsegments was also noted (23).

Historical Development

The development of PTA can be viewed as having

four stages: 1) the technical development leading toDotter and Judkin’s initial peripheral angioplasty,2) the period thereafter during which clinical tests reaf-firmed the efficacy of the procedure and began to es-tablish clear-cut indications, 3) the modification of themethod by different types of catheters for the dilationtechnique, which lowered the rate of complications,and 4) the extension of the technique to coronary

angioplasty (PTCA).The technological development leading up to

Dotter’s initial procedure primarily related to therefinement of the catheter. Seidlinger (Sweden) firstintroduced the flexible catheter in 1953 (25). Mean-while, Dotter (U.S. ) had been refining a catheter foruse in occlusion angiography (1958) and diagnosticangiography (1951). In 1962, at about the time Dot-ter commenced post mortem investigations using acoaxial dilation catheter, Nordenstrom introduced fivetypes of balloon catheters for percutaneous insertionusing a modified Seidlinger technique (22). Prior toNordenstrom, balloon catheters had been introducedinto the vessel via an incision. In February 1963, Fogar-ty (England) used a balloon catheter for the extractionof distal thrombosis (8). In the early 1970’s, Postmann’s(Germany) caged balloon catheter and Wholey’s (U. S.)balloon catheters preceded Güntzig’s (Sweden) devel-opment of a viable and effective balloon-tipped cath-eter in 1974. It appears as though subsequent develop-ment of the catheter instrument has been primarily arefinement of Grüntzig’s model.

It is clear from this brief overview that researchersin a number of different countries were essentially

working on the same problem simultaneously. Twoprimary dynamics in the catheter development werethe desire to minimize the possibility of embolism as

98-144 9 - 82 - 1’2


a result of PTA and the desire to improve the patencyrate on followup studies. In regard to this secondpoint, Zeitler reports that between 1968 and 1980, heconducted a randomized study of 1,217 procedures.Treatment conditions included different types ofcatheters and different drug regimens (29). Zeitlerfound the patency rate of a newer, improved singleTeflon catheter and the Grüntzig balloon catheter tobe three times that of the coaxial dilating set.

Numerous other clinical trials have been conductedin addition to Zeitler’s study (see table E-l). Two pointsshould be made regarding these studies. One is thatthe outcomes seem to substantiate the claim that PTAis a viable alternative to incisive treatment and, in fact,can succeed where surgery might fail. The second pointis that the trials in table E-1 are not, strictly speaking,comparable with each other. A number of flaws pre-vent this comparison:

In later years, better patient selection increasedthe probability of successful treatment.The catheters were constantly being improved, es-pecially in Grüntzig’s balloon catheter.Criteria for a “successful outcome” are not con-sistent across trials.

● The skills of the physicians differed.Additionally, the trials were not randomized clinicaltrials, in which patients are randomly assigned to treat-ment/no treatment conditions, or to treatment A(PTA) /treatment B (e.g., surgery) conditions. The rel-ative efficacy of PTA was determined by comparingtreatment outcome v. historical data on outcome usingalternative methods. In spite of these studies’ methodo-logical flaws mentioned above, their results were suf-ficiently positive to encourage continued research.

Grüntzig’s balloon catheter (1974) is generally cred-ited with being the key to PTA’s rapid diffusion (3,4).Grüntzig received research support to develop theballoon catheter from a European firm named Snyder.This seems to have been a significant variable. Dotterhad been attempting to refine the balloon catheter fora number of years prior to Grüntzig’s success. Dot-ter’s lack of funding support hampered his efforts (4).

Notwithstanding a decision by Medicare to discon-tinue reimbursement of PTA, substantial sales growth(in units) is expected for the next few years (see fig.E-l). Based on figures provided by Cook, Inc., the sizeof the PTA catheter market for 1981 was approximate-ly 100,000 units. This was expected to increase to near-

Table E-1.—Reported Clinical Trials of Peripheral Transluminal Angioplasty

Publication Trialdate dates Observations a + %b Comments c

196419661968196919711973197319731974

1975

19751978197819781978

1979

19641/64-9/65

●

●

●

●

●

●

●

●

1969-75●

9177-817811/68-12/73

●

3178-41791971-3/78

15113153

59161100237

2543

210

5346961

1381,184

206

6418848

>50%- 50+ %71 “/0640/o (WA)d

70%71-890/o80-900/0840/o81 0/0

780/o

740/0 (WA)d

●

510!075 ”/0 (500/0)74%920/o

●

86°/0 (balloon)64°/0 (coax)

Dotter, and Judkins: F,P,IDotter, et al.Dotter, et al.: FBrahme, et al. (Sweden): FZeitler (Germany): F,P,IWierny, et al.: FDotter: FGrüntzig (Switzerland):Dotter: 1, with ballon

catheterZeitler: 1, with balloon

catheterZeitler, et al.:F,PZeitler: randomized catheterGrüntzig: PTCASchoop, et al.: IGerman study withnumerous contributors:

Grüntzig and Kumpe

197919801980

●

1978-801968-80

300

43172

1.217

Katzen: balloon catheterColapin, et al. (Canada)Zeitler

F,l

Key: “Unstated or unclear.alnd~cates “um~r of Procedures pe~ormed, Later reports undoubtedly include outcomes of earlY studiesbGenerally stated as patency rate within 2 weeks of procedure.

C F = femoral,d

“ P = popiiteal; I - iliac.WA indicates weighted average primary success rate across various trial conditions,

Appendix E—Case Studies of Medical Technologies ● 171

Figure E-1 .–Cook, Inc., Annual PTACatheter Sales

(actual and projected)

— Actual or forecast , 0

- - - P r o j e c t e d

I1979 1980 1981 1982 1983

YearCook, Inc 65 to 75 percent market share, $70,000 per unit

SOURCE M Kanne, Cook, Inc , Bloomington, Ind

ly 300,000 units by 1983. After that, growth shouldlevel off to an annual rate of 10 to 15 percent (17).

Innovation of PTCA

In 1977’, Gruntzig performed the first PTCA (seetable E-2). There appear to be two major factors lead-ing to the innovation of PTCA:

● PTCA is, essentially, an extension of the sameconcept, methodology, and technology as thatused in PTA; to extend the concept to coronaryarteries was nearly inevitable.

● Gruntzig’s refinement of the balloon catheter pro-vided the technology necessary to safely performthe procedure.

Diffusion of PTCA

At present, three factors are driving the diffusionof PTCA. The first factor is Gruntzig himself. Grunt-zig’s impact has been felt through the instruments hedeveloped to perform the technique; his contributionto the literature (see table E- 2); his refinement of theprocedure, with meticulous recording of numerousclinical trials; and the training he has provided to otherphysicians, both in Zurich and other sites to super-vise initial operations.

A second factor is the Food and Drug Administra-tion’s (FDA’s) decision in late 1980 to release the PTCAcatheter device from investigational device exemption(IDE) status (19). An IDE status limits a company’sability to market a product by placing strict guidelineson the product’s distribution and use. Each purchas-ing institution has to be designated as an investigatorby FDA, and this involves considerable documentationand takes 2 to 3 months for approval. With removalof IDE status, the restrictions are removed, and mar-keting and purchasing of the catheter are simplified.USCI, one of the two manufacturers, claims that sinceits catheter received premarketing approval status, thenumber of centers which have ordered equipment toperform PTCA has doubled (9).

The third factor is the Interim Registry of PTCA atthe National Heart, Lung, and Blood Institute. Thisregistry receives voluntary information from centersperforming PTCA within the United States and reportson successes, measurements, complications, and fol-lowups. The availability from the Interim Registry ofinformation showing outcomes which are consistent-ly comparable with alternative, invasive proceduresnevertheless has a powerful influence in predisposingphysicians to accept PTCA. It also creates pressureson third-party reimbursers to extend coverage to a pro-cedure whose efficacy does not appear to be in doubt.

Table E-2.—Reported Clinical Trials of PTCA

TrialPublication date dates Observations + % Comments

1977 — — — Gruntzig: animals andpost mortem studies

1978 — — — 3 post mortem and 1animal study reporteda

1978 9/77-12/77 7 86 % Gruntzig1978 1977-78 29 70% Gruntzig, et al.1979 ● 65 600/0 Gruntzig, et al.

United States andSwitzerland, 5 centers

1979 1/78-7179 50 680/0 Gruntzig, et al.1979 9/77-10/79 163 620/o GruntzigKey: ‘Unstated or unclear.

acirculation 5 7 (supPI 2)80, 1978

—. —


Considerable uncertainty exists over the complete-ness of the registry’s data base. The cumulative numberof procedures reported to the registry is shown in tableE-3, As of May 1981, the registry had 100 centers re-porting to it (19); however, USCI and Advanced Cath-eter Systems (ACS), the two manufacturers of thecatheters used to perform MCA, estimate the numberof purchasers (i.e., centers) to be between 300 and 400.FDA’s release of the catheter from IDE status in late1980 made it available to users other than reportingcenters. According to the manufacturers, therefore, theregistry’s data undercount the number of proceduresactually performed.

Third-Party Reimbursement

It was initially thought that PTCA was not reim-bursed by third-party carriers. This is widely believedby numerous professionals in the field. However, sub-sequent information indicated that this was not thecase and that the situation is as follows.

First, Medicare, MediCal, Blue Cross, and BlueShield do not reimburse for PTCA. The procedure isviewed as being investigational, and the primary reser-vation is the lack of data on clinical effectiveness. Datacoming out of the Interim Registry and FDA’s deci-sion are viewed as important precursors to recogni-tion from the aforementioned carriers (7,20).

Second, grants and free service have covered thecosts in some institutions. For example, at StanfordUniversity Hospital, PTCA has been recognized as a“research procedure. ” Thus, either research grantshave covered the costs, or the service has been pro-vided free (l). In 1981, PTCA was being reviewed todetermine whether it could be called a “billable pro-cedure, ” in which case third-party reimbursementwould be sought to the extent that it is available.

Third, most private carriers* reimburse for PTCAto some extent, either knowingly or otherwise. In some

● Excluding Blue Cross and Blue Shield.

Table E-3.– Reported Use of PTCA

Cumulativenumber of Number of

Date Procedures reported centers

June 1979 ......., . . . 61 5December 1979 . . . . . . 200 —June 1980 . . . . . . . . . . . 60Fall 1980—FDA released catheter from list of

investigational devicesOctober 1980 . . . . . . . . 504 —February 1981 . . . . . . . 1,016 —

May 1981 . . . . . . . . . . . 1,800 100SOURCE S. Mull in, Interim Registry of PTCA, Cardiac Disease Branch, National

H e a r t , L u n g , a n d B l o o d I n s t i t u t e .

cases, the charges for PTCA are “buried” in other,chargeable, items, such as catheter laboratory charges,coronary angiograms, etc. (7,19,26). More often, itseems, the procedure is openly identified as PTCA andcharged accordingly.

The latter is the procedure used, for example, bySteven Myler of San Francisco, who has an activepractice in PTCA. * The patient is billed and pays forthe procedure. The patient is then reimbursed for ex-penses by the insurance carrier to the extent designatedwithin the policy. Insurance coverage appears to havebegun in 1980 or 1981, though it is difficult to deter-mine exactly when. St. Mary’s Hospital, in whichMyler does his work, indicates that third-parties havenot refused payment, though some have questionedthe new procedure, The only difficulty is in being reim-bursed for the procedure itself. The hospital visits arereimbursed without question.

A number of major health insurance carriers werecontacted. One carrier, New York Life Insurance, said“anything ordered by an M.D. is covered, unless spe-cifically excluded” (21). A second carrier, Mutual ofOmaha, stated that their major catastrophic insurancecovered “treatment by a physician or surgeon” and“service by a radiologist for diagnosis of treatment”(20).

The fact that PTA is a familiar procedure withproven efficacy makes PTCA more acceptable to car-riers. (HCFA has approved payment for the PTA pro-cedure when used in lower extremities. )

Additionally, FDA’s decision to release the catheterfrom the investigational devices list is viewed as im-portant. With various restrictions due to an IDE status,health care facilities are reticent to do PTCAs Dur-ing the investigational stage, physicians keep informedof the development of the procedure. Once it is clearedby FDA, acceptance is fairly rapid (see fig. E-2) (20).

Market Factors

Of the approximately 100,000 CABG proceduresperformed each year, approximately 10 percent ofthese could be replaced with PTCA. Thus, an estimateof the maximum annual demand of the primary marketfor PTCA is: 100,000 X 10 percent = 10,000 pro-cedures per year. Additional demand could be realizedif indications for the procedure were broadened.

Currently, there are two manufacturers of the spe-cial catheter used in PTCA: USCI of Massachusettsand ACS of Santa Clara, Calif. It seems unlikely thatadditional companies will enter the field. The marketis limited in size. Also, the technological barriers toentry are substantial, By 1981, USCI’s catheter had

● Ann, of Dr. Myler’s office, personal communication.

—


Figure E-2.—Reported Cases of PTCA

June Sept. Dec. Mar. June Sept. Dec. Mar. June

1979 1980 1981

SOURCE: S, Mullin, Interim Registry of PTCA, Cardiac Disease Elranch, National Heart, Lung, and Blood Institute.

been released from IDE status, whereas ACS’s catheter Table E-4.—Cost Comparison: PTCA, CABGhad not. ACS expected to receive premarketing ap-proval within 6 months. CABG . . . . . . . . . . . . . . . . . . . . . . . ... ... ... ... .. .$20,000

PTCA:The price to the patient of a CABG is quoted as

$20,000 at Stanford University Hospital. The price ofPTCA is $3,000 (see table E-4). The source of the costof PTCA was unable to state unequivocally whetherthe price included a backup team for CABG. However,the patient would be charged $3,000.

Some percent of those patients undergoing PTCAeventually have CABG performed anyway. A true costcomparison should factor this in (see table E-5). Thecost savings are such that it would be necessary for85 percent of the patients who receive PTCA to alsohave CABG before the cost savings advantage ofPTCA would be nullified.

Supplies:Catheter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . $387Other cathetersMedical supplies . . . . . . . . . . . . . . . . . . . . . . . . . . . 500Other . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

$980Personnel: Clerical, technician, two cardiac

cath lab nurses (3 hours for procedure) . . . . . $74Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

$89Coronary hemodynamic (partial charge) . . . . . . . . . $75Indirect costs, departments and hospital . . . . . . . . 137

$1,281Adjusted for inflation, 15°/0 . . . . . . . . . . . . . . . . . . . . 1.15

$1,473Profit markup, 100/0 . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.10

$1,620Physician fees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1,275

Total cost (price to patient) . . . . . . . . . . . . . . . . . . $2,895SOURCE: P Berry, Cardiac/EKG Department, Stanford Univerwty, 1981.


Table E-5.—Expected Cost and Break-Even

Expected costAssume: 10 to 15°/0 of PTCAs are followed up with CABGs.Expected cost per patient:

(3,000) (1.00) + (20,000) (0.10) = $5,000(3,000) (1.00) + (20,000) (0.15) = $6,000

Expected cost per patient = $5,000 to $6,000.Break-even:$3,000 + $20,000 (x) = $20,000

20,000 (x) = 17,000X = 0.85

where X = percent of patients having PTCAwho also need CABG.

Sawi’s References1. Berry, P., Business Manager, Cardiac/EKG Department,

Stanford University Hospital, personal communication,June 1, 1981.

2. Brahme, F., et al, “Evaluation of TransluminalRecanalisation of the Femoral Artery,” Acta Chir. Scand.135:679, 1969.

3. Colapin, R. F., et al., “Percutaneous TransluminalAngioplasty of Peripheral Vascular Experience: A TwoYear Experience, “ in Cardiovascular and IhterventionalRadiology, E. Zeitler (cd. ) (Berlin, Heidelberg: Springer-Verlag, 1980).

4. Dotter, C. T., Oregon State University, Portland, Oreg.,personal communication, Apr. 24, 1981.

5. Dotter, C. T., ‘Tranduminal Angioplasty, ” in Angi-graphy, vol. 2, 3d cd., H. L. Abrams (cd. ) (Boston: Lit-tle Brown & Co., 1980).

6. Dotter, C. T., and Judkins, M. P., “Transluminal Treat-ment of Arteriosclerotic Obstruction, ” Circulation,30:654, 1964.

7. Finklestein, S., Alfred P. Sloan School of Management,Massachusetts Institute of Technology, personal com-munication, May 29, 1981.

8. Fogarty, T. J., et al., “A Method for Extraction ofArterial Emboli and Thrombi, ” Surg. & Gynec. Obstet.116:241, 1963.

9. Grossman, N., Product Manager, Cardiac Catheters,USCI, Billerica, Mass., personal communication, May29, 1981.

10. Gri.intzig, A. R., “Coronary Percutaneous TransluminalAngioplasty: Preliminary Results, ” Circulation 58(%PP1. 2):56, 1978.

11. Gfintzig, A. R., “Coronary Transluminal Angigraphy,”Circulation 56 (Suppl. 3):84, 1977.

12. Gfintzig, A. R., “Percutaneous Transluminal CoronaryAngioplasty,” unpublished document obtained from H.Abrams, Stanford University, April 1981.

13. Grtintzig, A. R., “Percutaneous Transluminal CoronaryAngioplasty—Present State of the Art, ” Circulation60:1031, 1979.

14, Griuntzig, A. R., “Transluminal Dilatation of Coronary

15

16

Artery Stenosis, ” letter, Lancet 1:263, 1978.Griintzig, A. R., et al., “Nonoperative Dilatation of Cor-onary Artery Stenosis: Percutaneous Transluminal Cor-onary Angioplasty, ” N. &g. J. Med. 301:61, 1979.G&ntzig, A. R., et al., “Percutaneous Transluminal Cor-onary Angioplasty: A Cooperative Study, ” Am. J. Car-diol. 43:384, 1979.

17.

18.

19.

20.

21.

22.

23,

24,

25,

26,

27.

28.

29.

30.

31.

Kanne, M., Executive Vice-President and Sales Manager,Cook, Inc., Bloomington, Ind., personal communica-tion, May 29, 1981.Katzen, B. T., and Chang, J., “Percutaneous Trans-Iuminal Coronary Angioplasty With Grtintzig BalloonCatheter,” Radiology 130:623, 1979.Mullin, S., Interim Registry of PTCA, Cardiac DiseaseBranch, National Heart, Lung, and Blood Institute,Bethesda, Md., personal communication, May 14, 1981.Mutual of Omaha, Medical Coverage Representative,personal communication, May 29, 1981.New York Life Insurance, Coverage Representative, per-sonal communication, May 29, 1981.NordenstrGm, B., “Balloon Catheters for PercutaneousInsertion Into the Vascular System, ” Acta Radio)ogica57:411, 1962.Rapaport, E., “Percutaneous Transluminal CoronaryAngioplasty—Editorial, ” Circulation 60(5):969, 1979.Schoop, W., et al., “Early and Late Results of PTD inIliaca Stenosis, ” in Percutaneous Vascular Recanaliza-tion: Technique, Application, Clinical Results, E. Zeitler,et al. (eds. ) (Berlin, Heidelberg: Springer-Verlag, 1978).Seidlinger, S. I., “Catheter Replacement of the Needlein Percutaneous Angiography, ” Acta Radiological 39:368,1953.Simpson, C., Advanced Catheter Systems, Santa Clara,Calif., personal communication, May 22, 1981.Van Andel, G. J., Percutaneous TransluminaJAngiopZas-ty: The Dotter Procedure (Amsterdam-Oxford: Excerp-ta Medica, 1976).Wierny, et al., “Long-Term Results in 100 ConsecutivePatients Treated by TransluminaI Angioplasty” Radi-ology 112:543, 1974.Zeitler, E., “Percutaneous Dilatation and Recanalizationof Iliac and Femoral Arteries, ” in Cardiovascular andZnterventional Radiology, E. Zeitler (cd. ) (Berlin,Heidelberg: Springer-Verlag, 1980).Zeitler, E., et al. (eds. ), Percutaneous Vascular Recon-struction: Technique, Application, Clinical Resufts(Berlin, Heidelberg: Springer-Verlag, 1978).Zeitler, E., et al., “Treatment of Occlusive Arterial Dis-ease by Transluminal Catheter Angioplasty, ” Radiology99:19, 1971.

Maternal Serum Alpha= Fetoprotein

The Federal involvement with MSAFP for detectionof fetal neural tube defects is a case study that il-lustrates successful technology assessment monitoringand interagency coordination The determination of thelevel of alpha-fetoprotein in maternal serum is the firststep in a sequence of diagnostic tests used to screenand diagnose fetal neural tube defects. It aids in detect-ing two types of defects, anencephalia (absent or un-developed brain) and open spina bifida (failure of thespine and overlying skin to close over the spinal cord),which together affect 3,000 to 6,000 newborns in theUnited States each year. Followup procedures, for use


when results of screening are abnormal, include repeatserum testing, ultrasonography, and amniocentesis(108).

In late 1978/early 1979, FDA was on the verge ofapproving a 2-year interim period for widespreadusage of MSAFP. However, a special interest group,the spina bifida parents, questioned the quality of theFDA data and the impending diffusion of MSAFP. TheCenters for Disease Control (CDC) also became con-cerned over manufacturers’ ability to consistently pro-duce quality components for a marketable kit, as wellas over physicians’ readiness to work in close coopera-tion with their patients. Compounded further by eth-ical and reimbursement issues, CDC sent a formalmemorandum to the Office of Health Research, Sta-tistics, and Technology Director, enumerating the con-cerns with the technology. As a result, the MSAFPscreening test was discussed at a meeting of theTechnology Coordinating Committee of the Depart-ment of Health and Human Services; and NCHCT wasasked, through the committee, to coordinate develop-ment of departmental policy (108,315).

In November of 1979, former Secretary Harris wasbriefed, at her request, on departmental activitiesrelating to MSAFP. NCHCT coordinated the briefing.It was agreed that the Public Health Service (PHS),through CDC, should conduct a controlled epidemio-logic field study to obtain needed clinical data onMSAFP and to supplement FDA’s postmarketing sur-veillance and data collection program. CDC’s protocolwas subsequently reviewed by the Technology Coor-dinating Committee. In July of 1980, NCHCT andFDA cosponsored a national educational conferenceon MSAFP. By November of 1980, regulatory pro-posals had been readied for publication (119).

In the November 7, 1980, issue of the Federal Reg-ister, FDA published proposed regulations to restrictthe sale, distribution, and use of alpha-fetoprotein testkits used in detecting fetal neural tube defects. Also,in that same edition of the Federal Register, CDC andHCFA published jointly proposed regulations pertain-ing to quality control and proficiency testing forclinical laboratories engaged in alpha-fetoproteintesting. Public hearings on these regulatory proposalswere held on January 15-16, 1981. Testimony was pre-sented by some 40 to 50 individuals at these hearings,and approximately 650 written comments were re-ceived by FDA and CDC/HCFA subsequent to thepublication of the proposed rules. These responseswere analyzed and assessed by FDA and CDC/HCFA.FDA found that it had several options in regard to therestricted or unrestricted release for marketing of thealpha-fetoprotein test kits; these were under review asof January 1982 by Arthur Hayes, the FDA Commis-sioner (120).

Hemodialysis and Kidney TransplantSurgery*

by Katherine R. Jones, Ph.D. (Candidate)School of Education, Stanford University

Background Information

Hemodialysis and kidney transplant surgery are twoalternative forms of therapy for chronic renal failure.Permanent or chronic or end-stage renal disease(ESRD) occurs when an individual irreversibly losesa sufficient amount of kidney function so that life can-not be sustained without treatment intervention.Chronic renal failure may be caused by any of a num-ber of separate diseases (glomerulonephritis, pyelon-ephritis, polycystic kidney disease, hypertension,diabetes mellitus, and others), but the uremic or ESRDstate is their final outcome.

Hemodialysis is a treatment that involves using amachine—the artificial kidney—to achieve the vitalfunctions previously performed by the kidneys. A pa-tient undergoes the treatment from 4 to 6 hours a day,two or three times a week. The treatment can be givenin hospital-based dialysis units, freestanding units (for-profit or not-for-profit), or in the patient’s home. Afast growing substitute for hemodialysis is continuousambulatory peritoneal dialysis (CAPD), a techniquethat uses the peritoneum (lining of the abdominal cavi-ty) to cleanse the blood of its impurities.

The alternative treatment to dialysis is kidney trans-plantation, which is performed with kidneys fromeither living related donors or cadaveric donors. Thebest kidney survival rates are achieved with livingrelated donors, especially siblings.

DEVELOPMENT OF THE TECHNOLOGY

Development of the hemodialysis technology beganas early as 1913, when Abel, Rowntree, and Turnerperformed the first dialysis in animals at the JohnsHopkins Medical School (8,27). This team built theirown dialyzing system and coined the expression “ar-tificial kidney.” On February 28, 1926, Haas per-formed the first hemodialysis in a human being, usingHirudin (prepared from leech heads) as the anticoagu-lant (9). The development of the anticoagulant Heparingreatly enhanced the dialysis procedure, as did themarketing of cellophane for use as the artificial dialyz-ing membrane. Because of an inability to repeatedlyaccess the bloodstream of people suffering fromchronic renal failure, however, the use of hemodialysis

● NOTE: Reference citations for Katherine Jones’ case study on hemodialysisand kidney transplant surgery appear on p. 183.


was limited to patients with acute, reversible kidneyfailure.

In 1943, Kolff developed the first practical modelfor human hemodialysis, building an artificial kidneyusing a rotating drum (27). It took 2 more years beforeKolff achieved his first success in treating acute renalfailure patients with hemodialysis. Independently ofKolff, Alwall in Sweden and Murray in Canada werealso developing types of hemodialysis machines, andall three published their experiences at about the sametime (9,27).

Use of the artificial kidney for patients with acuterenal failure continued until 1960. In that year, acritical technological advance was made when Quin-ton and Scribner reported the use of a subcutaneousarteriovenous shunt (a plastic tube connected to anartery and a vein in the arm or leg), which allowedrepeated access to the circulatory system and thus per-mitted continuous dialysis treatments (27). After thisimportant development, technological advances weremade in the areas of improved blood access devices,dialyzing membranes, dialysis machines, and types ofartificial kidneys (8,9).

The first kidney transplant was performed at Har-vard by Hufnagal and Hume in 1947 (27). A cadaverkidney was transplanted into the antecubital fossa(area in front of the elbow) of a young woman dyingfrom acute renal failure. The kidney functioned for 2days, lasting long enough for the patient to regain herown renal function. From 1951 to 1953, Thorn andMerrill referred several patients to Hume, who per-formed several transplants. They watched patients ex-perience a reversal of their uremic state only to ex-perience a rejection of the kidney in a few days (27).

In February of 1953, the first success was achievedby Hume and his associates. A person survived witha functioning kidney transplant for 5 months and 25days and demonstrated the potential of the procedure(21). This case also marked the end of experimentaltransplantation without the use of immunosuppressionto prevent graft rejection (21,27).

In 1954, the first transplant between monozygotictwins was performed by Murray and his associates inBoston (2). This case demonstrated that monozygotictwins were, indeed, immunologically identical. Within5 years, this group had performed eight transplantsbetween twins and had perfected the surgical techniqueof retroperitoneal iliac fossa (groin area) placement ofthe graft (2). The initial transplant recipient lived for8 years before dying of myocardial infarction (21,27).

Clinical and animal experiments with different formsof immunosuppression occurred in 1958 and 1959.Total body irradiation usually proved to be fatal andwas soon abandoned (2). Schwartz next introduced

drug-induced immunological tolerance, first using mer-captopurine, then switching to a superior derivativecalled Imuran, a drug still in use today (14,27). Steroidsjoined Imuran in treatment of graft rejection in 1962(2). In 1963, Terasaki began using serotyping tech-niques to select immunologically favorable kidneydonors (2). New tissue-matching techniques, organ ac-quisition and preservation procedures, and immuno-suppressive drugs have since been introduced, al-though improvement is still needed in long-term organsurvival rates (21,27).

DIFFUSION OF THE TECHNOLOGY

In the early 1960’s, hemodialysis and kidney trans-plant became accepted as life-extending therapies forvictims of chronic renal failure. In America, much ofthe research was initially supported by the John A.Hartford Foundation and later by the Artificial Kid-ney/Chronic Uremia Program (AKCUP) of the Na-tional Institutes of Health (NIH). AKCUP in the Na-tional Institute of Arthritis, Metabolism, and DigestiveDiseases (NIAMDD) was founded in 1965 with a con-tract research program to build a better artificialkidney. This program was mandated by the House andSenate Appropriations Committees 1 year after the ar-tificial heart program, although the artificial kidneywas more developed at the time (27).

The development of hemodialysis as a mode of ther-apy posed complications for NIH. NIH found it dif-ficult to support Scribners development of clinical ap-plications of the artificial kidney and was neverprepared to do so on the scale he requested (2). TheNIH orientation toward biological and biochemicalprocesses led to a preference to fund research onkidney disease etiology, not the clinical applicationsof that research. Funding for transplant research wasnot as problematic, since those involved with immuno-logical research were well known as basic researchers(27).

In 1964, the Senate Appropriations Committeestated that PHS had the authority to provide demon-stration and training funds for artificial kidney pro-grams. In 1965, PHS established the Kidney DiseaseControl Program (KDCP), which funded 14 communi-ty treatment centers around the country and demon-strated the organizational feasibility of dialysis invarious settings. Although these contracts were grad-ually phased out beginning in 1968, many PHS-fundeddialysis centers became nationally prominent hemo-dialysis provider institutions (27).

In 1969, KDCP became part of the Regional MedicalProgram (RMP), and the emphasis shifted from dem-onstrations of feasibility to the building of dialysis


capacity (27). This was accomplished through cen-tralized funding and policy control in Washington, anddecentralized funding of facilities through the RMPagencies (27).

A very significant role in the diffusion of dialysistechnology was played by the Veterans Administra-tion (VA). In 1963, 2 years before the initiation ofAKCUP and KDCP, VA announced its intention toestablish dialysis centers in 30 VA hospitals (27). It pro-ceeded to do so over the next several years. Dialysisand transplantation were provided for all qualifiedveterans with chronic kidney failure, whether or notservice-connected. Public Law 89-785 (Nov. 7, 1966)provided that nonveterans could also receive suchservices from VA hospitals if facilities were available,but that VA had to be reimbursed the full cost of serv-ices rendered under agreements with other hospitals(27,31).

By 1971, VA had initiated a home dialysis program,had opened its first home training unit, and had alsoinitiated satellite dialysis. By 1972, VA was dialyzing25 percent (979 patients) of the Nation’s dialysis pa-tients in 44 treatment centers and another 26 patientsin branch centers. As of March 1973, the VA systemhad 501 dialysis beds (including 123 for home train-ing) and 10 hospitals that were operating branch dial-ysis centers. VA had also provided backup medicalservices for 766 patients being dialyzed at home.Thirty-three VA hospitals reported a total of 327transplants performed in 1972.

The activity in PHS and VA related to hemodialysisand transplantation prompted the Government to con-duct a high-level policy review of the situation. TheBureau of the Budget established the Gottschalk com-mittee in 1965 to review the implications of therapyfor ESRD for the entire Nation (27). In 1967, the Gotts-chalk committee (12) declared that hemodialysis andtransplantation were acceptable forms of therapy andrecommended that a national treatment benefit pro-gram be established by amending title XVIII of theSocial Security Act. No legislation was enacted as aresult of this report, partly because of the simultaneousrelease of a PHS report that emphasized research andprevention rather than treatment.

Gottschalk (12) estimated that in 1962, one out ofevery five patients dying from chronic uremia wasmedically suitable for dialysis and transplantation. Ofthe 7,000 new renal patients in 1968 who would besuitable for treatment, transplants were available forapproximately 450 and chronic dialysis for approx-imately 550 (12). The treatment technology was avail-able but was prohibitively expensive, In 1965, the costsfor dialysis and transplant were as follows (12):

●

●

●

home dialysis: average $6,000 per year, range$3,750 to $9,800;in-center/hospital dialysis: average $10,000 peryear, range $8,400 to $21,000; andtransplant: average $13,300 surgery and recoveryplus $200 to $1,000 per year, range $10,000 to$22,000.

The eight primary sources of funding for dialysis andtransplantation prior to the establishment of the ESRDprogram were the following (7):

1.

2.

3.

4.

5.

6.

7.

Medicare. —This program helped finance treat-ments for persons 65 and over (but few patientsthis age were selected for treatment).Medicaid. —Under this program administered bythe Social and Rehabilitation Service, the Statescould assist in paying for kidney disease treat-ment. Eligible individuals were those who re-ceived public assistance under certain titles of theSocial Security Act, or those persons in certainStates whose income and resources were insuffi-cient to meet medical needs. Services variedamong the States.Vocational Rehabilitation Act. —Using Federalgrants to the States, this program helped 300 to400 individuals with kidney disease annually ata cost of about $1 million.Comprehensive Health Planning (authorizationexpired June 30, 1974). —According to the De-partment of Health, Education, and Welfare,RMP of the Health Resources Administrationwas, as of April 1975, investing $4.8 million ingrants to develop chronic kidney disease treat-ment services reimbursable under the Medicareprogram. Funds were primarily aimed at startupcosts of cadaver kidney procurement systems andspecialized laboratory services.Military. —The Army, Navy, and Air Force pro-vided a limited number of dialysis machines forhome and center programs through the CivilianHealth and Medical Program of the UniformedServices.Research, —NIAMDD was the NIH componentprimarily responsible for supporting kidney dis-ease research. The National Institute of Allergy

and Infectious Diseases (NIAID) of NIH sup-ported research on the infectious and immuno-logical aspects of kidney disease and was themajor source of funds for transplantation.State and Private Involvement. —A General Ac-

counting Office (GAO) survey of 14 States in1973 found 8 States that were appropriating fundsto directly assist patients with kidney disease.Wide variation existed between the States: One


8.

State legislature appropriated about $4,400 per100,000 population, while another appropriatedabout $11,000 per 100,000 population. Two ofthe States operated their own dialysis facilities.Private Sources. —Private sources included out-of-pocket payments, savings withdrawals, pri-vate health insurance, the National Kidney Foun-dation, local community organizations and funddrives, and employer contributions.

ESRD Program

From the mid-1960’s until 1972, a long policy debateoccurred over who should be responsible for fundingtreatment of patients with chronic renal failure. Theissue was publicized in the media, and the existenceof “death committees” (groups of physicians or healthprofessionals who decided who would be dialyzed andwho would be allowed to die) became well known (18).There was mounting pressure for the Federal Govern-ment to take action that would relieve the patients andother payers of the expensive burden of this lifesav-ing technology.

The debate culminated in 1972 with the passage ofsection 2991 of Public Law 92-603 of the Social Securi-ty Amendments of 1972, a law that extended coveragefor renal disease treatment to over 90 percent of thepopulation (30). Medicare eligibility began in the thirdmonth after the month in which a course of dialysiswas initiated and ended in the 12th month after themonth in which a person had a functioning kidneytransplant. Factors that led to the congressional deci-sion to pay for ESRD treatment included a recogni-tion that the alternative to life sustainment by dialysiswas death, that ESRD treatment was very expensive,and that there occurred 7,000 to 10,000 uremic deathsa year because of the limited availability of dialysisfacilities.

HCFA assumed responsibility for the ESRD programin 1978. HCFA prescribes standards for treatment byits regulations and approves payments for servicesthrough local insurance companies or other interme-diary agencies. HCFA is also responsible for the quali-ty of care that patients receive and exercises thatresponsibility through existing National, regional, andState agencies.

CHANGES IN THE PATIENT POPULATION

There have been changes in the number of patientsreceiving transplants and the proportion of patientsreceiving home dialysis since institution of the ESRDprogram.

Transplants.—Between 1963 and 1972, the numberof kidney transplants had been increasing steadily,

going from 163 to 1,993 a year (7). After 1972, thenumber grew at a slower pace, and it plateaued in 1975(10,26) (see table E-6).

The reasons for the decline in transplants includedlack of improvement in graft success rates (althoughpatient survival rates had improved), decreased donorpool due to smaller families, and financial disincen-tives in the Medicare regulations. Benefits ended the12th month after transplant surgery. If a person losthis or her kidney after this time period, that personhad to undergo another 3-month waiting period beforeMedicare began paying for the resumed dialysis treat-ments. The patient also had to pay the costs associatedwith transplant failure, In 1975, GAO (7) recommend-ed that the waiting period and associated disincentivesfor transplant, a less costly treatment modality, beeliminated from the law. These recommendations wereimplemented in 1978.

Hemodialysis.—The number of patients receivinghemodialysis grew rapidly after 1970. From 11,000dialysis patients in 1973, the program expanded toabout 50,000 dialysis patients in 1980 (see table E-7).This growth occurred primarily as a result of changesin patient selection criteria. Originally, selection waslimited to patients 15 to 45 years of age who were ingood health apart from their renal disease. Today,there is essentially one criterion for acceptance—

Table E-6.—Number of Kidney TransplantsPerformed in the United States, 1951.79

Year Number a

1951 -62 . . . . . . . . . . . . . . . . . . . . . 751963 . . . . . . . . . . . . . . . . . . . . . . . 1631964 . . . . . . . . . . . . . . . . . . . . . . . 2391965 . . . . . . . . . . . . . . . . . . . . . . . 3051966 . . . . . . . . . . . . . . . . . . . . . . . 3381967 . . . . . . . . . . . . . . . . . . . . . . . 4481968 . . . . . . . . . . . . . . . . . . . . . . . 6761969 . . . . . . . . . . . . . . . . . . . . . . . 8381970 . . . . . . . . . . . . . . . . . . . . . . . 1,091 (1,460)1971 . . . . . . . . . . . . . . . . . . . . . . . 1,616 (2,909)1972 . . . . . . . . . . . . . . . . . . . . . . . 1,993 (2,852)1973 (Medicare coverage) . ....3,0171974 . . . . . . . . . . . . . . . . . . . . .. .3,1901975 . . . . . . . . . . . . . . . . . . . . .. .3,7301976 . . . . . . . . . . . . . . . . . . . . .. .3,5041977 . . . . . . . . . . . . . . . . . . . . .. .3,9731978 . . . . . . . . . . . . . . . . . . . . .. .3,9491979 . . . . . . . . . . . . . . . ........4,271

Percentagechange

117.3”/046.627.610.832.550.924.030.248.123.351.4

5.716,9–6.113.4–0.6

8.2aNumber~ in parentheses reflect discrepancies in the literature.

SOURCES: Comptroller General of the United States, Treatment of ChronicKidney Failures: Dialysis, Transplant Costs, and the Need for MoreVigorous Efforts, 1975; Office of Special Programs, Office of End-Stage Renal Disease, Health Care Financing Administration, r3rd-Stage Renal Disease Second Annual Report to Congress, Fkcal YearIW, 1980; and E. A. Freedman, et al., “Pragmatic Realities in UremiaTherapy,” N. Eng J. Med. 298:388, 1978


Table E-7.–ERSD Patient Population, 1972.80

Number of Percentage of Percentage ofhemodialysis patients on Number of patients over Number of

Year patients home dialysis a patients on CAPD b 65 years dialysis facilities

1972 ., . . . . . . . 10,000 400/o — NA NA1973 . . . . . . . . . 11,000 35.90/o — NA NA1974 . . . . . . . . . 18,875 32.70/o — 50/0 6641975 . . . . . . . . . 22,000 280/o — 11% NA1976 . . . . . . . . . 30,131 23.70/o (13°/0) — 17 ”/0 7 0 01977 . . . . . . . . . 3 2 , 4 3 5 11.56% (20%) — 2 0 % 7611978 . . . . . . . . . 3 6 , 4 6 3 12.42 ’10 8 1 9 % 8 5 01979 . . . . . . . . . 4 5 , 5 6 5 130 /o ( 10%) 1 , 8 0 0 N A 9 5 01980 . . . . . . . . . 5 0 , 0 0 0 N A 3 , 0 0 0 N A 1 , 0 0 0

afQumber~ i n P a r e n t h e s i s reflect conf l ict ing reports in the l i terature.

bMedicare coverage began in 1978NA = no t ava i l ab le .

SOURCES’ Off ice of Special Programs, Off ice of End-Stage Renal Disease, Health Care Financing Administrat ion, End-StageRenal Disease Second Annual Report to Congress, Fiscal Year 1980, 1980; R. Rettig, Implementing the End-StageRena/ Disease Program of Medicare, 1980; and Health Services Administration, End-Stage Renal Disease Medical/formation System–Facility Report Number 1, 1976.

disabling uremia. A large proportion of the dialysispopulation is either very young or very old and suf-fers from other serious diseases, such as liver disease,cancer, and diabetes. Table E-7 also shows that therehas been a fairly rapid decrease in the proportion ofhome dialysis patients since the beginning of the ESRDprogram.

When the Social Security Amendments of 1972 werepassed, 40 patients per million were receiving long-term hemodialysis treatment in the United States,almost entirely under the auspices of nonprofit orga-nizations (24). The number of patients now receivingdialysis treatments in the United States exceeds 200 permillion population, an eightfold increase, and is thehighest in the world (24).

The composition of patients receiving dialysis treat-ment in the United States has also changed since 1972.The average age of the maintenance dialysis popula-tion has increased, The mean age of dialysis patientsrose from 42 in 1970 to 50 in 1977 (18,31). Americanswell past retirement age may be placed on dialysis. Asurvey 01 European dialysis centers in 1977 revealedthat 70 percent had no age limit to dialysis; 22 per-cent as a general rule excluded patients over 65; and8 percent excluded patients over age 55 (l).

CHANGES IN THE USE OF THE TECHNOLOGY

The changes in the treated renal patient populationhave occurred mainly as a result of increased avail-ability of funding for dialysis and transplantation. Theincreasing number of new patients presenting eachyear, increasing average age of new patients, and in-creasing number of patients with serious complicationshave contributed to the decreasing use of home dialysisand transplantation and to an overall rise in morbidi-ty and mortality.

There has been much national concern expressedover the declining proportion of home dialysis pa-tients, since significantly lower costs are associatedwith home treatment, especially after the first year.The Medicare regulations themselves included disin-centives for home dialysis (3,7). For example, theMedicare regulations did not require centers to pro-vide training programs for home dialysis (7). Instead,they required more out-of-pocket costs for home dial-ysis supplies and equipment and did not provide reim-bursement for the services of a home dialysis assistantnor for the effort involved in renting equipment, order-ing supplies, and other bookkeeping requirements.Home patients incurred additional costs for homemodification and higher electric and water bills (13,19).

Some of the movement back to facility dialysis wasthe result of the stresses on family life caused by homedialysis. Opponents of home dialysis have stated thatmany patients were initially placed on home dialysissolely because of limited funds for facility dialysistreatment and that Medicare has removed these finan-cial barriers to the preferred treatment setting. Otherfactors in the home dialysis/facility dialysis controver-sy include the personal philosophy of the physicianor hospital treating the patient, increased age and mor-bidity of dialysis patients that reduce their suitabilityfor home treatment, and the impact of facility propri-etary-status on the location decision outcome.

Proponents of home dialysis have argued that thecosts of home dialysis are lower than those for facili-ty dialysis, that quality of life is improved as patientsare more independent and able to work, and that mor-bidity and mortality rates are lower (5). Using datafrom 1972, the NIH National Dialysis Registry re-ported a home dialysis 3-year mortality rate of 21.4percent and a facility dialysis 3-year mortality rate of28.6 percent (13). Using 1976 cumulative data, it re-


ported the following annual death rates: home dialysis,6.7 percent; freestanding-unit dialysis, 7.5 percent;hospital-based-unit dialysis, 10 percent.

GAO in 1977 reported that mortality rates for dial-ysis patients were unavailable to it and that GAO wastherefore unable to compare mortality rates of home-treated v, center-treated dialysis patients (6). Neitherthe Medicare billing system nor the ESRD medical in-formation system recorded morbidity data (4). TheESRD amendments of 1978 mandated HCFA to col-lect such data, and information on hospital admis-sions, average length of stay by disease category, sur-vival rates, and program costs associated with alter-native treatments of ESRD was to be reported in 1981.

GAO was able to collect cost information and in1975 reported the following data based on 1972-73 costor charge information (7):

Type of dlalysis Average yearly cost Range Per treatment costHome $14,900-first year $9,300-$22,200 $96

$7,000 -following years $3,900-$10,300 $46Freestanding $27,600 $16,440-$41,003 $203Hospital-based $30,500 $11,500-$49,100 $202

NIH cost estimates (excluding physician fees) based on1973 cost data (5 dialysis centers) were as follows (10):

Type of dialysis Average yearly cost Per treatment costH o m e $15,000-first year $33-$66

$6,500-following yearsFreestanding $16,520 $100-$116Hospital-based. ... .. $24,738 $146-$259

In 1973, the Medicare reimbursement rate was $150for center dialysis and $50 for home dialysis, both in-cluding physicians’ fees and assuming 156 treatmentsa year.

In 1977, GAO estimated the following costs forhome dialysis (6):First yearEquipment . . . . . . . . . . . . . . . . . . . ., ., . . . . . . . . . . . . . $6,800Training—24 treatments @$158 . . . . . . . . . . . . . . . . 3,792

Physician fees (training) . . . . . . . . . . . . . . . ., . . . . . . 500Physician fees (supervision)–

12 months @$140 ... ... ., . . . . ., . . . . . . . . . 1,680Backup dialysis–16 treatments @$138 . . . . . . . . . . 2,208Supplies and equipment–116 treatments @$55. . . . . 6,380Reasonable charges covered by Medicare . . . . . . . . . . . $21,360

Second yearBackup dialysis–19 treatments @$138 . . . $2,622Physician fees–12 months @$140 . . . . . . . . . . . . . . . . . 1,680Supplies and equipment–137 treatments @$55. . . . . . 7,535Reasonable charges covered by Medicare . . . . . . . $11,837

A study by Roberts, et al. (29), estimated the fol-lowing costs for dialysis and transplantation in 1980:H o m e d i a l y s i s — f i r s t y e a r .$22,760H o m e d i a l y s i s – r e m a i n i n g y e a r s ( p e r y e a r ) . , 13,237In-center dialysis (per year) 24,800C a d a v e r t r a n s p l a n t – f i r s t y e a r 23,400C a d a v e r t r a n s p l a n t — s e c o n d y e a r . 3,000C a d a v e r t r a n s p l a n t — t h i r d y e a r 1,500Cadaver transplant–remaining years (per year) 750

Related donor transplant—first year 20,700R e l a t e d d o n o r t r a n s p l a n t — s e c o n d y e a r 1,500Related donor transplant–remaining years (per year) 500G r a f t r e j e c t i o n ( p e r y e a r ) 9,000

For the years 1976 through 1979, the ESRD program(23) estimated the following kidney acquisition costs(donor surgery costs):

Year Average Low High1976. . . . . . $4,223 $1,003 $13,1971977 . . . . . . 4,690 1,000 15,0001978 . . . . . . 5,790 1,000 12,6831979 . . . . . . 5,906 1,000 15,000

Average transplant charges in 1973 were $12,800, witha range of $5,500 to $20,500. Average transplantcharges in 1979 were $23,000, and cadaveric kidneytransplants averaged $25,000 in 1978 (30).

The above cost per year figures show that substan-tial savings could be achieved by shifting more patientsto treatment by home dialysis or transplant (30).Roberts and his associates reported in 1980 that liv-ing, related donor transplants were the least costlytreatment modality and had the greatest survival time;center dialysis (hospital-based) was the least cost ef-fective (29). Shifts to either home dialysis or cadavericdonor transplantation would save from $7,000 to$8,000 per life year or $284 million per year for theexisting ESRD program (29).

GROWTH OF PROPRIETARY FACILITIES

More than 20 percent of the Nation’s hemodialysispatients are now dialyzed in for-profit units. Pro-prietary facilities present difficult ethical problems forthe medical community (20). Physicians who rendercare to a patient also share in the profitability of thatfunction. For-profit dialysis units tend to prefer main-taining patients in an outpatient setting rather than ahome setting. There are many cost disincentives tohome hemodialysis. Since home dialysis means lessprofit, profit may influence the choice of mode of care(20). On the other hand, proprietary facilities filled aneed when university hospitals and the Governmentcould not or would not expand facilities. Such facilitiesare extremely cost effective, and all function withinthe Medicare reimbursement screen. Freestanding unitscan deal with Medicare billing, private insurers, andother funding sources more efficiently than hospitalbilling departments (3). They also have financial in-centives to maintain a high patient census and to pro-vide short, highly efficient dialysis using ultrafiltrationtechnology in order to maximize the business efficiencyof the facility. Physician reimbursement for servicesalso gives for-profit units an incentive to encouragefacility dialysis, since home dialysis generates fees atthe office visit level only (3).

Appendix E—Case Studies of Medical Technologies . 181

Function and Structure of the ESRD Program

REIMBURSEMENT FOR PHYSICIANS’ SERVICES

Hemodialysis.—Under the ESRD program, there aretwo methods of reimbursement for physicians’ serv-ices: initial method and alternative payment method.By the initial method, physicians are paid directly bythe facility for their supervisory services during dial-ysis. For other nonroutine services required by the pa-tient, physicians bill on a fee-for-service basis. Theaverage payment rate to the physician for supervisoryservices during dialysis is part of the overall dialysischarge and averages $13 per treatment ($12 for free-standing units). By the alternative reimbursementmethod, physicians are paid a monthly fee for eachpatient for the full renal care of that patient. Such careincludes supervisory services during dialysis plus allother related services furnished during a particularmonth. Prior to July 1, 1978, alternative monthly al-lowances for physician services to patients dialyzingin facilities ranged from a minimum level of $160 toa maximum level of $240. Allowances for physicianservices for treatment of patients dialyzing at homeranged from a minimum of $112 to a maximum of$168. These amounts are subject to Medicare Part Bcoinsurance and remained constant from time of theirimplementation in 1974 to July 1, 1978.

The monthly allowances were increased on July 1,1978, to reflect changes in the customary and prevail-ing charges for internists and routine followup officevisits, but were not to exceed the increase in the med-ical care index, which rose 20.9 percent from July 1975to July 1978. The resulting revised monthly paymentsto the 957 physicians using the alternative method ofpayment now range from $180 to $260 before coin-surance for facility patients and from $126 to $182 forhome patients. The price index adjustment resulted inan arithmetic mean payment before coinsurance of$220 a month for facility patients and $154 a monthfor home patients.

Transplant.—All physicians’ fees are covered asfollows:

Ž 1978-79—Excise surgeon (donor kidney): $350 for1 kidney, $700 for two kidneys.

● 1979—Transplant surgeon: $1,600 to $2,500, de-pending on number of services provided.

● 1979—Transplant surgeon: $1,690 to $2,730, de-pending on number of services provided.

ESRD NETWORKS

ESRD networks in 32 geographic areas covering theUnited States were established by regulations publishedon June 30, 1976. Their role and function were restated

in section 1881 of Public Law 95-292. In 1977, Federalfunding policies were changed to encourage networkorganizational efforts. The year 1978 was devoted tothe establishment of viable organizations and the con-duct of multiple functions: inviting Medicare-approvedESRD facilities to join the network coordinating coun-cils, developing operating rules and procedures, andhiring professional and technical staff personnel, All32 networks had secured the services of an executivedirector by September 1978.

Medical review boards began in 1978 to fulfill theirregulatory functions (14): 1) monitoring the effect oflong-term programs by assessing appropriateness ofpatients for proposed treatment procedures, 2) review-ing the comparative performance of facilities andphysicians for areas of patient care, 3) conductingmedical care evaluation studies, and 4) performing

other studies as needed.Network coordinating councils review and recom-

mend approval/disapproval of applications for newor expanded dialysis facilities, a process that has ledto charges of conflict of interest. HCFA regional of-fices consider the network recommendations togetherwith recommendations from the health systems agen-cy and State health planning agencies in making theirfacility certification decision. In many networks, 75to 80 percent of the facility review committee mayhave a direct proprietary interest in the decision be-ing made.

The network coordinating councils were requestedby HCFA in November/December 1978 to developgoals relating to self-dialysis and transplantation. Inmid-January, they were asked to submit interim state-ments of 1979 goals to HCFA. As of March 15, 1977,12 of the 32 networks had submitted statements. Ac-cording to the 1980 annual report of the ESRD pro-gram, 24 of the networks had established goals for self-dialysis training, home dialysis, or for self-dialysis pro-grams and kidney transplantation (23). Eight networksrefused or evaded the HCFA operating guidelines: Net-work 3 (Northern California), 6 (Arizona and NewMexico), 15 (Illinois), 23 (Washington, D.C. ), 24(Delaware), 27 (Connecticut), 28 (Maine, Vermont,New Hampshire, Massachusetts, and Rhode Island),and 32 (New Jersey).

DIALYSIS PAYMENT RATES (22,23)

In 1979, the average payment rate was $149 pertreatment. This is a combined weighted average ofpayments to hospital-based and non-hospital-basedunits. Hospital facilities were paid the lesser of theircosts or a national payment limit (the screen), and theaverage payment was $159 per treatment. Independent


facilities were paid the lesser of their charges or thenational payment limit, and the average payment was$138 per treatment.

Any ESRD facility desiring a payment rate abovethe national limit had to request a reimbursement ex-ception and submit documentation of its higher costs.In 1978, HCFA approved 208 ESRD facility requestsfor reimbursement exceptions. Some of the primaryreasons for approval included:

● treatment of an unusually ill population;● treatment of an unusual patient population (e.g.,

children);• location in a high-cost or low-utilization area;• recent approval and continuous experience of low

utilization;● demonstrated low utilization due to sporadic

workload (referral hospital).The number and range of payments approved in ex-

cess of program payment screens in 1-9-78were as follows:

Home training,Range 1978 1979 1979

Up to $150 . . . . . . 28 “27 o$151-$170 . . . . . . . 63 80 8$171 -$190 . . . . . . . 69 62 15$191 -$210 . . . . . . . 30 35 20——$211 + . . 208 221 52

According to the 1980 ESRD annual report,

and 1979

Peritonealo014

5

however,HCFA actually approved 278 reimbursement excep-tion requests (in full or in part) in 1979, while deny-ing or returning another 23 requests (23).

GROWTH IN ESRD PROGRAM COSTS

The cost of the ESRD program grew from $250 mil-lion in 1974 to over $1 billion in 1979, greatly ex-ceeding original congressional estimates of potentialcosts (29). However, the number of patients receivingtreatment also exceeded estimates. According toKolata (18), when inflation is taken into account, thecosts of dialysis per patient have decreased since theprogram began. (But total published costs underesti-mate true costs to the Government because a substan-tial number of patients collect Federal disability pay-ments (16). )

In March 1977, the average weekly payment for theESRD program was $6 million; in July 1978, it was$10 million a week; and by August 1978, it hadreached $12 million a week (20). Between 1973 and1977, the annual cost per patient had increased at lessthan half the annual rate of inflation. From 1977 to1978, the per capita payment increased from $15,295to $16,300, a 6.5-percent increase.

However, growing concern has been expressed overthe large costs for a program benefiting a rather smallnumber of people (29). In 1979, benefit payments for

ESRD exceeded 5 percent of total Medicare expendi-tures, and were fully 10 percent of expenditures fromthe Supplemental Medical Insurance fund (Part B) ofMedicare, although renal patients comprise only 0.2percent of the Medicare population (4,29). In addition,fully one-third of ESRD beneficiaries were eligible forSocial Security monthly disability benefits. Morespecifically, 10 percent of Part B funds went to 50,000ESRD beneficiaries, while 90 percent of the funds wentto 23 million elderly enrollees; as of 1977, the ESRDpatients received $13,555 per capita, while the elderlyenrollees received $218.37 per capita (28). The Govern-ment set a fee of about $28,000 per year for each pa-tient in an outpatient facility in 1979. Medicare paid80 percent and the rest was covered by the States andprivate insurance carriers or was absorbed by thecenters. Table E-8 shows the growth in program costssince the beginning of the ESRD program.

Quality of Life on Dialysis

Now that physicians no longer have to make pain-ful choices regarding who gets selected for dialysis, thequestion has been raised whether too many people inthe United States are now being dialyzed. Blagg andScribner (4) believe that for an ever-increasing propor-tion of dialysis patients, the quality of life is unaccept-able and increasingly costly. Many patients now beingaccepted into dialysis programs have such severe com-plicating illnesses that they may be unable to live athome. Known examples include a blind diabetic withsevere angina and a nursing home patient who getstransported by ambulance twice weekly to the dialysiscenter (18). Quality-of-life issues also relate to the “sickroom” atmosphere of hospital-based dialysis units (9).Freeman (11) recommends removing dialysis unitsfrom hospitals to better designed, less expensive, morecheerful, freestanding units, with the hope that the“mass production line” approach to dialysis can beavoided.

There are now 55,000 patients on hemodialysis. Aninformal survey of 21 dialysis centers found that 44

Table E-8.—Growth in ESRD Costs, 1974-80

Year Per capita costs Program costs (millions)a

1974 . . . . . . . $250 (283)1975 . . . . . . . $11,000 330 (450)1976 . . . . . . . 12,300 450 (598)1977 . . . . . . . 16,800 600 (722)1978 . . . . . . . 17,300 737 (947) (1.1 billion)1979 . . . . . . . 23,500 850.5 (1.2 billion)1980 . . . . . . . 28,000 NAaNumber~ in parentheses reflect discrepancies in the literature.

NA = indicates not available.

SOURCE: R. Rettig, Ur@emenfirrg the .Errd-Stage Renal Disease Program ofMedicare, 1980.

percent of the dialysis patients were not working andthat more than 50 percent of this population wereprobably too sick to work. Twenty percent of the non-diabetic patients were unable to care for themselvescompletely, and 50 percent of the diabetic patientswere unable to care for themselves completely. Skewedincentives may have encouraged insufficient discrim-ination in the selection of candidates for dialysis (15).

In Britain, many physicians decide not to refer cer-tain types of patients for hemodialysis because of theirbelief that it is inappropriate treatment for the situa-tion. “In the absence of personal financial incentivesto treat more patients with dialysis, the NHS [NationalHealth Service] doctor is more free to decide that theseextraordinary procedures for prolonging life do notconfer a good enough quality of life to make them suit-able for all patients dying of renal failure. Not to treatmay be kinder and wiser. ” (1).

Generalizations About the ESRD Program

According to Rettig (28), most of the problems ofthe ESRD program have arisen from its administrativesystem, the planning and operational stages of its im-plementation, and the substance of reimbursement andmedical issues. The most important reimbursementpolicy has been the screen, or de facto ceiling, on theper treatment reimbursement of outpatient mainte-nance dialysis. This screen has provided a strong in-centive to cost containment. On the other hand, thefinancial disincentives to home dialysis have been oneof the last defensible aspects of reimbursement (28).Rennie (26) has stated that all the problems with theESRD program can be attributed to inappropriate eco-nomic incentives and inappropriate economic deter-rents in the present law and regulation.

Blagg and Scribner (3) have identified several prob-lems with respect to the ESRD program’s implemen-tation: lack of effective leadership, no continuity inadministrative policy, slow and haphazard implemen-tation; increased vulnerability to political lobbying,piecemeal regulations, and lack of meaningful data.The absence of data has made it impossible to assessand compare the quality of care and patient outcomes.It has been impossible to pull out of HCFA’s computersdata such as percentages of patients dialyzed at homeand at centers and such patients’ relative mortalityrates, ages, and illness levels.

Three important innovations have been introducedby the ESRD program (28). One is the screen on facili-ty reimbursement, which creates strong incentives fordelivering outpatient dialysis. The second is the proc-ess of reimbursing physicians indirectly by a monthlycavitation method, which departs from the traditionalfee-for-service reimbursement. The third innovation


is the facility certification process, which permits thenumber and capacity of treatment facilities to increasein reasonable relation to the growth in the patientpopulation.

Jones’ References1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17,

18,

Anonymous, “Selection of Patients for Dialysis andTransplantation, ” Br. Med. J. 6150:1449, 1978.Bergan, J. H., “History of Renal Transplant Registry, ”in NIAID, U.S. Kidney Transpkmt Fact Book (Wash-ington, D. C.: Department of Health, Education, andWelfare, 1972).Blagg, C. R., and Scribner, B., “Long-Term Dialysis:Current Problems and Future Prospects,” Amer. J Med.68:633, 1980.Blagg, C. R., and Scribner, B., “Medicare End-stageRenal Disease Program: More Than A Billion DollarQuestion, “ Ann. Inter. Med. 93:501, 1980.Cestero, R., “A Regional End-Stage Renal Disease Pro-gram: Twelve Years’ Experience, ” Ann. Inter. &led.93:494, 1980.Comptroller General of the United States, Request ToUpdate Information in the 1975 GAO Report (Wash-ington, D. C.: General Accounting Office, 1977).Comptroller General of the United States, Treatment ofChronic Kidney Failure: Dialysis, Transplant Costs, andthe N*dfor More V&orous Efforts (Washington, D. C.:General Accounting Office, 1975).Czaczkes, J. W., and De-Nour, A. K., Chronic Hemo-dialysis as a Way of Life (New York: Brunner/Mazel,1978).Drukker, W., et al. (eds. ), Replacement of Renal Func-tion by Dialysis (The Hague: Mortinus Nijhaff MedicalDivision, 1978).Freedman, E. A., et al. “Pragmatic Realities in UremiaTherapy,” N. Eng, J. Med. 298:368, 1978.Freeman, R., ‘The Social Impact of Chronic MaintenanceHemodialysis, ” in Replacement of Renal Function byDialysis, W. Drukker, et al. (eds. ) (The Hague: MortinusNijhaff Medical Division, 1978).Gottschalk, C, W., Report of the Committee on ChronicKidney Disease (Washington, D. C.: Bureau of the Budg-et, 1967).Greenberg, D., “Renal Politics, ” N. Eng. J. Med.298:1427, 1978.Gutch, C. K., and Stoner, M., Review of Hemodialysisfor Nurses and Technicians (New York: C. V. Mosby,1979).Gutman, R., et al. “Physical Activity and EmploymentStatus of Patients on Maintenance Dialysis, ” N. Eng. J.Med. 304:309, 1981.Hampers, C., and Hager, E., “The Delivery of DialysisServices on a Nationwide Basis,r’ Dialysis& Transplant.8:417, 1979.Health Services Administration, Ehd-Stage Renal Dis-ease Medical kformation System, FaciJity Report Num-ber I (Washington, D. C.: Department of Health, Educa-tion, and Welfare, 1976).Kolata, G. B., “Dialysis After Nearly a Decade, ” Sci-ence 208:473, 1980.


19

20

21

Kolata, G. B., “NMC Thrives Selling Dialysis, ” Science208:379, 1980.Moore, F. D., “End-Stage Renal Disease and the Gov-ernment,” Dialysis & Transplant. 7:258, 1978.Moore, F. D., Transplant: The Give and Take of TissueTransplantation (New York: Simon & Schuster, 1972).

22. Office of End-Stage Renal Disease, Health Care Financ-ing Administration, End-Stage Renal Disease ProgramAnnuaJ Report to Congress, 19J’9 (Washington, D. C.:Department of Health, Education, and Welfare, 1979).

23. Office of End-Stage Renal Disease, Health Care Financ-ing Administration, End-Stage Renal Disease ProgramSecond Annual Report to Congress, 1980 (Washington,D. C.: Department of Health and Human Services, 1980).

24. Relman, A., “The New Medical-Industrial Complex,” N.Eng. J, Med. 303:963, 1980.

25. Rennie, D., “Home Dialysis and the Costs of Uremia, ”N. l%g. J. Med. 298:393,1978.

26. Rennie, D., “Renal Rehabilitation—Where Are theData?” N. Eng. J. Med. 304:351, 1981.

27. Rettig, R., “End-Stage Renal Disease and the “Cost” ofMedical Technology:” in Medical Twhnology: The CW-

28.

29.

30.

31.

prit Behind Health Care Costs? S. Altman ;nd R. Blen-don (eds. ) (Washington, D. C.: Department of Health,Education, and Welfare, 1977).Rettig, R., Implementing the Ehd-Stage Renal DiseaseProgram of Medicare (Santa Monica, Calif.: RandCorp., 1980).Roberts, S., et al., “Cost-Effective Care of End-StageRenal Disease,” N. Ehg. J. Med. 92:243, 1980.Stange, P., and Summer, A., “Predicting Costs and LifeExpectancy for End-Stage Renal Disease, ” N. fig. J.Med. 298:372, 1978.Steinman, T. I., letter to the editor, N. Eng. J. Med.298:1086, 1978.

Appendix F.— Model for an Institute forHealth Care Evaluation*

by John P. Bunker, M. D., and Jinnet Fowles, Ph.D.Division of Health Services Research,

Family, Community and Preventive Medicine,Standard University School of Medicine

Rationale

Current Federal policy is to reduce the Government’sresponsibility for health care, substituting whereverpossible market mechanisms, and to vest residual con-trol in regional and local authorities. Towards this end,the Reagan administration has recommended to Con-gress sharp reductions in expenditures for medicaltechnology assessment. This approach is reflected inthe failure to fund the National Center for Health CareTechnology (NCHCT) and major cutbacks for the Na-tional Center of Health Services Research (NCHSR),the National Center for Health Statistics, and the Of-fice of Research and Demonstrations of the HealthCare Financing Administration (HCFA).

The budget of the National Institutes of Health(NIH) has been relatively spared, but, with evenmodest decreases in NIH funding, any cutbacks canbe expected to occur primarily in the areas of evalua-tion and clinical trials (4). This reduction would occurat a time when the pressures for more comprehensiveevaluation are increasing, both from academic institu-tions and from private and governmental insurers. Onepartial solution to this conflict might be to develop aprivate Institute for Health Care Evaluation (IHCE),which would operate as a nonprofit corporation (per-haps replacing NCHCT) and extend the Nation’s ca-pacity to evaluate medical technologies.

IHCE could be composed of members from severalgroups concerned with the evaluation of health care:governmental insurers (HCFA); private medical in-surers (Blue Cross, Blue Shield, and commercial car-riers), health maintenance organizations (HMOs); pro-fessional associations (represented, perhaps, by theCouncil of Medical Specialty Societies and its programfor clinical procedure review); and health consumers.Each of the parties could benefit from the data thatIHCE generated. Health care professionals could usethe data to improve the quality of patient care; healthconsumers could have increased information on whichto base their selection of coverage; and insurers couldhave access to data allowing them to make more ra-tional and timely coverage and reimbursement deci-sions.

● Reference citations for this appendix on p. 190

Technology assessment is a classic illustration offree-market failure—i.e., market forces will not com-pel the generation of the optimal amount of perform-ance data essential for the effective and efficient run-ning of the system. Thus, the responsibility for assess-ing medical technologies cannot be left to the unaided,competitive forces of the marketplace. Because theywould receive direct financial benefits, both Federaland private insurers would be expected to providefinancial support for IHCE’s work. However, becausethey have vested interests in certain outcomes (e.g.,results justifying decreased utilization or advancing

lower cost alternatives), these insurers should not haveexclusive control of IHCE’s operation. In order tomaintain its credibility, the organization would haveto be governed through a system of checks and bal-ances involving constituent groups.

Goals and Objectives

IHCE’s goal would be to generate cost-effectivenessdata with a strong emphasis on the measurement ofoutcomes of therapeutic intervention. These data areneeded by medical professionals as a basis for mak-ing decisions and informing patients about theirchoices in medical care; they are needed by health careconsumers who are increasingly expected to assumeresponsibility for their own health and to participatein therapeutic decisions; and they are required by theinsurance industry in order to design rational healthinsurance plans. Adequate technology assessment rep-resents the core of the two major issues facing healthcare today: 1) how best to employ complex technol-ogies, old as well as new, to meet the public’s medicalneeds; and 2) how to limit the costs of medical carewithout jeopardizing its quality.

The proposed IHCE would have four major objec-tives:

● development of a uniform data base;● systematic identification of agenda issues;● generation of new data and analyses; and• dissemination of information to carriers, profes-

sionals, and the public.The achievement of the first objective, development

of a uniform data base, is necessary to facilitate thecollection of information from diverse sources. At

185

98-144 9 - 82 - : 3

—.-.


present, a modest amount of relevant but oftennonuniform data is generated by many health pro-viders. For example, the Kaiser Foundation HealthPlan, as part of patient registration, collects detailedinformation on service utilization at its varioushospitals and clinics, then aggregates the informationon a regional basis for planning of resource allocations.The proposed IHCE could develop guidelines for aninstrument, e.g., a patient registration form, whichcould serve as both a receipt for billing and methodfor monitoring utilization and identifying new pro-cedures. Decentralization of data storage at a regional,local, or even health plan level would help address theneed for confidentiality of the sources of information.Each plan, locality, etc., might store its own standard-ized data and make these data available to IHCE-authorized researchers.

The second objective of the proposed IHCE, to serveas a communication clearinghouse that would syste-matically collect priority issues from its members, isan objective that has not been satisfactorily met in thecurrent system, which relies primarily on signals re-ceived from claims data. IHCE would have the author-ity and capability for routinely surveying professionalsto elicit their opinions about the future directions ofinnovations in their specialities. This model is currentlybeing used by Kaiser-Oakland’s Technology Assess-ment Division. Innovations and medical problem areascould also be identified in a number of other ways,including systematic literature reviews and monitor-ing of professional meetings. The patient registrationforms could act as one “flagging” device. This taskmight be under the purview of existing and surviv-ing Professional Standards Review Organizations(PSROs). Finally, health plans, private enterprise, andothers might directly identify a new procedure and re-quest evaluation by IHCE.

IHCE’s third objective would be to participate in thegeneration of new data, specifically, through the sup-port of clinical trials, retrospective studies, data banks,and possibly surveillance. Currently, a small numberof clinical trials are funded primarily by NIH, but itcan be anticipated that the NIH investment in this func-tion will diminish in the near future (4), and currentlyproposed legislation specifically enjoins NCHSR fromfunding “clinical” studies. Clinical trials, althoughcomplex, time consuming, and resource intensive, re-main the best method for determining the relativevalue of alternative medical technologies. Rather thanbeing retreated from, clinical trials should be usedmore extensively within the limitations of theirmethodology and the clinical circumstances.

The fourth objective of IHCE could be a secondfunction of its communication clearinghouse: dissem-ination of results of analysis back to the participants.

Such dissemination could be achieved though the pro-vision of access to computerized information, whichcould include annual reports, strategic objectives, find-ings of prior assessments, and listings of studies pres-ently in progress. Participants would then be able toincorporate these results as they saw fit into theirrespective decisions. The proposed IHCE would nothave responsibilities in policymaking. Its respon-sibilities would rest solely in the areas of data collec-tion and analysis.

Funding

The perception of a need for an IHCE is based onthe recognition that health care assessment is a publicgood. The marginal cost of assessment for any in-dividual or group generally far exceeds the marginalbenefit derived for any individual or group. The gen-eration of information as a public good creates a “free-rider” problem. After an individual or group pays todetermine that a particular procedure, protocol, de-vice, etc., is more cost effective, the very act of cap-italizing on that information makes it public knowl-edge; the information is freely available to other in-dividuals and groups who did not share the cost ofmaking the determination. The ethics of medical careencourage the early and broad dissemination of infor-mation.

The proposed IHCE would be a nonprofit organiza-tion funded by a per capita assessment or levy to bereceived from all qualified health plans. The fundingfor IHCE could be established on either a mandatoryor voluntary basis.

With a mandatory system, health plans (for-profitand nonprofit) would be required to support IHCE asa condition of their receiving recognition as a “qual-ified” health plan—and therefore becoming eligible toreceive tax credits, vouchers, or Medicare payments.To prevent the problem of free-riders (i.e., competinginsurance programs which gain access to informationwithout paying for the costs of its generation), a feewould be required to support IHCE. Although thereare increasing numbers of industries that self-insure fortheir employees’ medical care, most still carry ad-ministrative contracts with private insurers (or claimsadministrators). Under a mandatory structure, the feefor health care evaluation would be a required com-ponent of this administrative contract.

Other service organizations have set up similarmodels of cooperative research. One model is the Elec-tric Power Research Institute, which assumes respon-sibility for part of the research agenda of the electricpower providers. Participants in the Electric Power Re-search Institute contribute to the support of the in-stitute without resorting to taxation such as that pro-

Appendix F—Model for an Institute for Health Care Evaluation . 187

posed in the mandatory version of IHCE suggestedabove, However, their situation differs from that ofhealth care in two significant respects. First, the elec-tric utilities do not compete with one another; theyare a regulated monopoly. Second, these utilities allhave a cost-based price regulation, allowing them topass the cost of membership directly on to the con-sumers.

A second model for cooperative research is theHealth Effects Institute, a nonprofit organizationestablished in 1980 to study the health effects ofautomotive emissions. This institute is funded jointlyby grants from governmental and charitable servicesand by additional funds contributed by participatingautomobile manufacturers according to a formula de-veloped by industry. The Health Effects Institute is anindependent organization that has no actual gover-nance link to the Environmental Protection Agency,automotive industry, or public participants. Its healthresearch committee establishes research priorities, de-velops research programs and protocols, obtains ex-haust samples from manufacturers, and contracts withresearch centers to perform specified tasks.

Yet another example of voluntary cooperative re-search comes from the insurance industry itself. TheInsurance Institute for Highway Safety, founded in1959, is a nonprofit corporation established to studythe contributory factors of drivers, vehicle design, androadways to highway safety. The Insurance Institute’sannual budget is based on contributions from threeautomotive insurance trade associations and oneassociated insurance group (whose members do notbelong to any of the trade associations). These groups,in turn, raise their contributions from individual com-panies on the basis of their total premiums. The re-search protocols are developed by the Insurance In-stitute and contracted out to academic research centers.These voluntary models might not be feasible in anincreasingly competitive health care environment,where some carriers could gain access to the informa-tion without paying for it and, thus, offer lower ratesto subscribers than could carriers who were contrib-uting members of IHCE.

Nevertheless, it might be possible to fund IHCEthrough voluntary contributions. Although this wouldcreate the free-rider problem described above, it mightalso alleviate some of the initial resistance to theestablishment of such an institute. With voluntaryfunding, there would still be a membership fee requiredof all participants, which would cover IHCE’s basicadministrative costs. IHCE’s governing body woulddevelop an agenda of research topic alternatives onwhich the members of IHCE would vote. Alternativeswould be given priorities by the membership, and

members would subscribe in advance to cover the costsof conducting specific research studies.

What it would cost to develop the proposed IHCEis of obvious concern to those who would be expectedto bear the burden of expenses. Relman has suggestedthat two-tenths of 1 percent (0.002) of expendituresfor medical care might bean appropriate sum to allo-cate for this purpose (9). Since the current expendituresfor medical care of private insurance and HCFA areapproximately $160 billion and $85 billion, respective-ly, this would amount to nearly $500 million. Whetherthis is more or less than the task will require is by nomeans clear. If IHCE succeeds in its mission, this willbe a small price to have paid. Indeed, as some sug-gest, the potential savings that could be expected toaccrue as a result of better data on cost effectivenesswould be many times greater than this amount (3,9).

IHCE’s success, however, cannot be guaranteed.Therefore, rather than making an all-to-nothing com-mitment to a program of this magnitude, it would beprudent to proceed in modest steps. To test the pro-posed IHCE’s potential, its board or council mightbegin by identifying those areas of medical caredeemed to be in greatest need of evaluation, raisingfunds by assessment as needed for each subject of in-quiry or analysis. Indeed, this would appear to be anappropriate method for the future funding of all proj-ects: funds being generated only as the potential usersof information judge necessary and appropriate.

Mechanism and Structure

IHCE could select topics for evaluation from thosegenerated by its technology surveys. The topics wouldbe given priority by the appropriate committee (boardof directors or council). IHCE would let out contractsfor clinical trials, retrospective studies, and technologyassessments to carriers, as well as contracts for dataanalysis to professional organizations. In addition,IHCE would review and fund independently submittedproposals for clinical trials and for technology assess-ment. For example, investigators planning new pro-cedures might apply directly to IHCE for funding, in-cluding the clinical costs of particular innovations.IHCE would coordinate its activities with those ofother research organizations such as the disease-specific private foundations and Federal research agen-cies such as NCHSR, NCHCT (if funded), NIH, andthe Food and Drug Administration.

It is anticipated that IHCE would be able to use somealready established mechanisms for data collection, in-cluding claims data, health systems agencies (HSAs),and PSROs. HSAs and PSROs may succumb to cur-rent budget cuts, but the evaluation capabilities

In Strategies for Medical Technology Assessment

developed by the more successful PSROs could prof-itably be put to work on contract by IHCE (8).

Although the agenda would be set as a consensusat the national level, most of its implementation wouldtake place under local control and responsibility. In-dividual institutions, through their institutional reviewboards, would determine whether proposed new or ex-perimental procedures are used with appropriatestandards for patient safety and whether standards forinformed consent have been met.

The proposed IHCE would be governed by a boardof directors or council composed of representativesfrom member groups: private insurance carriers, gov-ernmental insurers, HMOs, professional associations,and consumers. In addition to topical subgroups, therewould be specific departments for legal affairs, com-munication, and publications.

Concerns With the Model

Legal Problems

Funding Sources. —If funding for IHCE were man-dated by a tax, levy, or assessment to be paid by allinsurers according to the number of individuals theycover, it would require new legislation. A Federal taxwould presumably violate the current position of theinsurance industry, which has been exempt from Fed-eral legislation under the McCarran-Ferguson Act.Thus, taxation on private carriers would represent asubstantial departure from this position. A tax on non-profit organizations, including Blue Cross and BlueShield and HMOs, would present similar problems.However, such a recommendation has a precedent ina recent proposal by David Stockman, Director of theOffice of Management and Budget. This proposal, theGephart-Stockman National Health Care Reform Act,would levy a tax on health insurers to provide fundsfor insuring subscribers against the financial failure oftheir selected health plans, The alternative of volun-tary funding does not present these legal problems.

Research Authority.—Current insurance trends aregenerally to avoid involvement in research exceptunder certain explicit circumstances. Therefore, it isuncertain on what legal grounds insurers would par-ticipate. Some legal advice indicates that insurers dohave a research authority, but this concept is not ex-plicit nor does it appear to have been tested in thecourts.

Antitrust. -Various insurers have suggested that theoperation of IHCE might be in violation of the Sher-man Antitrust Act. However, it should be noted thatmany commercial carriers, through the Health In-surance Association of America, already jointly usethe coverage recommendations of the Council of Med-

ical Specialty Societies based on its program for clinicalprocedure review. The program for clinical procedurereview identifies clinical procedures that are obsolete,duplicative, or not yet clinically proven. The Councilof Medical Specialty Societies then recommends con-tinuation or withdrawal of reimbursement. The pro-posed IHCE would differ from the Council of MedicalSpecialty Societies in that its function would be limitedto the development, analysis, and distribution of dataand would not include policy recommendations.

Selective Coverage. —The need for selectivecoverage has been widely recognized by third-partypayers. To refuse payment to a hospital on the basisof inadequate experience, equipment, or trained staff,or failure to adhere to published standards, can be ex-pected to lead to litigation—and has already done so.Part of the difficulty results from the wording of cur-rent insurance policies and contracts. Future policiesshould be drawn up explicitly indicating that coveragefor specified procedures will be limited to certain pro-viders. (Blue Shield of California is currently explor-ing the feasibility of contracting with better qualifiedhospitals and physicians for heart surgery at set pricessubstantially lower than standard or average fees; thisis similar to the arrangement of preferred providerssuggested for HMOs by Interstudy (7). ) The possibleneed for legislation to protect third-party payers undersuch arrangements is deserving of exploration.

Selective coverage for new and experimental medicaland surgical procedures might be provided by com-mercial carriers or by HCFA in conjunction with theproposed IHCE’s program, with hospitals and physi-cians selected for participation in clinical trials on acase-by-case basis. Reimbursement under these con-ditions would represent what amounts to a researchaward and would presumably present less of a threatof litigation by nonparticipating and unapproved hos-pitals and physicians or their patients.

A serious problem that would remain, however,would be the opportunity for those physicians and/orhospitals not selected for reimbursement for a new (orold) procedure to do it anyway and to submit chargesusing billing codes for other, standard, procedures.This is one of the difficulties encountered in the cur-rent system of reimbursement.

Comprehensive restructuring of the method of reim-bursement (e.g., cavitation, prepayment, or paymentby voucher) would partially resolve this problem, sincea fixed amount of money would be available for allprocedures. Additional funds for new procedureswould have to be negotiated on a procedure-by-pro-cedure basis. Short of such radical changes in methodof reimbursement, natural forces within the presentsystem may be expected to exert some, perhaps con-siderable, corrective influence. Malpractice suits

Appendix F—Model for an Institute for Health Care Evaluation ● 189

against physicians who deviate from established stand-ards are occurring with greater frequency. With in-creasing professional consensus that it is poor prac-tice to perform specified complex procedures on an oc-casional basis or in facilities not equipped or staffedfor such procedures, it can be anticipated that lawsuitswill be brought against physicians and institutions thatdo this. Physicians will be forced to be more circum-spect, and hospitals will look more and more to theirinstitutional review boards for guidance in undertak-ing new and experimental procedures.

Ethical Issues

A fundamental principle of justice that has oftenbeen urged is that innovations of established efficacybe available to all of the population. This principle wasmost dramatically implemented by the MedicareAmendments of 1972, by which entitlement to medicalcare for end-stage renal disease (hemodialysis, kidneytransplant) was conferred on all citizens of the UnitedStates. This principle has been repeatedly invoked inpolicy analyses of the artificial heart program (1), andit underlies current deliberations concerning reimburse-ment for heart transplants.

Heart transplantation in the United States has beenconcentrated primarily at Stanford University, whereit is now considered an established clinical procedurewith survival results comparable to those achievedwith cadaver kidney transplants. Funded originallythrough an NIH research grant, the clinical costs ofheart transplantation at Stanford were reimbursed byHCFA from 1979 until early 1981, and many privateinsurance companies now reimburse for heart trans-plantation. Relatively few heart transplants have beenperformed at other institutions, and their combinedresults have not matched Stanford’s. Reimbursementby HCFA has not been made available to these otherinstitutions because of their less satisfactory results.When challenged on this apparent inequity by anotherinstitution, HCFA responded by withdrawing reim-bursement for heart transplantation at Stanford andannouncing plans for a 2-year study of ethical, legal,and economic aspects of heart transplants. This iswhere the issue now stands.

When a medical or surgical procedure is clearly ex-perimental, there can be no ethical obligation to makesuch a procedure available to all. An experimental pro-cedure is, by definition, of unknown benefit. It maybe better, or worse, or equal to previously available,established therapies or to no treatment at all. It isbecause of these procedures’ unknown efficacy thatmechanisms such as informed consent and institutionalreview committees have been established to advise pro-

spective patients of the risks of such procedures andto reduce the possibilities of harm.

Approval for the performance of experimental pro-cedures must rest on the qualifications of the in-vestigators and on research resources and priorities,not solely on the medical needs of the patients or theavailability of reimbursement from the insurers. Localinstitutional review committees are the appropriateagents to determine whether a proposed research pro-cedure is scientifically and ethically justified andwhether the interests of the patient, as experimentalsubject, are adequately protected. There are clearguidelines for these committees to follow in makingthese determinations.

The just distribution of efficacious medical care, inaccordance with the foregoing general principles, re-quires better data than the data currently available.It is the exception, rather than the rule, that newtherapies are introduced with well-controlled clinicaltrials leading to definitive evidence of therapeuticworth. In the absence of such data, effective treatmentsmay be withheld or ineffective treatment may be given,Both errors seem likely to occur, in view of the manyvariations in procedure and hospitalization rates re-ported by Wennberg and others (12). Both errors, tothe extent that they are avoidable, may be consideredserious injustices; it is not clear that one is more seriousthan the other. Neither is it clear which error is themore common. However, there is strong presumptiveevidence that when efficacy data are absent, physician-investigators tend to err in the direction of overesti-mating the potential benefits of therapy (5,6) and thatmany “unnecessary” procedures are carried out as aresult of professional enthusiasm or optimism. Theargument that there is widespread overprescribing oftherapy has been developed in detail elsewhere (2).Overutilization of unproven medical interventions hasimmediate and urgent implications for distributivejustice. Soon, society will no longer be able or willingto pay for all treatments that might be effective. Pur-chase of care that is ineffective or of undocumentedefficacy for some patients will almost certainly resultin the failure to provide effective care to other patients.

Quality of Information

A final and important concern relates to the quali-ty of information to be collected by IHCE. Toweryand Perry, at NCHCT, have proposed that “third-party payers, including Medicare . . . make reim-bursement to providers contingent on their submitting

certain minimal data under a previously agreed on pro-tocol” (11). Sherman, Fineberg, and Frazier, at theHarvard School of Public Health, have made a similar

——

190 Strategies for Medical Technology Assessment— — . —

proposal and have identified a number of contingen-cies for reimbursement (10), These proposals addressthe problem from the perspective of an agency whoseprincipal responsibility is to provide reimbursement,and for whom the collection of data is by definitiona secondary priority. The mandatory submission ofdata by medical care providers can also be assumedto be a secondary priority, and the quality of theresulting information may be poor.

The primary responsibility of the proposed IHCE,in contrast, would be to collect reliable information.Grants and contracts would be awarded on the basisof the anticipated quality of that information. Reim-bursement for clinical services might be provided inconjunction with the grant or contract, but it wouldbe a secondary consideration.

Discussion and Conclusions

The absence of a consistent and explicit policy ofreimbursement for new technologies results in theescalation of charges and at the same time providesincentives to conceal innovations. In addition, thereis no single organized and adequately funded programor agency charged with the responsibility for the gen-eration of data with which to evaluate new (and old)technologies. As a result, not only are good outcomedata lacking; often it is not even possible to identifywhen new procedures are performed. Indeed, the cur-rent system provides incentives which actively dis-courage the explicit identification of new and ex-perimental technologies. An additional difficulty is theinability to provide reimbursement for these pro-cedures on a selective basis. This inability makes it im-possible to achieve the orderly development and eval-uation of new technologies.

Potential remedies for the foregoing difficulties arereadily at hand. At the regional and local levels, Med-icare contractors, such as Blue Shield of Californiathrough its Medical Policy Committee, are already re-viewing claims for new therapies. Unproven therapiesare currently rejected for coverage, but could, havingbeen identified, be selected for coverage contingent oncollection of appropriate evaluation data and/or pres-entation of an appropriate experimental design.

To justify such a major shift in coverage policywould require major new funding for the evaluativeprocess. There are, at present, instances where fund-ing for clinical procedures and their evaluation is pro-vided on an individual basis, at least in part, by thethird-party payer. For example, Blue Shield of Mas-sachusetts has paid the clinical costs of three diagnosticexaminations for tumors of the adrenal, kidneys, andpancreas (ultrasound, computed tomography scan,

and radionucleotides), the costs of data collection andanalysis being funded by a foundation source. BlueShield of California is funding an analysis of the costeffectiveness of ambulatory surgery. But even suchmodest efforts severely extend the fiscal capacity ofindividual insurers—and, at a time of intense competi-tion for subscribers, tend to worsen rather than im-prove their competitive position. (The individual in-surer must charge its subscribers the extra cost, butall insurers can use the information resulting from theinvestigation. ) A resolution of this dilemma might befor the insurers to join in common purpose and tocreate a joint fund, from which amounts could beawarded by contract for proposals to evaluate specificprocedures.

The basic provisions of this option can be summar-ized as follows: First, an IHCE would be created underthe control of: 1) third-party payers, including BlueShield, Blue Cross, and the commercial health insur-ance companies; 2) HMOs, represented by the GroupHealth Association of America and the AmericanAssociation of Foundations of Medical Care; 3) theGovernment, represented by HCFA and (if funded)NCHCT; 4) the medical profession, represented by theCouncil of Medical Specialty Societies and/or theAmerican Medical Association Council on ScientificAffairs; and 5) representatives of the public at large(consumers). Second, IHCE’s goals would be: 1) theestablishment of a uniform data base; 2) the systematicidentification of agenda issues; 3) the generation of newdata and analysis; and 4) the dissemination of infor-mation to carriers, professionals, and consumers.Third, IHCE would be funded through fees or con-tributions from public and private insurers (includingself-insurers) and/or from HMOs on either a man-datory or voluntary basis. Finally, new and experimen-tal medical and surgical procedures would be selective-ly covered on the basis of locally approved researchprotocols and the availability of data for independentanalysis.

Appendix F References

1.

2.

3.

4.

5.

Artificial Heart Assessment Panel of the National Heartand Lung Institute, The Totally Zmplantalde ArtificialHeart: Legal, Social, Ethical, Medical, Economic, Psy-chological Implications, DHEW publication No. (NIH)74-191, June 1973.Bunker, J. P., “Health Care Costs, ” in Encyclopedia Bri-tanica, Medical and Health Annual, in press, 1982.Bunker, J. P., “Hard Times for the National Centers,”N. Ehg. J Med. 303:580, 1980.Fredenckson, D. S., “Biomedical Research in the 1980’s, ”N. Eng. J. Med. 304:509, 1981.Gilbert, J. P., et al., “Statistics and Ethics in Surgery andAnesthesia,” Science 198:684, 1977.

Appendix F—Model for an Institute for Health Care Evaluation ● 191— . — . . .

6. Grace, N. D., et al., “The Present Status of Shunts for 9.Portal Hypertension in Cirrhosis, ” Gastroenterology50:684, 1966. 10.

7. Interstudy, Minneapolis, Minn., “Preferred Provider Ar-rangements-Calif ornia, ” internal memorandum to P.Ellwood from K. Lewis, Nov. 18, 1980.

8. Lohr, K. N., et al., Professional Standards Review Or- 11.ganizations and Technology Assessment in Medicine,prepared by the Rand Corp. for the Office of TechnologyAssessment, U.S. Congress, Washington, D. C., March 12.1981.

Relman, A. S., “Assessment of Medical Practices, ” N.Ehg. J. Med. 303:153, 1980.Sherman, H., et al., “Issues Raised by Third Party Reim-bursement for Investigational Procedures: Options andCriteria, ” report to the National Center for Health CareTechnology, Feb. 2, 1980.Towery, 0, B., and Perry, S., “The Scientific Basis forCoverage Decisions by Third-Pary Payers, ” J. A.M.A.245:59, 1981.Wennberg, J. E., and Gittelsohn, A., “Small Area Varia-tions in Health Care Delivery, ” Science 182:1102, 1973.

Appendix G.— Method of the Study andDescription of Other Volumes

Method of the Study

The study Strategies for Medical Technology Assess-ment began on July 1, 1980. Immediately thereafter,a planning period was begun, and an advisory panelwas selected.

Most of the studies undertaken at OTA rely on theadvice and assistance of an advisory panel of experts.The advisory panel for a particular assessment sug-gests source materials, subject areas, case studies, andperspectives to consider; assists in interpreting infor-mation and points of view that are assembled by OTAstaff; and suggests possible findings and conclusionsbased on the accumulation of information producedby the study. The panel members review staff and con-tract materials for accuracy and validity, discuss policyoptions of the study, and present arguments for andagainst the options and conclusions. However, theydo not determine the report’s final form and are notresponsible for its content, direction, or conclusions.

The advisory panel for this assessment consisted of19 men and women with backgrounds in medicine,public health, sociology, information and libraryscience, economics, law, psychiatry, consumer ad-vocacy, technology assessment, industry, healthpolicy, ethics, and health insurance. The panel waschaired by Lester Breslow of the University of Califor-nia at Los Angeles. One member of the OTA HealthProgram Advisory Committee, Kerr White, alsoserved on the panel.

The first panel meeting was held on September 12,1980, in Washington, D.C. (the site of all three panelmeetings). Prior to the meeting, panel members weresent a detailed study plan, including a suggested out-line, and several pertinent articles as background fordiscussion. During the meeting, panel members dis-cussed the overall study plan for the assessment andhelped OTA staff refine the goals for the project. Thepanel examined the project boundaries and definitionalissues and was key in sharpening the study’s focus. Thepanel was also helpful in reviewing the primary issueareas to be covered and in providing suggestions ofindividuals and organizations to contact for informa-tion and assistance. The panel was particularly helpfulin suggesting modifications in several of the contrac-tors’ reports (which were just beginning). Several con-tractors were present and participated in the meeting.

By the fall ofthe main reportscribed below:

192

1980, all-of themajor contracts forwere let. Each contract effort is de-

John Williamson (Johns Hopkins University) de-veloped a helpful and imaginative theoreticalframework for a strategy for medical technologyassessment.Kathleen Lohr and Robert Brook (Rand Corp.)were asked to explore the potential role of theProfessional Standards Review Organizations(PSROs) in a medical technology assessment sys-tem. With the assistance of John Winkler (Rand),they examined how physicians received new med-ical knowledge, the potential for PSROs to trans-fer that knowledge, and the ability of PSROs totest technologies for safety and effectiveness.Their work is available from Rand as a publisheddocument.Paul Wortman (University of Michigan) andLeonard Saxe (Boston University) analyzed themethods of testing medical technologies, syn-thesizing information and soliciting group opin-ions, Their paper, reproduced as appendix C,formed the basis for much of chapters 3 and 5 ofthis report.John Reiss (Baker& Hostetler) was asked to writea paper on the role of reimbursement policy inan assessment strategy. His ideas are included inmany sections of this report.John Bunker collaborated with Jinnet Fowles(both of Stanford) and a number associates towrite a paper on the effects of reimbursement onthe innovation process for medical and surgicalprocedures. They developed the concept for the“Institute for Health Care Evaluation, ” repro-duced as appendix F, and submitted several casestudies, two of which are included in appendixE. Their main work is published as a two-partseries by the New England Journal of Medicine(Mar. 4 & 11, 1982).John Wennberg (Dartmouth) submitted an inter-esting paper on the use of health insurance claimsand patient outcome data to evaluate health caretechnologies after they are generally available.Patricia Woolf (Princeton) prepared a paperdescribing private sector activities in bothbibliographic data base production and vending.Appendix B contains a listing of those biblio-graphic-related resources.

Contractors’ reports were reviewed both by OTAstaff and by a large number of outside experts. Review-er’s comments were forwarded to the authors, who in-corporated them in revising drafts.

Appendix G—Method of the Study and Description of Other Volumes 193

In January 1981, a workshop was held in Boston toreview the first draft of the Wortman/Saxe paper onthe role of various methods for testing medical tech-nologies for safety, efficacy, and effectiveness. Par-ticipants included the authors and other scientists fromseveral research disciplines, an advisory panel mem-ber, and OTA staff, Wortman and Saxe prepared asecond draft of their paper on methods based largelyon that workshop.

The second advisory panel meeting was held onJanuary 28, 1981. The discussion of that meeting cen-tered around three main topics: 1) the selection ofmethods for testing medical technologies, 2) the useof the PSROs in technology assessment, and 3) theMedical Literature Analysis and Retrieval System(MEDLARS) and the National Library of Medicine(NLM). Panel comments were helpful in all three areasof study and were instrumental in determining the roleof all three areas in an assessment strategy.

OTA staff produced two staff papers for congres-sional hearings held by the House Energy and Com-merce Committee and the Senate Labor and HumanResources Committee in March 1981. One paper dealtwith NLM, the other with the National Center forHealth Care Technology (NCHCT). After the hear-ings, both papers were reviewed widely and weresubsequently incorporated in this report and relatedtechnical memorandum.

A workshop on reimbursement policies and innova-tion met in Washington on May 13, 1981. The pur-pose of this workshop was to review the papers byReiss and by Bunker and generally to discuss the ef-fects that medical technology assessment and reim-bursement policy have on innovation. In addition tothe authors and OTA staff, workshop participants in-cluded several members from the study advisory panel,an inventor, representatives from industry and re-searchers and academicians interested in medical tech-nology innovation. The workshop was helpful to Reissand Bunker in revising their papers and was particular-ly helpful to OTA in understanding both the innova-tion process and the effects that an assessment policymay have on that process.

While the main project was proceeding, several sub-components began to take on added significance asseparate projects in their own right. Following theMarch Senate Labor and Human Resources Commit-tee hearing, OTA was asked by Chairman Hatch toprepare a separate technical memorandum by expand-ing the NLM component of the study to examine therole of both the private and public sectors in produc-ing and vending bibliographic data bases. In addition,a separate volume concerning the postmarketing sur-veillance of drugs was prepared in draft. Subsequent-

—

Iy, OTA was asked by the House Committee on Ener-gy and Commerce to elevate the postmarketing sur-veillance effort to full report status (i.e., complete withoptions). It is being published as volume II of thisassessment. And finally, the House Committee onEnergy and Commerce also asked OTA to write aseparate report on the effects which competitive healthcare system proposals may have on medical technol-ogies. The Senate Committee on Labor and HumanResources endorsed that request.

Prior to the third panel meeting, held on September22, 1981, an initial draft of the final report wasprepared and sent to panel members. The entire meet-ing was spent reviewing that draft and focused primari-ly on the policy analysis and options for Congress.

The draft was then revised by OTA staff on the basisof the suggestions and comments of the advisory panel.The revised draft was then sent for a further roundof review by a much broader range of experts in a di-versity of settings: Federal agencies, private and non-profit organizations, academic institutions, practicing

health professionals, consumer groups, and other se-lected individuals. Altogether, more than 150 in-dividuals or organizations were asked to comment ondrafts of the main volume or other components of thisassessment. The main volume, containing policy op-tions, was reviewed by approximately 100. Followingrevisions by OTA staff, the report was submitted tothe Technology Assessment Board.

Description of Other Volumes

This assessment has resulted in seven documents:1. the main report, of which this index is a part;2. a brochure that summarizes the main report;3. a staff paper on NCHCT, issued to Congress in

March 1981;4. a staff paper on NLM, also issued to Congress

in March 1981;S. a monograph published by Rand Corp. entitled

Peer Review and Technology Assessment inMedicine;

6. a full report, which is volume II of this assess-ment, entitled Postmarketing Surveillance of Pre-scription Drugs; and

7. a technical memorandum on MEDLARS andHealth Information Policy.

Brief descriptions of the last two volumes are pro-vided below. Also described below is a volume entitledMedical Technology Under Proposals To IncreaseCompetition in Health Care. This report grew out ofthe Strategies assessment but is now being publishedseparately.


Postmarketing Surveillance of PrescriptionDrugs (Vol. II of Strategies forMedical Technology Assessment)

To market a drug, manufacturers must provide evi-dence of its efficacy and safety to the U.S. Food andDrug Administration (FDA). Once these premarketingrequirements are met and a drug is released, FDA cansuggest but cannot impose restrictions on the drug’suse. However, it can remove the drug from the marketfor reasons such as new evidence on safety or efficacy,any untrue statement of a material fact, or failure tomeet manufacturing standards.

In the premarketing clinical tests, controlled clinicaltrials with limited numbers of test subjects are used.Observation of limited numbers of patients for a shortperiod of time uncovers minimal information abouta drug’s potential uses and dangers, and postmarketingactivities of various types have been proposed overthe past decade. However, postmarketing surveillancehas been linked to other policy objectives, such asspeeding up the premarketing approval process andusing postmarketing information to improve physicianprescribing practices.

In OTA’s report, the analysis of these issues isframed around the following questions: 1) what as-pects of the premarketing requirements might be cur-tailed to shorten the drug approval process? 2) whatadditional powers would help strengthen FDA’s activ-ities in the postmarketing period? and 3) what possi-ble tradeoffs might there be between curtailing somepremarketing requirements and strengthening FDA’srole in monitoring drugs once they are released intothe marketplace?

MEDLARS and Health Information Policy

This technical memorandum examines the role ofNLM’s computerized bibliographic retrieval and tech-

nical processing system, MEDLARS, with respect tothe role of private sector information systems in thecreation and distribution of computerized biblio-graphic health-related information.

The study examines two specific sets of issues: therange of NLM’s computerized products and services,and NLM’s pricing structure for leasing data base tapesand for online access to the data bases. It focuses onthe domestic and international implications of theseissues and stresses the importance of new and emerg-ing computer and communications technologies onbiomedical information policy.

The issues are considered within a general frame-work of the Government’s role in the allocation ofresources to information development and distribu-tion, the effect of the Government’s involvement inallocative activities on certain segments of the privateinformation sector and the health community, and thehistoric role of the Government in health informationactivities.

Medical Technology Under ProposalsTo Increase Competition in Health Care

Proposals to stimulate competition in medical caretend to fall into three categories: 1) increased cost shar-ing by patients for services, 2) increased competitionand hence greater pressure for efficiency among healthplans or providers, and 3) increased antitrust activities.OTA’s study considers only the first two.

The study focuses on the effects and policy implica-tions of the different proposals in three major areasof medical care: 1) consumer information, 2) qualityof care, and 3) technology innovation and use. In eachof these areas, the study examines the situation thatwould pertain under each type of proposal and anydifferences from the present situation, effects thatcould be expected, any problems that would arise, andmethods of addressing these problems.

Appendix H.— Acknowledgments

OTA Contractors

The following individuals participated in preparingbackground papers or other material for this study.Many of them also provided valuable advice and com-ments during the course of the assessment.

Herbert AbramsStanford University

Robert H, BrookRand Corp.

Alain C. EnthovenStanford University

Renee FordTifford Producers, Ltd.

Jinnet FowlesStanford University

Peter G. GoldschmidtVeterans Administration

H. Diane GoroumStanford University

Katherine JonesStanford University

Aida A. LeRoyHealth Information Designs

Kathleen N. LohrRand Corp.

Barbara S. LynchRockville, Md.

Margaret MarnellStanford University

M. Lee MorseHealth Information Designs

David SawiStanford University

Ralph SchaffanzickBlue Shield, North Carolina

Anne A. ScitovskyPalo Alto Research Foundation

John E. WennbergDartmouth College

John D. WinklerRand Corp.

Patricia WoolfPrinceton University

Study Workshops

Two helpful workshops were held during the courseof this assessment, one on methodological issues, theother on reimbursement and innovation issues. OTAappreciates the participation of the following people,many of whom also provided thoughtful commentson drafts of papers and of this report.

Robert AustrianUniversity of Pennsylvania

David BlumenthalHarvard University

Lester BreslowUniversity of California, Los Angeles

John P. BunkerStanford University

Ralph D’AgostinoBoston University

D. V. d’ArbeloffMillipore Corp.

Harvey FinebergHarvard University

Stanley B. JonesBlue Cross/Blue

Alan KahnMadison, Wis.

John KimberlyYale University

Aida A. LeRoy

Shield

Health Information Designs

M, Lee MorseHealth Information Designs

Vernon L. NickelSharp Rehabilitation Center

John B. ReissBaker & Hostetler

Richard A. RettigRand Corp.

Louise B. RussellThe Brookings Institution

Leonard SaxeBoston University

Alexander M. WalkerHarvard University

195


Richard N. WatkinsGroup Health Cooperative

John E. WennbergDartmouth Medical School

John W. WilliamsonThe Johns Hopkins University

Paul M. WortmanUniversity of Michigan

Reviewers

A large number of individuals provided valuable ad-vice and comments to OTA during this assessment.OTA is grateful for their assistance. In particular, wewould like to thank the following people:

John C. Bailer, IIIEnvironmental Protection Agency

Constance BartonNational Center for Health Care Technology

Dwight BlankenbakerNational Center for Health Care Technology

Queta BondInstitute of Medicine

John T. BruerRockefeller Foundation

Arthur CaplanHastings Center

Gerald CohenNational Center for Health Services Research

Thi D. DaoPharmaceutical Manufacturers Association

Carolyne K. DavisHealth Care Financing Administration

W. Palmer DearingBlue Cross/Blue Shield

Robert A. DerzonLewin & Associates

Manning FeinleibNational Institutes of Health

Deborah A. FreundUniversity of North Carolina at Chapel Hill

Sydney P. GallowayHealth Care Financing Administration

Stephen F. GibbonsDepartment of Health and Human Services

Donald GoldstoneNational Center for Health Services Research

Roger GrahamBlue Cross/Blue Shield

Nancy GreenspanHealth Care Financing Administration

Ruth S. HanftDepartment of Health and Human Services

Dorcas R. HardyDepartment of Health and Human Services

Jeffrey E. HarrisMassachusetts Institute of Technology

Itzhak JacobyNational Institutes of Health

Judith K. JonesFood and Drug Administration

John T. Kalberer, Jr.National Institutes of Health

James M. KapleHealth Care Financing Administration

Joel KavetDepartment of Health and Human Services

James R. KimmeyInstitute for Health Planning

Daniel KoretzCongressional Budget Office

Sam KorperDepartment of Health and Human Services

Peter KramerAlcohol, Drug Abuse, and Mental Health

Administration

Marvin KristeinNational Institutes of Health

Charles U. LoweNational Institutes of Health

David L. MartinHealth and Welfare, Canada

Alan E. MayersNational Center for Health Services Research

David McCallumNational Institues of Health

Mary Louise McIntyreHealth Care Financing Administration

H. Calvin MeadowsHealth Resources Administration

Peggy A. MillerKaye, Scholer, Fierman, Hays & Handler

Anthony T. MottFinger Lakes Health Systems Agency

Appendix H—Acknowledgments ● 197

Donald MurphyNational Institutes of Health

Donald MuseHealth Care Financing Administration

Beverlee A. MyersCalifornia Department of Health Services

Seymour PerryNational Center for Health Care Technology

Harold A. PincusNational Institute of Mental Health

Pierre RenaultNational Center for Health Care Technology

John RodakNational Center for Health Care Technology

Wayne I. RoeHealth Industry Manufacturers Association

Lawrence RoseNational Center for Health Services Research

Gerald RosenthalPublic Health Service

Theodore RyersonNational Technical Information Service

Linda SchaefferNational Center for Health Care Technology

Henry SchoolmanNational Library of Medicine

Stuart O. SchweitzerUniversity of California, Los Angeles

Ira D. SingerProject Hope

Kent A. SmithNational Library of Medicine

Helen SmitsYale University

Edward J. SondikNational Institutes of Health

Barbara StarfieldThe Johns Hopkins University

Michael A. StotoHarvard University

Stephen M. StoweUniversity of Southern California

Paul R. TorrensUniversity of California, Los Angeles

James R. UllomNational Center for Health Services Research

Donna ValtriOffice of Technology Assessment

Sally W. VernonUniversity of Texas, Houston

Judith L. WagnerUrban Institute

Monica WaltersNational Institutes of Health

John M. WeingerUniversity of Southern California

Victoria WeisfeldInstitute of Medicine

Norman WeissmanNational Center for Health Care Technology

Elizabeth WenchelArthur Young & Co.

Daniel WiklerUniversity of Wisconsin, Madison

Irwin WolksteinHealth Policy Alternatives, Inc.

Daniel I. ZwickBethesda, Md.

98-144 0 - 82 - 14

Appendix L —Glossary of Acronyms and Terms

Glossary

ADAMHA

AHAAKCUP

AMAANDABC/BSBHPBLSBRSCABGCAPD

CASCAT

CBACBO

CCUCDCCEA

of Acronyms

Alcohol, Drug Abuse, and MentalHealth Administration (PHS)

American Hospital AssociationArtificial Kidney-Chronic Uremia

program (NIH)American Medical Associationabbreviated new drug applicationBlue Cross/Blue ShieldBureau of Health Planning (HRA)Bureau of Labor Statistics (DOL)Bibliographic Retrieval Servicecoronary artery bypass graftcontinuous ambulatory peritoneal

dialysisChemical Abstracts Servicecomputerized axial tomography

(scanner)cost-benefit analysisCongressional Budget Office (U.S.

Congress)coronary care unitCenters for Disease Control (PHS)cost-effectiveness analysis

CEA/CBA cost-effectiveness analysis/cost-benefitanalysis (when referred to as a classof analytical techniques)

CEAP Clinical Efficacy Assessment ProjectCHAMPUS Civilian Health and Medical Program

CHSSCONCPHA

CTDHEW

DHHS

DIALOGDODDOEDOLECRIEFMESRDFDAGAO

HCFA

198

of the Uniformed ServicesCooperative Health Statistics Systemcertificate of needCommission on Professional and

Hospital Activitiescomputed tomography (scanner)Department of Health, Education, and

Welfare (now DHHS)Department of Health and Human

Services (formerly DHEW)DIALOG Information Services, Inc.Department of DefenseDepartment of EnergyDepartment of LaborEmergency Care Research Instituteelectronic fetal monitoringend-stage renal diseaseFood and Drug Administration (PHS)General Accounting Office (U.S.

Congress)Health Care Financing Administration

(DHHS)

HEW

HHS

HMOHRA

HSAHSQB

HUPIDEIHCE

IND

IOM

IPDKDCP

MEDLARS

MEDLINEMSAFPNASNASA

NCHCT

NCHS

NCHSR

NCINDANDINECGDNEINGTNHLBI

NIANIAAA

NIADDK

NIAID

NIAMDD

Department of Health, Education, andWelfare (now DHHS)

Department of Health and HumanServices (formerly DHEW)

health maintenance organizationHealth Resources Administration

(PHS)health systems agencyHealth Standards and Quality Bureau

(HCFA)Hospital Utilization Projectinvestigational device exemptionInstitute for Health Care Evaluation

(proposed)investigational exemption for a new

drugInstitute of Medicine (National

Academy of Sciences)intermittent peritoneal dialysisKidney Disease Control program

(PHS)Medical Literature Analysis and

Retrieval System (NLM)MEDLARS On-line (NLM)maternal serum alpha-fetoproteinNational Academy of SciencesNational Aeronautics and Space

AdministrationNational Center for Health Care

Technology (OASH)National Center for Health Statistics

(OASH)National Center for Health Services

Research (OASH)National Cancer Institute (NIH)new drug applicationnational death indexnonequivalent control group designNational Eye Institute (NIH)nominal group techniqueNational Heart, Lung, and Blood

Institute (NIH)National Institute on Aging (NIH)National Institute on Alcohol Abuse

and Alcoholism (ADAMHA)National Institute of Arthritis,

Diabetes, and Digestive and KidneyDiseases (NIH)

National Institute of Allergy andInfectious Diseases (NIH)

National Institute of Arthritis,Metabolism, and Digestive Diseases(now NIADDK)

Appendix l—Glossary of Acronyms and Terms ● 199—

NICHD

NIDA

NIDR

NIEHS

NIGMS

NIHNIMH

NINCDS

NLMNSFNTIS

OASH

OASPE

OHRST

OMAR

OPE

OPEL

OPPR

ORD

OTA

OTC

PASPHSPSRO

PTAPTCA

QALYR&DRCTSDCSEER

National Institute of Child Health andHuman Development (NIH)

National Institute on Drug Abuse(ADAMHA)

National Institute of Dental Research(NIH)

National Institute of EnvironmentalHealth Sciences (NIH)

National Institute of General MedicalSciences (NIH)

National Institutes of Health (PHS)National Institute of Mental Health

(ADAMHA)National Institute of Neurological and

Communicative Disorders andStroke (NIH)

National Library of Medicine (NIH)National Science FoundationNational Technical Information

Service (Department of Commerce)Office of the Assistant Secretary for

Health (DHHS)Office of the Assistant Secretary for

Planning and Education (DHHS)Office of Health Research, Statistics,

and Technology (OASH)Office for Medical Applications of

Research (NIH)Office of Planning and Evaluation

(OASH)Office of Planning, Evaluation, and

LegislationOffice of Policy Planning and

Research (HCFA)Office of Research and

Demonstrations (HCFA)Office of Technology Assessment

(U.S. Congress)over-the-counter (Drug Review

program) (FDA)Professional Activity StudyPublic Health Service (DHHS)Professional Standards Review

Organization, Office of PSRO(HCFA)

percutaneous transluminal angioplastypercutaneous transluminal coronary

angioplastyquality-adjusted life yearresearch and developmentrandomized clinical trialSystem Development Corp.Surveillance, Epidemiology,

Results program (NCI)and End

SSA Social Security Administration(DHHS)

TA technology assessmentTCC Technology Coordinating Committee

(DHHS)VA Veterans Administration

Glossary of Terms

Biomedical and behavioral research: A combinationof biological, medical, psychological, social, andphysical scientific investigations focused on eradi-cating disease and generating new scientific knowl-edge.

Cavitation financing method: The method of payingfor medical care on a fixed, periodic prepaymentbasis per individual enrolled in a health plan. Pay-ment by “cavitation” implies that the amount paidby the individual is independent of the number ofservices that individual has received.

Case-control study: An observational study design, re-ferred to by some authors as “retrospective,” inwhich individuals with a condition of interest (e.g.,a suspected adverse effect of a medical treatment),i.e., cases, are compared to individuals without thecondition, i.e., controls, with respect to factors(e.g., previous exposure to the treatment) whichare judged relevant.

Certificate of need (CON): A regulatory planningmechanism required by the National Health Plan-ning Resources Development Act of 1974 to con-trol large health care capital expenditures. EachState is required to enact a CON law. CON appli-cations by institutions are reviewed by local healthsystems agencies, who recommend approval or dis-approval; they are denied or approved by Statehealth planning and development agencies.

Cohort study: An observational study design, referredto by some authors as “prospective,” in which two(or more) groups who vary with respect to theirexposure to a factor of interest (e. g., a treatmentmethod) are observed over a period of time. Thestatus of individuals in all groups is assessed afteran appropriate interval, and the outcomes com-pared to determine the effect of the factor ofinterest.

Consensus development conference: A process inwhich biomedical researchers, practicing healthprofessionals, and others, as appropriate, arebrought together by the National Institutes ofHealth to explore publicly the scientific back-ground, state of knowledge, proper use(s), and anyother issues pertinent to the technology under con-sideration.


Cost-benefit analysis (CBA): An analytical techniquethat compares the costs of a project or techno-logical application to the resultant benefits, withboth costs and benefits expressed by the samemeasure. This measure is nearly always monetary.

Cost-effectiveness analysis (CEA): An analyticaltechnique that compares the costs of a project orof alternative projects to the resultant benefits, withcosts and benefits/effectiveness expressed by dif-ferent measures. Costs are usually expressed in dol-lars, but benefits/effectiveness are ordinarily ex-pressed in terms such as “lives saved,” “disabilityavoided, “ “quality-adjusted life years saved, ” orany other relevant objectives. Also, when benefits/effectiveness are difficult to express in a commonmetric, the may be presented as an “array”.

CEA/CBA: A composite term referring to a family ofanalytical techniques that are employed to com-pare costs and benefits of programs or technol-ogies. Literally, the term as used in this assessmentmeans “cost-effectiveness analysis/cost-benefitanalysis.”

Data base: An organized collection of information inmachine-readable form and accessible by com-puter.

Device (medical): Any physical item, excluding drugs,used in medical care (including instruments, ap-paratus, machines, implants, and reagents).

Discounting: A procedure used in economic analysisto reduce to present value those costs and effectsthat occur in future years. Discounting is based ontwo premises: 1) individuals prefer to receive bene-fits today rather than in the future; and 2) resourcesinvested today in alternative programs could earna return over time.

Distributive justice: A philosophical concept whoseobjective is to ensure that benefits in society areallocated in proper proportion to each individual’slegitimate claim to them.

Drug: Any chemical or biological substance that maybe applied to, ingested by, or injected into humansin order to prevent, treat, or diagnose disease orother medical conditions.

Effectiveness: Same as efficacy (see below) except thatit refers to “ . . . average or actual conditions ofuse. ”

Efficacy: The probability of benefit to individuals ina defined population from a medical technologyapplied for a given medical problem under idealconditions of use.

Epidemiology: The study of the distribution, deter-minants, and control of diseases in humanpopulations.

Experimental method: Any method of hypothesis-test-ing in which the investigator controls the applica-tion or withholding of the factor under study toindividuals (or animals). Clinical trials (with con-trol groups) of all types fall into this category.

Fee-for-service: A method of paying for medical careon a retrospective basis by which each service ac-tually received by an individual bears a relatedcharge.

Health maintenance organization (HMO): A healthcare organization that acts as both insurer and pro-vider of comprehensive but specified medical serv-ices by a defined set of physicians to a voluntarilyenrolled population paying a prospective per capitafee (i.e., paying by “cavitation”).

Health services research: A field of inquiry that focuseson the structure, production, distribution, and ef-fects of delivering personal health services.

Health systems agency (HSA): One of the approx-imately 200 local health planning agencies desig-nated under the National Health Planning and Re-sources Development Act of 1974 to develop localhealth planning goals and implement plans in con-sonance with State and national health care goals.HSAs are federally funded and are governed by abody which is broadly representative of both pro-vider and consumer interests, the latter being in themajority.

Incidence: In epidemiology, the number of cases ofdisease, infection, or some other event having theironset during a prescribed period of time in rela-tion to the unit of population in which they occur.It measures morbidity or other events as they hap-pen over a period of time.

Marginal benefit: An economic concept referring tothe additional benefit achieved by incurring an ad-ditional unit of cost.

Marginal cost: An economic concept referring to theadditional cost of achieving one more unit of ben-efit.

Medicaid: A Federal program that is administered andoperated individually by each participating Stategovernment that provides medical benefits to cer-tain low-income persons in need of health andmedical care.

Medical technology: The drugs, devices, and medicaland surgical procedures used in medical care, and

Appendix l—Glossary of Acronyms and Terms ● 201

the organizational and supportive systems withinwhich such care is provided.

Medicare: A nationwide, federally administered healthinsurance program authorized in 1965 to cover thecost of hospitalization, medical care, and somerelated services for eligible persons over age 65,persons receiving Social Security Disability In-surance payments for 2 years, and persons withend-stage renal disease. Medicare consists of twoseparate but coordinated programs—hospital in-surance (part A) and supplementary medical in-surance (part B). Health insurance protection isavailable to insured persons without regard toincome.

MEDLARS: The computerized Medical LiteratureAnalysis and Retrieval System of the National Li-brary of Medicine. Available through a networkof centers at more than 1,000 universities, medicalschools, hospitals, Government agencies, and com-mercial organizations, MEDLARS contains some4,500,000 references to journal articles and booksin the health sciences published after 1965.

MEDLINE (MEDLARS On-line): The National Libraryof Medicine’s online data base containing approx-imately 600,000 references to biomedical journalarticles published in the U.S. and 70 foreign coun-tries in the current and preceding 2 years.

Morbidity: A measure of illness, injury, or disabilityin a defined population. It is usually expressed ingeneral or specific rates of incidence or prevalence.Sometimes used to refer to any episode of disease.See also “mortality (death).”

Mortality (death): A measure of deaths, used todescribe the relation of deaths to the populationin which they occur. The mortality rate (death rate)expresses the number of deaths in a unit of popula-tion within a prescribed time.

Observational method: Any method of hypothesis-testing in which the investigator does not controlthe application or withholding of the factor understudy to individuals (or animals).

On-line: A term applied to a computerized “inter-active” information retrieval system that allows aninformation specialist (or other user) sitting at aremote processing facility (i.e., typewriter or videoterminal) to engage in a direct dialog with a cen-tral computer on which information (e.g., databases, indexes) is stored, and thus to have im-mediate access to that information. The centralcomputer and the information stored on the com-puter are said to be on-line to the remote process-ing facility (ies).

Prevalence: In epidemiology, the number of cases ordisease, infected persons, or persons with disabili-ties or some other condition, present at a particulartime and in relation to the size of the population,It is a measure of morbidity at a point in time.

Procedure (medical or surgical): A medical technologyinvolving any combination of drugs, devices, andprovider skills and abilities. Appendectomy, for ex-ample, may involve at least drugs (for anesthesia),monitoring devices, surgical devices, and theskilled actions of physicians, nurses, and supportstaffs.

Professional Standards Review Organizations(PSROs): Community-based, physician-directed,nonprofit agencies established under the SocialSecurity Amendments of 1972 to monitor the quali-ty and appropriateness of institutional health careprovided to Medicare and Medicaid beneficiaries.

Randomization: The assignment by an investigator ofindividuals to treatment or control groups basedon chance alone.

Randomized clinical trial (RCT): An experimentaldesign by which human or animal subjects are ran-domly assigned, either to an experimental group(in which subjects receive the treatment beingstudied) or to a control group (in which subjectsdo not receive the treatment being studied). Alsoreferred to as “randomized controlled clinical trial”or “controlled clinical trial. ”

Reliability: A measure of the consistency of a methodin producing results. A reliable test gives the sameresults when applied more than once under thesame conditions. Also called “precision. ”

Risk: A measure of the probability of an adverse oruntoward outcome and the severity of the result-ant harm to health of individuals in a defined pop-ulation associated with use of a medical technologyapplied for a given medical problem under specifiedconditions of use.

Risk-benefit analysis: The formal comparison of theprobability and level of adverse or untoward out-comes v. positive outcomes for any given action.The comparison of outcomes does not take intoconsideration the resource costs involved in the in-tended action.

Safety: A judgment of the acceptability of risk (seeabove) in a specified situation.

Technology: The application of organized knowledgeto practical ends.

Technology assessment: A comprehensive form ofpolicy research that examines the technical, eco-nomic, and social consequences of technological

202 ● Strategies for Medical Technology Assessment——.

applications. It is especially concerned with unin-tended, indirect, or delayed social impacts. Inhealth policy, the term has also come to mean anyform of policy analysis concerned with medicaltechnology, especially the evaluation of efficacyand safety. The comprehensive form of technologyassessment is then termed “comprehensive technol-ogy assessment. ”

Validity: A measure of the extent to which an observedsituation reflects the “true” situation. Internal

validity is a measure of the extent to which studyresults reflect the true relationship of a “risk fac-tor” (e.g., treatment or technology) to the outcomeof interest in study subjects. External validity is ameasure of the extent to which study results canbe generalized to the population which is repre-sented by individuals in the study which assumesthat the characteristics of that population are ac-curately specified.

.

References

(References for authored papers in apps. E and F arecited

1.

2.

3.

4.

5.

6.

7.

8.

9,

10,

11,

12.

at the end of each paper. )

“Adjuvant Chemotherapy of Breast Cancer, ” Na-tional Institutes of Health Consensus-Develop-ment Conference Statement, N. Eng. J Med.303(14):831, 1980.Almy, T., Sounding Board, “The Role of the Pri-mary Physician in the Health-Care Industry, ” N.Eng. -1. Med. 304(4):225, 1 9 8 1 .Altman, S. H., and Blendon, R. (eds. ), MedicalTechnology: The Culprit Behind Health CareCosts, proceedings of the 1977 Sun Valley Forumon National Health, publication No. (PHS)79-3216 (Washington, D. C.: U.S. GovernmentPrinting Office, 1979).American College of Physicians, Philadelphia,Pa., “ACP News . . . ,“ Nov. 24, 1980.American College of Physicians, Philadelphia,Pa., “Clinical Efficacy Assessment Project: A Pro-posal for Expansion, ” mimeo, n.d.Applied Management Sciences, Inc., and CuadraAssociates, Inc., Catalog of Human Services In-formation Resources Organizations, prepared forthe Office of the Assistant Secretary for Planningand Evaluation, Department of Health and Hu-man Services, contract No. HEW 100-79-0183,June 1980; Project Share Human Services A40no-graph No. 15, October 1980.Arnstein, S. R., “Technology Assessment: Op-portunities and Obstacles,” L5’EE Trans. Syst.Man & Cybern. SMC-7:571, 1977.Arthur Young & Co., and Emergency Care Re-search Institute, A Profile of the MedicaZ TwhnoZ-ogy Industry and Government PoZicies, draft finalreport to the National Center for Health ServicesResearch, Hyattsville, Md., Mar. 31, 1981.Astvad, K., et al., “Mortality From Acute Myo-cardial Infarction Before and After Establishmentof a Coronary Care Unit, ” Br. Med. J. 1:567,1974.Bader, M., letter to the editor, N. Eng. J Med.96(1):46, 1977.Banta, H. D., “Biomedical Research and Technol-ogy Assessment in the Major Medical Clinics, ”presented at the Project HOPE Conference, “TheRole of Major Medical Clinics in Meeting FutureNational Health Needs, ” Millwood, Va., Mar.11-13, 1981.Banta H. D., “The Diffusion of the ComputedTomography (CT) Scanner in the United States, ”Int. J. Health Serv. 10(2):251, 1981.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

Banta, H. D., and Behney, C. J., “Medical Tech-nology: Policy and Problems, ” HeaZth Care Mgt.Rev. 5:45, 1980.Banta, H. D., and Behney, C. J., “Policy Formu-lation and Technology Assessment,” prepared forthe Experiment and Social Policy Issue of Healthand Society, Mi]bank Mere. Fund Q. 59(3):445,1981.Banta, H. D., and McNeil, B. J., “Evaluation ofthe CAT Scanner and Other Diagnostic Technol-ogies,” Health Care Mgt. Rev. 3:7, 1978.Banta, H. D., and Sanes, J. R., “Assessing theSocial Impacts of Medical Technologies, ” J.Comm. Health 3(3):245, 1978.Banta, H. D., and Thacker, S. B., “Assessing theCosts and Benefits of Electronic Fetal Monitor-ing,” Obstet. & Gyn. Survey 34:627, 1979.Banta, H. D., et al., “Implications of the 1976Medical Devices Legislation,” Man & Medicine3:131, 1978.Banta, H. D., et al., Toward Rational Technol-ogy in Medicine (New York: Springer PublishingCo., 1981).Barsamian, E. M., “The Rise and Fall of InternalMammary Artery Ligation in the Treatment ofAngina Pectoris and the Lessons Learned, ” inCosts, Risks, and Benefits of Surgery, J. P.Bunker, et al. (eds. ) (New York: Oxford Univer-sity Press, 1977).Bauer, R. A., and Wortzel, L. H., “DoctorsChoice: The Physician and His Sources of Infor-mation About Drugs, ” J. Mktg. Research 3:4o,1966.Beebe, G. W., “Record Linkage and Needed Im-provements in Existing Data Resources, ” pre-sented at the Banbury Conference on Occupa-tional Cancer, Cold Spring Harbor Laboratory,Cold Spring Harbor, N. Y., Mar. 29-Apr. 2, 1981,Beebe, G. W., “Record Linkage Systems—Can-ada vs. the United States, ” Am. J. Public Health70(12):1246, 1980.Beebe, G. W., “Some Aspects of Record Linkagein Epidemiologic Research in the U.S., ” paperpresented at the Workshop on Computerized Rec-ord Linkage in Cancer Epidemiology, Ottawa,Canada, Aug. 8-10, 1979.Behney, C. J., “The Turning Outward of the Na-tional Institutes of Health,” presented at the 13thAnnual Meeting of the Association for the Ad-vancement of Medical Instrumentation, Washing-ton, D. C., Apr. 1, 1978.

205


26.

27.

28.

29.

30.

31.

32.

33.

34,

35.

36.

374

38,

39,

Berk, A. A., and Chalmers, T. C., “Cost and Ef-ficacy of the Substitution of Ambulatory for In-patient Care,” N. Ehg. J. Med. 304:393, 1981.Bessant, J., “Innovation Research: To What Ex-tent Can It Provide Answers?” Planned hmova-tion 2(2):52, 1979.Bice, T., et al., Executive Summary of the Con-fidence on Capital Fhancing for Health Facilities,sponsored by the Health Resources Administra-tion and the University of Pittsburgh, Pittsburgh,Pa., Nov. 19-21, 1976,Bilitch, M., Project Director, University of South-ern California, Los Angeles, California, l?egist~for Implanted Artificial Cardiac Pacemakers: 66Month Report, contract No. 223-79-5060, sub-mitted to the Food and Drug Administration,Rockville, Md., Mar. 28, 1980.Birch & Davis Associates, Inc., IDD EvaluationProject Hospital FieZdManual, contract No. 500-80-0047, submitted to the Health Care FinancingAdministration, Washington, D. C., 1980.Birch & Davis Associates, Inc., IDD GranteeProject Synopsis, contract No. 500-80-0047, sub-mitted to the Health Care Financing Administra-tion, Washington, D. C., Dec. 17, 1980.Birch & Davis Associates, Inc., Integrated DataBmonstration Evaluation Study, contract No.500-80-0047, submitted to the Health Care Fi-nancing Administration, Washington, D. C., Oct.8, 1980.The Blue Sheet, “Coordinated Health Informa-tion System Feasibility ‘Yet To Be Demonstrated, ’HHS’ Brandt Tells Hill; Cost, Quality, Confiden-tiality Are Seen As Problems, ” 24:9, June 10,1981.Bonchek, L. I., “Are Randomized Trials Appro-priate for Evaluating New Operations?” N. Eng.J. Med. 301:44, 1979.Breslow, L., University of California, Los Ange-les, personal communication, 1981.Bryant, F. B., and Wortman, P. M., “SecondaryAnalysis: The Case for Data Archives,” Am. Psy-chol. 33:381, 1978.Bumpus, F. M., “The Role of Major Medical Clin-ics in Research, ” presented at the Project HOPEconference, “The Role of Major Medical Clinicsin Meeting Future National Health Needs, ” Mill-wood, Va., Mar. 11-1,3, 1981.Bunker, J. P., Stanford, Calif., personal commu-nication, June 16, 1981.Bunker, J. P., et al., “Assessment of MedicalTechnology Strategies: Coverage and Reimburse-ment Options for Experimental Medical and Sur-gical Procedures, ” prepared for the Office of

40.

41.

42.

43.

44,

45.

46

47

48.

49.

50.

51

52.

53.

Technology Assessment, U.S. Congress, Wash-ington, D. C., September 1981.Bunker, J. P., et al., “Surgical Innovation and ItsEvaluation,” %ience 200:937, 1978.Bunning, R. L. “The Delphi Technique: A Projec-tion Tool for Serious Inquiry,” in The 1979 h-nual Himdbook for Group Facilitators, J. E. Jones,J. W, Pfeiffer (eds. ) (La Jolla, Calif.: UniversityAssociates, 1979).Byar, D. P., et al., “Randomized Clinical Trials:Perspectives on Some Recent Ideas, ” N. Eng. J.Med. 295(2):74, 1976.Campbell, D. T., and Boruch, R. F., “Making theCase for Randomized Assignment to Treatmentsby Considering the Alternatives: Six Ways inWhich Quasi-Experimental Evaluations in Com-pensatory Education Tend To Underestimate Ef-fects,” in Evaluation and &periment: Some Giti-cal Issues in Assessing Social Programs, C. A.Bennett and A. A. Lumsdaine (eds. ) (New York:Academic Press, 1975).Campbell, D. T., and Erlebacher, A., “How Re-gression Artifacts in Quasi-Experimental Evalua-tions Can Mistakenly Make Compensatory Edu-cation Look Harmful, ” in Compensatory Educa-tion: A NationaZ Debate, J. Helmuth (cd.), vol.111 of The ll”sadvantaged Child (New York: Brun-ner/Mazel, 1970).Campbell, D. T., and Stanley, J. C., lkperimen-tal and Quasi-Experimental Bsi@s for Research(Chicago: Rand-McNally, 1966).CapIan, A. L., Hastings Center, Hastings-on-the-Hudson, N. Y., personal communication, Novem-ber 1981.Caplan, A. L., “Ethical Engineers Need Not Ap-ply: The State of Applied Ethics Today, ” Sci.Technol Human VaZ. 6(33):24, 1980.Caplan, A. L., “Kidneys, Ethics, and Politics:Policy Lessons of the ESRD Experience,” J. HeaZthPolit. Policy Law 6:488, 1981.Caplow, T., Principles of Organization (NewYork: Harcourt, Brace & World, 1964).Center for the Analysis of Health Practices, Har-vard School of Public Health, “Impact on HealthCosts of NCHCT Recommendations To With-hold Reimbursement for New Medical Proce-dures,” unpublished, February 1981.Chalmers, T. C., “The Clinical Trial, ” MilbankMere. Fund Q. 59(3):324, 1981.Chalmers, T. C., “Randomization Coronary Ar-tery Surgery,” Ann. Thorac. Surg. 14:323, 1972.Chalmers, T. C., et al., “Controlled Studies inClinical Cancer Research, ” N. Eng. J. Med.287:75, 1972.

References . 207

54.

55.

56.

57.

58.

59.

60.

61,

62.

63.

64.

65,

66.

67.

68.

69.

Chalmers, T. C., et al., “A Method for Assess-ing the Quality of a Randomized Clinical Trial, ”Contr. C/in. Triak 2:31, 1 9 8 1 .Charles River Associates, Inc. “The Effects ofPatent-Term Restoration on the PharmaceuticalIndustry,” prepared for the Office of TechnologyAssessment, U. S. Congress, Washington, D. C.,May 4, 1981.Charters, A. C., and Bailey, R. N., “ExperienceWith a Simplified Trauma Registry: Profile ofTrauma at a University Hospital, ” J Trauma19:13, 1979.Childress, J. F., Priorities in Biomedical Ethics(Philadelphia: Westminster Press, 1981).Christenson, D. B., and Wertheimer, A. I.,“Source of Information and Influence on NewDrug Prescribing Among Physicians in anHMO,” Soc. Sci. Med. 13A:313, 1979.Clarke, R. V. G., and Cornish, D. B., The Con-trolled Trial Institutional Research (London:Home Office Research Studies, 1972).Cochrane, A. L., Effectiveness and Efficiency:Random Reflections on Health Services (London:Burgess, 1972).Cole, P., “The Evolving Case-Control Study,” JCYwon. Dis. 32:15, 1979.Coleman, J. S., et al., MedikaZ Innovation: A Dif-fusion Study (Indianapolis: Bobbs-Merrill, 1966).Committee on the Life Sciences and Social Policy,Assessing Biomedical Technologies: An InquiryInto the Nature of the Process (Washington,D. C.: National Academy of Sciences, 1975).Comroe, J. H., Jr., and Dripps, R. D., The TopTen Clinical Advances in Cardiovascular-Pulmo-nary Medicine and Surgery, 1945-1975, vol. I,Final Report, DHEW Publication No. (NIH) 78-1521, Jan. 31, 1977.Congressional Research Service, Library of Con-gress, Competitive Problems in the Drug Indus-try: Fixed-Dose Combination Antibiotic Drugs,prepared for the Select Committee on Small Busi-ness, U.S. Senate, May 1, 1979 (Washington,D. C.: U.S. Government Printing Office, 1979).Conner, R. F., “Selecting a Control Group: AnAnalysis of the Randomization Process in TwelveSocial Reform Programs, ” Eva]. Q. 1:195, 1977.“Consensus Conference: Diagnosis and Treat-ment of Reye’s Syndrome, ” J. A.M.A. 246:2441,1981.Cook, T. D., and Campbell, D. T., Quasi-13qeri-mentation: Zkign and Analysis of Research inField Settings (Chicago: Rand McNally, 1979).Cook, T. D., and Leviton, L. C., “Reviewing theLiterature: A Comparison of Traditional Methods

70,

71,

72<

73,

74$

75.

76,

77,

78,

With Meta-Analysis, ” J. Personality 48(4):449,1980.Cook, T. D., and Reichardt, C. S., “Guidelines:Statistical Analysis of Nonequivalent ControlGroup Designs: A Guide to Some Current Litera-ture,” Evaluation 3(1-2):136, 1976.Cordray, D. S., and Lipsey, M. W., “Usir-tg Pat-terns of Evidence in Social Program Evaluation:An Overview and Illustration, ” presented at themeeting of the Evaluation Research Society, Min-neapolis, 1979.Coronary Drug Project Research Group, “Influ-ence of Adherence to Treatment and Response ofCholesterol on Mortality in the Coronary DrugProject,” N. Eng. J. Med. 303:18, 1980.Cowley, M. J., and Block, P. C., “PercutaneousTransluminal Coronary Angioplasty, ” M o d .Concepts Cardiovasc. Dis. 50(5):2, 1981.Crawford, S., “Health Science Libraries in theUnited States, 1969 to 1979, ’’J. A. A4. A. 245:237,1981.Cromwell, J., et al., Incentives and DecisionsUnderlying Hospitals’ Adoption and Utilizationof Major CapitaZ Equipment (Cambridge, Mass,:Abt Associates, 1975).Cronbach, L. J., and Furby, L., “How Should WeMeasure ‘Change’—Or Should We?” Psycho].Bull. 74:68, 1970.Dalkey, N. C., The Delphi Method: An Experi-mental Study of Group Opinion (Santa Monica,Calif.: Rand Corp., 1969).Dalkey, N. C., and Helmer, O., “An Experimen-tal Application of the Delphi Method t: the Useof Experts,” A@. Sci. 9(3):458, 1963.

79. D’Arbeloff, D. V., Chairman and Chief Executive

80,

81

Officer, Millipore Corp., statement at Workshopon Innovation and Reimbursement for the StudyStrategies for Medical Technology Assessment,Office of Technology Assessment, U.S. Congress,Washington, D. C,, May 13, 1981.David, G,r Commission on Professional and Hos-pital Activities, Ann Arbor, Mich., personal com-munication, May 21, 1981.Deckert, T., et al., “Prognosis of Diabetics WithDiabetes Onset Before tie Age of Thirty-One,’i

Diabetologia 14:363, 1978.82. Delbecq, A., et al., Group Techniques for Pro-

gram Planning (Glenview, Ill.: Scottf Foresman& co., 1975).

83. Department of Commerce, National TechnicalInformation Service, Biomedical TechnoZogyana’Human Factors Engineering, Aug. 5, 1980,PB80-909532 (Springfield, Va.: NTIS, 1981).

84. Department of Commerce, National Technical


85,

86.

87.

88.

89.

90.

91.

92,

93.

94.

Information Service, General Catalog of I.nfor-rnation %-vices, No. 7a (Springfield, Va.: NTIS,1981).Department of Commerce, National TechnicalInformation Service, Health PZanning artd HealthServices Research, May 19, 1981, PB81-904420 (Springfield, Va.: NTIS, 1981).Department of Commerce, National TechnicalInformation Service, JVTIS Information Servicesfor Productivity, Technology, and Innovation,information packet (Springfield, Va.: NTIS,1981).Department of Health, Education, and Welfare,National Center for Health Services Research,Medical Technology, publication No. (PHS)79-3254 (Washington, D. C.: U.S. GovernmentPrinting Office, 1979).Department of Health, Education, and Welfare,National Commission for the Protection of Hu-man Subjects of Biomedical and Behavioral Re-search, The Belmont Report, DHEW publicationNo. (OS) 78-0012, 1978.Department of Health, Education, and Welfare,National Commission for the Protection of Hu-man Subjects of Biomedical and Behavioral Re-search, The S’cial Study: Implications of Ad-vances in Biomedical and Behavioral Research,1978.Department of Health, Education, and Welfare,Office of the Assistant Secretary for Health/Pub-lic Health Service, “NCHCT Fact Sheet,” n.d.Department of Health, Education, and Welfare,Office of the Assistant Secretary for Plaming andEvaluation and the Office of the Assistant Secre-tary for Health, Health Tmhno]ogy Managementat the Department of Health, Education, andWelfare, Final Phase I Report for the Secretary,1977.Department of Health, Education, and Welfare,Public Health Service, Health Resources Adminis-tration, Bureau of Health Planning, The NationalHealth Planning Information Center, DHEWpublication No. (HRA) 79-14007, October 1978.Department of Health, Education, and Welfare,Public Health Service, National Center for HealthServices Research, “Clinical Databanks—An Epi-demiological Approach to Technology Assess-ment and Consensus Development, ” presented atthe XII International Conference on Medical andBiological Engineering, Jerusalem, Israel, Aug.19-24, 1979.Department of Health, Education, and Welfare,Public Health Service, National Center for HealthServices Research, Computer Applications in

95.

96.

97,

98.

99.

100.

101.

102.

103.

104(

1054

Health Care, DHEW publication No. (PHS) 79-3251, June 1979.Department of Health, Education, and Welfare,Public Health Service, National Institutes ofHealth, “Consensus Development ConferenceStatement,” N. fig. J. Med. 304:680, 1981.Department of Health, Education, and Welfare,Public Health Service, National Institutes ofHealth, Consensus Development ConferenceSummaries, vol. 1, 1977-1978.Department of Health, Education, and Welfare,Public Health Service, National Institutes ofHealth, Consensus Development ConferenceSummaries, vol. 2, 1979.Department of Health, Education, and Welfare,Public Health Service, National Institutes ofHealth, Consensus IXvelopment ConferenceSummaries, vol. 3, 1980.Department of Health, Education, and Welfare,Public Health Service, National Institutes ofHealth, NIH Inventory of Clinical Trials: FiscalYear 1975, vols. I and II, 1977.Department of Health, Education, and Welfare,Public Health Service, National Institutes ofHealth, “Program Description and Data Format:Cancer Surveillance Epidemiology and End Re-sults (SEER) Reporting Program, ” unpublished,April 1976,Department of Health, Education, and Welfare,Public Health Service, National Institutes ofHealth, “The Responsibilities of NIH at theHealth Research/Health Care Interface, ” draft,Feb. 14, 1978.Department of Health and Human Services,Health Care Financing Administration, “HealthCare Information System of the 80’s: HCFA’S Per-spective and Initiatives, ” discussion paper, May14, 1980.Department of Health and Human Services,Health Care Financing Administration, “Operat-ing Guidelines for Determining Medical Neces-sity of Medicare Items and Services, ” draft, Feb.2, 1971.Department of Health and Human Services,Health Care Financing Administration, Office ofResearch, Demonstrations, and Statistics, Grantsfor Research and LXmonstrations, Fiscal Year1981, HCFA Publication No. 03074, December1980.Department of Health and Human Services,Health Care Financing Administration, Office ofResearch, Demonstrations, and Statistics, “AGuide to ORDS Publications, ” mimeo, August1980.

References ● 209

106.

107.

108.

109.

110.

111

112.

113.

114.

115.

Department of Health and Human Services,Health Care Financing Administration, Office ofResearch, Demonstrations, and Statistics, “ORDSResearch Publications, ” mimeo, n.d.Department of Health and Human Services,Health Care Financing Administration, Office ofResearch, Demonstrations, and Statistics, “Re-search Publications Program, ” mimeo, n.d.Department of Health and Human Services, Of-fice of the Assistant Secretary for Health/PublicHealth Service, Maternal Serum Alpha-Fetopro-tein: Issues in the Prenatal Screening and Diagno-sis of Neural Tube Bfects, National Center forHealth Care Technology Conference Proceedings,Washington, D. C., July 28-30, 1980.Department of Health and Human Services,Office of the Assistant Secretary for Health/Pub-lic Health Service, National Center for HealthCare Technology, “Hemodialysis for Treatmentof Schizophrenia, ” memorandum to the HealthCare Financing Administration, Sept. 18, 1980.Department of Health and Human Services,Office of Assistant Secretary for Health/PublicHealth Service, National Center for Health CareTechnology, “Procedures, Priorities and Policyfor Assessment of Health Care Technology,”Sept. 1, 1980.Department of Health and Human Services,Office of the Assistant Secretary for Health/Pub-lic Health Service, National Center for HealthCare Technology, “The Process for AssessingTechnologies, ” Jan. 21, 1981.Department of Health and Human Services,Office of the Assistant Secretary for Health/Pub-lic Health Service, National Center for HealthCare Technology, Report to the Congress, 1980.Department of Health and Human Services,Office of the Assistant Secretary for Health/Pub-lic Health Service, National Center for HealthServices Research, Program Description, Jan. 9,1981.Department of Health and Human Services,Office of the Assistant Secretary for Health/Pub-lic Health Service, National Center for HealthStatistics, Data Systems of the National Centerfor Health Statistics, DHEW publication No.(PHS) 80-1247, 1980.Department of Health and Human Services,Office of the Assistant Secretary for Health/Pub-1ic Health Service, National Center for HealthStatistics, Directions for the ‘80s: Final Report ofthe Panel To Evaluate the Cooperative HealthStatistics System, DHHS publication No. (PHS)80-1204, August 1980.

116.

117.

118.

119.

120.

121.

122.

123.

124.

125.

Department of Health and Human Services,Office of the Assistant Secretary for Health/Pub-lic Health Service, National Center for HealthStatistics, Environmental Health: A PZan for Col-lecting and Coordinating Statistical and Epide-miologic Data, DHHS publication No. (PHS) 80-1248, 1980.Department of Health and Human Services,Office of the Assistant Secretary for Health/Pub-lic Health Service, National Center for HealthStatistics, HeaZth; United States, 1980, DHHSpublication No. (PHS) 81-1232, October 1980.Department of Health and Human Services,Office of the Assistant Secretary for Health/Pub-lic Health Service, National Council for HealthCare Technology, Annual Report, Fiscal Year1979, n.d.Department of Health and Human Services,Office of the Assistant Secretary for Health/Pub-lic Health Service, National Council for HealthCare Technology, minutes of meeting, Washing-ton, D. C., Aug. 13, 1981.Department of Health and Human Services,Office of the Assistant Secretary for Health/Pub-lic Health Service, Technology Coordinating

Committee, minutes of meeting, Apr. 27, 1981.Department of Health and Human Services,Office of the Assistant Secretary for Planning andEvaluation, “Project Share: A National Clearing-house for Improving the Management of HumanServices,” DHHS publication No. 0$76-221,1981.Department of Health and Human Services, Pub-lic Health Service, Food and Drug Administra-tion, Office of Planning and Evaluation, Econom-ics Staff, “Baseline Data on Medical Device Indus-tries in the Census of Manufacturers,” OPE study53, 1980.Department of Health and Human Services, Pub-lic Health Service, Health Resources Administra-tion, Bureau of Health Planning, “The NationalHealth Planning Information Center: Its Concep-tion, Development, and Organization, ” mimeo,n.d.Department of Health and Human Services, Pub-lic Health Service, Health Resources Administra-tion, Bureau of Health Planning, “services Pro-vided by the National Health Planning Informa-tion Center,” mimeo, June 1980.Department of Health and Human Services, Pub-lic Health Service, National Institutes of Health,NIHPublic Advisory Groups: Authority, Struc-ture, Functions, Members, publication No. (NIH)79-lI, Jan. 1, 1979.


126<

127.

128.

129

130.

131

132

133

134.

135.

136.

137

138

Department of Health and Human Services, Pub-lic Health Service, National Institutes of Health,National Heart, Lung, and Blood Institute, Officeof Information, “Coronary Artery SurgeryStudy,” unpublished, 1981.Department of Health and Human Services, Pub-lic Health Service, National Institutes of Health,National Heart, Lung and Blood Institute, Officeof Information, “The Framingham Heart Study, ”unpublished, 1981.Department of Health and Human Services, Pub-lic Health Service, National Institutes of Health,National Heart, Lung, and Blood Institute, Officeof Information, “Multiple Risk Factor Inter-vention Trial for the Prevention of CoronaryHeart Disease, ” n.d.Department of Health and Human Services, Pub-lic Health Service, National Institutes of Health,National Library of Medicine, Lister Hill NationalCenter for Biomedical Communications, “TheHepatitis Knowledge Base,” Arm. ht. Nfed.93(l) (Part 2):165, 1980.Dowling, H. F., cited in Competitive Problemsin the Drug Industry: FhedDose CombinationAntibiotic Drugs, prepared by the CongressionalResearch Service, Library of Congress, for theSelect Committee on Small Business, U.S. Senate,May 1, 1979 (Washington, D. C.: U.S. Govern-ment Printing Office, 1979).DuMouchel, W. H., and Harris, J. E., “BayesMethods for Combining Cancer Experiments inMan and Other Species,”J. Amer. Statist. Assoc.,in press, 1982.Duncan, K. A., Palo Alto, Calif. “The TrendToward a National Health Information Systemin the United States, ” unpublished paper, n.d.“Editorial: New Insulin-Delivery Systems forDiabetics,” Lancet 1:1275, 1979.Einhorn, H. J., and Hogarth, R. M., “Confidencein Judgment: Persistence in the Illusion of Validi-ty,” Psycho]. Rev. 85:395, 1978.Enzer, S., et al., Some Societal Changes by 1985and Their Impacts on Time/Money Bu~et (Mid-dletown, Corm.: Institute for the Future, 1971).European Coronary Surgery Study Group,“Coronary-Artery Bypass Surgery in Stable An-gina Pectoris: Survival at Two Years, ” Lancet1:889, 1979.Eysenck, H, F., “An Exercise in Mega-Silliness, ”Am. Psychol. 33:517, 1978.Federal Coordinating Committee on Science, En-gineering and Technology, Ad Hoc Committeeon Scientific and Technical Information, “Reportof the Working Group on Private Sector/Govem-

139.

140,

141,

142.

143.

144.

145.

146,

147,

148.

149.

150.

151.

152(

153<

ment Relations for Scientific and Technical Infor-mation, ” mimeo obtained from the National Sci-ence Foundation, Washington, D. C., June 1980.Feer, H., et al,, Compr. Psychiatry 1:338, 1960,cited in S. C. Schulz, et al., “Dialysis in Schizo-phrenia: A Double-Blind Evaluation, ” Science211:1066, 1981.Fein, R., “On Measuring Economic Benefits ofHealth Programs, “ in Ethics and Health Policy,R. M. Veatch and R. Branson (eds.) (Cambridge,Mass.: Ballinger, 1976).Feinstein, A. P., “A Survey of the Statistical Pro-cedures in General Medical Journals,” C’lin. Phar-macol. Therapeut. 15:97, 1975.Feldstein, M., and Taylor, A., The Rapid Riseof Hospital Costs (Washington, D. C.: President’sCouncil on Wage and Price Stability, 1977).Fineberg, H. V., “Gastric Freezing—A Study ofDiffusion of Medical Innovation, ” in MedicalTechnology and the Health Care System (Wash-ington, D. C.: National Academy of Sciences,1979).Fineberg, H. V., and Hiatt, H. H., “Evaluationof Medical Practices: The Case for TechnologyAssessment,” IV. fig. J. Med. 301:1086, 1970.Fineberg, H. V., et al., “Acquisition and Applica-tion of New Medical Knowledge by Anesthesiolo-gists: Three Recent Examples,” Anesthesiology48:430, 1978.Fishbein, H. A., State of Rhode Island Depart-ment of Health, Providence, R. I., personal com-munication, May 7, 1981.Fisher, B., “Breast-Cancer Management: Alter-natives to Radical Mastectomy, ” IV. fig. J. Med.301:326, 1979.Fletcher, R. H., and Fletcher, S. W., “ClinicalResearch in General Medical Journals, ” Al. Eng.J Med. 301:180, 1979.Frank, J. D., Persuasion and Healing (Baltimore:Johns Hopkins University Press, 1973).Frankina, W. K., Ethics (Englewood Cliffs, N. J.:Prentice-Hall, 1973).Freeman, C., The Role of Small Ftims in Innova-tion in the United Kingdom Since 194s, reportto the Bolton Committee of Inquiry on SmallFirms, research report No. 6 (London: Her Maj-esty’s Stationery Office, 1971).Freeman, G,, Blue Cross/Blue Shield Association,Chicago, 111., personal communication, May1981.Freiman, J. A., et al., “The Importance of Beta,the Type II Error and Sample Size in the Designand Interpretation of the Randomized ControlTrial,” N. fig. J. Med. 299:690, 1978.

References 2 1 1——

154.

155.

156.

157.

158.

159<

160

161.

162.

163.

164.

165.

166.

167.

General Accounting Office, U.S. Congress, FDADrug Approval: A Lengthy Process That Delaysthe Availability of Important New Drugs, reportby the Comptroller General of the U.S. to theSubcommittee on Science, Research, and Tech-nology, House Committee on Science and Tech-nology, U.S. Congress, May 28, 1980.General Accounting Office, U.S. Congress, 0PA4Should Promote Medical Necessity Programs forFederal Employees’ Health Insurance (Washing-ton, D. C.: U.S. Government Printing Office, July29, 1980).Gerstenfeld, A., “A Study of Successful Projects,Unsuccessful Projects, and Projects in Process inWest Germany, ” WEE Trans. Eng. Mgt.EM-23(3):116, 1976.Gifford, R. H., and Feinstein, A. R., “A Critiqueof Methodology in Studies of AnticoagulantTherapy for Acute Myocardial Infarction, ” N.Eng. J. Med. 280:351, 1969.Gilbert, J. P., et al., “Assessing Social Innova-tions: An Empirical Base for Policy, in Evalua-tion and fiperiment: Some Critical Issues in As-sessing Social Programs, C. A. Bennett andA. A. Lumsdaine (eds. ) (New York: AcademicPress, 1975).Gilbert, J. P., et al., “Progress in Surgery andAnesthesia: Benefits and Risks of InnovativeTherapy, “ in Costs, Risks, and Benefits of Sur-gery, J. P. Bunker, et al. (eds. ) (New York: OX-ford University Press, 1977).Gilbert, J. P., et al., “Statistics and Ethics in Sur-gery and Anesthesia,” science 198:684, 1977.Gillings, D., et al., “Analysis of Interrupted TimeSeries Mortality Trends: An Example To EvaluateRegionalized Perinatal Care, ” Am. -1 PublicHealth 71:38, 1981.Ginsburg, P., “Resource Allocation in the Hospi-tal Industry: The Role of Capital Financing,” Soc.%?C. Bull. 35:20, 1972.Glaser, E. M., “Using Behavioral Science Strat-egies for Defining the State of the Art, ” -1 Ap-plied Behav. Sci. 16:79, 1980.Glass, G. V., “Integrating Findings: The Meta-Analysis of Research, “ in Review of Research inEducation, vol. 5, L. S. Shulman (cd.) (Itasca, Ill.:Peacock, 1977).Glass, G. V., “Primary Secondary, and Meta-Analysis of Research, ” Educ. Res. 5:3, 1976.G1ass, G. V., and Smith, M. L., “Meta-Analysisof Research on Class Size and Achievement, ”Educ. Evaluat. Policy Analysis 1:2, 1979.Glass, G. V., et al., The Design and Analysis of

168.

169.

170.

171.

172,

173.

174.

175,

176.

177.

178.

179,

180.

181.

182.

Time-Series Experiments (Boulder, Colo.: Colo-rado Associated University Press, 1975).Goldschmidt, P . , Veterans Administration,Washington, D. C., personal communication,January 1982.Gordis, L., et al., “Pitfalls in Evaluating the Im-pact of Coronary Care Units on Mortality FromMyocardial Infarctions,” Johns Hopkins Med. J.141:287, 1977.Gore, S. M., “Assessing Clinical Trials—FirstSteps,” Br. Med. J. 282:1605, 1981.Gore, S. M., “Assessing Clinical Trials—TrialSize,” Br. Med. J. 282:1687, 1981.Gore, S. M., et al., “Misuse of StatisticalMethods: Critical Assessment of Articles in BMJFrom January to March 1976,” Br. Med. J. 1:85,1977.Gottman, J. M., “N-of-One and N-of-Two Re-search in Psychotherapy, ” Psycho]. Bull. 80:93,1973.Grabowski, H. G., et al., “The Effects of Regula-tory Policy on the Incentives To Innovate: AnInternational Comparative Analysis, ” in Impactof Public Policy on Drug Innovation and Pric-ing, S. A. Mitchell and E. A. Links (eds. )(Washington, D. C.: The American University,1976).Grace, N. D., et al., “The Present Status ofShunts for Protal Hypertension Incirrhosis, ”Gastroent. 50:684, 1966.The Gray Sheet “Restricted Device Case-by-CaseAlternative Unanimously Urged by Associa-tions,” 7(4): I&W8, Jan. 26, 1981.The Gray Sheet, “Mandatory Experience Report-ing Proposal Unanimously Criticized by Manu-facturers,” 7(6), Feb. 9, 1981.Greer, A. L., “Advances in the Study of Diffu-sion of Innovation in Health Care Organiza-tions,” Milbank Mere. Fund Q. 55:5o5, 1977.Hale, W. E., et al., “Drug Use in a GeriatricPopulation,”J. Am. Geriat. SoC. 23(S) :374, 1979.Hammermeister, K. E., et al., “Effect of Aorto-coronary Saphenous Vein Bypass Grafting onDeath and Sudden Deatl-i, ” Am. J. CardioJ.39:925, 1977.Hammermeister, K. E., et al., “Evidence From aNonrandomized Study That Coronary SurgeryProlongs Survival in Patients With Two-VesselCoronary Disease, ” Circulation 59:430, 1979.Harris, J. E., IWacro-fiperiments Versus Micro-Ekperiments for Health Policy (No. 286A) (Cam-bridge, Mass.: Massachusetts Institute of Tech-nology Press, 1981).


183

184,

185,

186.

187.

188.

189.

190.

191.

192.

193.

194.

195.

196.

197.

198.

199.

Hayes, R. H., and Abernathy, W. J., “Manag-ing Our Way to Economic Decline, ” HarvardBus. Rev. 58(4):68, 1980.Hecht, A., “Drug Effects on the Eyes,” FDA Con-sumer, November 1978, p. 14.Hecht, A., “Tracking the Skin’s Reactions toDrugs,” FDA Consumer, April 1981, p. 5.Hedges, L. V., and Olkin, I., “Vote-CountingMethods in Research Synthesis,” Psychol. Bull.88:359, 1980.Hill, A. B., Controlled CZinicaZ Trials (Spring-field, Ill.: Charles C, Thomas, 1960).Hill, J. D., et al., “A Randomised Trial of Home-Versus-Hospital Management for Patients WithSuspected Myocardial Infarction, ” Lancet 1:837,1978.Hobbins, J. C., et al., “The Fetal Monitoring De-bate,” Pediatrics 63:942, 1979.Hoehler, F, K., et al., “Spinal Manipulation forLow Back Pain,” J. A.M.A. 245:1835, 1981.Horwitz, R. I., and Feinstein, A. R., “AlternativeAnalytic Methods for Case-Control Studies of Es-trogens and Endometrial Cancer, ” N. Eng. JMed. 299(20):1089, 16, 1978.)Horwitz, R. I., et al., “Necropsy Diagnosis of En-dometrial Cancer and Detection—Bias in Case/Control Studies,” Lancet 2:66, 1981.Hubbard, W. N., Jr., President, Upjohn Co., tes-timony at the hearings on The Depo-provera D-bate before the Select Committee on Population,U.S. House of Representatives, Aug. 8, 1978(Washington, D. C.: U.S. Government Printingoffice, 1978).Hulka, B. S., et al., “Physician Management inPrimary Care,” Am. J. Public Health 66:1173,1966.Hummel-Rossi, B., and Weinberg, S. L., “Prac-tical Guidelines in Applying Current Theories tothe Measurement of Change,” JSAS Catalog ofSHected Documents in Psychology (Ms. No. 916)5:226, 1975.Hutchinson, G. B., and Rothman, K. J., “Cor-recting a Bias?” N. Eng. J. Med. 299:1129, 1978.Institute of Medicine, National Academy ofSciences, Conference Summary: Evaluating Med-ical Technologies in Clinical Use (Washington,D. C.: NAS, 1981).Institute of Medicine, National Academy of Sci-ences, Reliability of Hospital Discharge Abstracts(Washington, D. C.: NAS, 1980).Institute of Medicine, National Academy of Sci-ences, Reliability of Medicare Hospital Discharge

200,

201.

202.

203.

204.

205.

206

207,

208.

209.

210.

211.

212.

213.

214.

215.

216.

Records (Washington, D. C.: NAS, November1977).Jick, H., et al., “Vaginal Spermicides and Con-genital Disorders, ’’J.A. M.A. 245(13):1329, 1981.]oint Commission on Prescription Drug Use, Inc.,Report of the Joint Commission on PrescriptionDrug Use: Appendix IL Minutes of Meetingsr

minutes of June 20-21, 1978 (Rockville, Md.: Jan.23, 1980).Jones, R, R., et al., “Time-Series Analysis inOperant Research, ” J AppZied Behav. Analysis10:151, 1977.Jones, S. E., “Adjuvant Chemotherapy for BreastCancerJ” J. A.M.A. 245:1527, 1981.Jensen, A. R,, “The Totally Implantable ArtificialHeart,” Hast. Cent. Rpt. 3(5):1, 1973.Jensen, A. R., and Butler, L. H., “Public Ethicsand Policy Making, ” Hast. Cent. Z?pt. 5(4):19,1975.Joreskog, K. G., “A General Approach to Con-firmatory Maximum Likelihood Factor Analysis,”Psychometrika 34:183, 1969.Kaiser Foundation Health Plan, Oakland, Calif.,“Kaiser-Permanente Medical Care Program An-nual Report 1980, ” n.d.Katz, E., et al., “Traditions of Research on theDiffusion of Innovation,” Amer. Sociol. 28:237,1963.Kennedy, R. H., et al., “Necrologic ComputedTomography in a Defined Population Group,”Radiology 106:1, 1979.Kennedy, R. H., et al., “Trends in Cardiac Sur-gery: Five-Year Study of a Defined Population, ”Thoracic Cardiovasc. Surg. 80(5):702, 1980.Kennedy, R. H., et al., “Use of the Cardiac-Cath-eterization Laboratory in a Defined Population, ”N. Eng. J, Med. 303:1273, 1980.Kimberly, J. R., “Hospital Adoption of Innova-tion: The Role of Integration Into External Infor-mation Environments, ” J. Health Soc. Behav.19:361, 1978.Kimberly, J. R., “Managerial Innovation, ” inHandbook of Organizational Design, P. C.Nystrom and W. H. Starbuck (eds. ) (New York:Oxford University Press, 1981).Kohrrnan, A., Washington, D. C., personal com-munication, 1981.Kolata, G. B., “Dilemma in Cancer Treatment, ”%ience 209:792, 1980.Krol, R, A., “A Meta-Analysis of ComparativeResearch on the Effects of Desegregation on Aca-demic Achievement, ” unpublished doctoral dis-

References ● 213

217,

218,

219

220.

221.

222.

223.

224,

225.

226.

227,

228.

229.

230.

231,

232.

sertation, University of Michigan, Ann Arbor,Mich., 1979.Kuhn, T. S., The Structure of Scientific Revolu-tions (Chicago: University of Chicago Press,1970).Kunitz, S., et al., “Pilot Data Bank Networks forNeurological Disorders, ” in Third Annual Pro-ceedings of the Symposium of Medical Care(Long Beach, Calif.: IEEE Computer Society,1979).Langrish, J., et al., Wealth From Knowle~e(London: Macmillan, 1972).Lave, L. B., and Seskin, E. P., “Epidemiology,Causality and Public Policy, ” Am. Sci. 67:178,1979.Lebacqz, K., “Bioethics: Some Challenges Froma Libertarian Perspective, ” presented at the Con-ference on Faith, Science, and the Future, Cam-bridge, Mass,, July 1979.Levy, R. I., and Sondik, E. J., “Decision-Makingin Planning Large-Scale Comparative Studies, ”Ann. N. Y. Acad. Sci. 304:441, 1978.Levy, R. I., and Sondik, E. J., “The Managementof Biomedical Research: A Case in Point, ” in Bio-medical Innovation, E. B. Roberts, et al. (eds. )(Cambridge, Mass.: MIT Press, 1981).Levy, D., New York Hospital, personal commu-nication, April 1981.Levy, D., et al., “Prognosis in NontraumaticComa, ” Ann. Int. Med. 94(3):293, 1981.Light, R. J., and Smith, P. V., “AccumulatingEvidence: Procedures for Resolving Contradic-tions Among Research Studies, ” Harvard Educ.Rev. 41:429, 1976.Lohr, K. N., et al., Peer Review and TechnologyAssessment in Medicine, prepared for the Officeof Technology Assessment, U.S. Congress (SantaMonica, Calif.: Rand Corp., August 1981).Lowe, C. U., Office of Medical Applications ofResearch, National Institutes of Health, Bethesda,Md., personal communication, May 15, 1981.Lowe, C. U., “The Consensus Development Pro-gramme: Technology Assessment at the NationalInstitutes of Health, ” Ilr. Med. J. 28:1583, 1980.Luborsky, L., et al., “Comparative Studies ofPsychotherapies: Is It True That Everyone HasWon and All Must Have Prizes?” Arch. Gen.Psychiatry 32:995, 1975.Malkiel, B. G., “Productivity: The Problem Be-hind the Headlines, ” Harvard Bus. Rev. 57(3):81,1980.Malone, W. F., et al., “The Delphi Technique asan Instrument in Dental Curriculum Revision, ”L ~nt. Educ. 40:298, 1 9 7 6 .

233.

234.

235.

236.

237.

238.

239.

240.

241.

242,

243.

244.

245.

246.

247.

248.

Manning, P. R., and Denson, T. A., “How Car-diologists Learn About Echocardiography: A Re-minder for Medical Educators and Legislators, ”Ann. Int. Med. 91:469, 1979.Manning, P. R., and Denson, T. A., “How in-ternists Learned About Cimetidine, ” Ann. ht.Med. 92:690, 1980.Mansfield, E., et al., The Production and Appli-cation of New Industrial Technology (New York:Norton, 1977).Mather, H. G., et al., “Myocardial Infarction: AComparison Between Home and Hospital Carefor Patients, ” Br. Med. J. 1:925, 1976.McCleary, R., and Hay, R. A., Jr., Applied TimeSeries for the SociaZ Sciences (Beverl y Hills,Calif.: Sage Publications, 1980).McGinnis, W., Department of Defense, Tri-Serv-ice Medical Information Systems Program Office,Bethesda, Md., personal communication, Dec.10, 1981.McGuire, W. J., “The Nature of Attitudes andAttitude Change, “ in Handbook of Social Psy-chology, vol. 3., G. Lindzey and E. Aronson(eds.) (Reading, Mass.: Addison-Wesley, 1969).McNeer, J. F., et al., “The Nature of TreatmentSelection in Coronary Artery Disease: ExperienceWith Medical and Surgical Treatment of a Chron-ic Disease,” Circulation 49:606, 1974.McNeil, G. J., and Adelstein, S. J., “Determining

the Value of Diagnostic and Screening Tests, ” J.Nucl. Med. 17:439, 1976.Meier, P., University of Chicago, personal com-munication, Dec. 4, 1980.Meier, P., ‘The Biggest Public Health ExperimentEver: The 1954 Field Trial of the Salk PoliomyelitisVaccine, ” in Statistics: A Guide to the Unknown,J. M. Tanur, et al. (eds.) (San Francisco: Holden-Day, 1972).Merrill, R. A., “FDA Modes of Regulation andTheir Relationship to Private Sector Innovation,”prepared for the Office of Technology Assess-ment, U.S. Congress, Washington, D. C., 1980,Miao, L. L. “Gastric Freezing: An Example of theEvaluation of Medical Therapy by RandomizedClinical Trials,” in Costs, Risks, and Benefits ofSurgery, J. P. Bunker, et al. (eds. ) (New York:Oxford University Press, 1977).Milholland, A. V., et al., “Medical Assessmentby a Delphi Group Opinion Technique, ” N. Eng.J. Med. 288:1272, 1973,Miller, M. J., “FDA Voluntary Standards En-dorsement Policy: Victim of Legal Homicide, ”Med. Instrumentation 14:331, 1980.MitcheIl, A. A., et al., “Adverse Drug Effects and

98-144 0 - 82 - 15

—


249.

250.

251.

252.

253.

254.

255.

256.

257.

258.

259.

260.

261.

262.

263.

264.

Drug Surveillance, ” in Pediatric F%arma-cology—Therapeutic Principles in Practice, S. J.Yaffe (cd.) (New York: Grune & Stratton, 1980).Moloney, T. W., and Rogers, D. E., “MedicalTechnology: A Different View of the ContentiousDebate Over Costs,” N. Eng. J A4ed. 301(26):1413, 1979.Moscovice, I., et al., “Health Services Researchfor Decision-Makers: The Use of the Delphi Tech-nique To Determine Health Priorities, ” -1. HealthPolit. Policy Law 3:388, 1977.Mosenkis, R., Emergency Care Research Insti-tute, Philadelphia, Pa., personal communication,August 1981.Mosteller, F., “Innovation and Evaluation, ” Sci-ence 211(4485):881, 1981.Muller, J. E., et al., “Let’s Not Let the GenieEscape From the Bottle—Again, ” N. Ehg. J. Med.304:1294, 1981.Murphy, M. L., et al., “Special Correspondence:A Debate on Coronary Bypass, ” N. Eng. J. Med.297:1470, 1977.Murphy, M. L., et al., “Treatment of ChronicStable Angina: A Preliminary Report of SurvivalData of the Randomized Veterans AdministrationCooperative Study,” N. Eng. J. Med. 297:621,1977.Myers, S., and Marquis, D. G., Successfid Indus-tria] Innovations, NSF 69-71 (Washington, D. C.:National Science Foundation, 1969).National Science Foundation, Science Indicators1978 (Washington, D. C.: NSF, 1978).Nelson, N. M., et al., “A Randomized ClinicalTrial of the Leboyer Approach to Childbirth,”N. Eng. J. Med. 302:655, 1980.Neustadt, R. D,, and Fineberg, H. V., The SwineFlu Affair: Decision-Making on a Slippery Dis-ease, GPO stock No. 017-000-002104 (Washing-ton, D. C.: U.S. Government Printing Office,1978).“New Device Introductions on the Rise,” IXv. &Diag. Letter 7:3, Aug. 12, 1980.Nine, H., Food and Drug Administration, Bureauof Medical Devices, Rockville, Md., personalcommunication, January 1981.Nisbett, R. E., and Ross, L. D., Human lhference:Strategies and Shortcomings of Social Ju~ment(Englewood Cliffs, N. J.: Prentice-Hall, 1980).Nobrega, F. T., et al., Mayo Clinic, Rochester,Minn., “Hospital Use in a Circumscribed Com-munity, ” unpublished paper, 1980.The Norwegian Multicenter Study Group,“Timolol-Induced Reduction in Mortality and Re-

265.

266.

267<

268.

269.

270.

271.

272.

273

274.

infarction in Patients Surviving Acute Myocar-dial Infarction,” N. Eng. J, Med. 304:801, 1981.Office of Technology Assessment, U.S. Congress,Annual Report to the Congress for 1977 (Wash-ington, D. C.: U.S. Government Printing Office,1978).Office of Technology Assessment, U.S. Congress,Assessing the Efficacy and Safety of MedicalTechnologies, GPO stock No. 052-003-00593-0(Washington, D. C.: U.S. Government PrintingOffice, September 1978).Office of Technology Assessment, U.S. Congress,Assessment of Technologies for fiterminingCancer Risks From the Environment, GPO stockNo. 052-003-00832-7 (Washington, D. C.: U.S.Goverment Printing Office, June 1981).Office of Technology Assessment, U.S. Congress,Computer Technology in Medical Education andAssessment, GPO stock No. 052-003-00704-5,(Washington, D. C.: U.S. Government PrintingOffice, September 1979).Office of Technology Assessment, U.S. Congress,fivelopment of Medical Technology: @portu-nities for Assessment, GPO stock No. 052-003-00217-5 (Washington, D. C.: U.S. GovernmentPrinting Office, August 1976).Office of Technology Assessment, U.S. Congress,The Implications of Cost-Effectiveness Analysisof Medical Technology, GPO stock No. 052-003-00765-7 (Washington, D. C.: U.S. GovernmentPrinting Office, August 1980).Office of Technology Assessment, U.S. Congress,The implications of Cost-Effectiveness Analysisof Medical Technology/Background Paper #l:Methodological Issues and Literature Review,GPO stock No. 052-003-00780-1 (Washington,D. C.: U.S. Government Printing Office, Septem-ber 1980).Office of Technology Assessment, U.S. Congress,The Implications of Cost-Effectiveness Analysisof Medical Technology/Background Paper #2:Case Studies of Medical Technologies (Washing-ton, D. C.: U.S. Government Printing Office,1981).Office of Technology Assessment, U.S. Congress,The Implications of Cost-Effectiveness Analysisof Medical Technology/Background Paper #3:The Efficacy and Cost Effectiveness of Psycho-therapy, GPO stock No, 052-003-00780-5 (Wash-ington, D. C.: U.S. Government Printing Office,October 1980).Office of Technology Assessment, U.S. Congress,The Implications of Cost-Effectiveness Analvsis.

References “ 215—-—— — - — — — —— . — - — . — . — — — — — — — — . . — — - . — — —

275.

276.

277.

278.

279,

280.

281.

282,

283.

284,

285,

286,

287,

of Medical Technology/Background Paper #4:The Management of Health Care Technology inTen Countries, GPO stock No. 052-003-00780-3(Washington, D. C.: [J.S. Government PrintingOffice, October 1980).Office of Technology Assessment, U.S. Congress,Medical Technology Under Proposals To IncreaseCompetition in Heahh Care, in press, 1982.Office of Technology Assessment U.S. Congress,MEDLARS and Health Information Policy, inpress, 1982.Office of Technology Assessment, U.S. Congress,“The National Center for Health Care Technol-ogy and Medical Technology Assessment Policy, ”staff paper, Mar. 16, 1981.Office of Technology Assessment, U.S. Congress,“The National Library of Medicine, ” staff paper,Mar. 17, 1981.Office of Technology Assessment, U.S. Congress,Policy Implications of the Computed Tomogra-phy (CT) Scanner, GPO stock No. 052-003-00565-4, (Washington, D. C.: U.S. GovernmentPrinting Office, August 1978).Office of Technology Assessment, U.S. Congress,Policy Implications of the Computed Tomogra-phy (CT) Scanner: An Update, GPO stock No.052-003-00793-2, (Washington, D. C.: U.S. Gov-ernment Printing Office, January 1981).Office of Technology Assessent, U.S. Congress,Postmarketing Surveillance of PrescriptionDrugs, in press, 1982.Office of Technology Assessment, U.S. Congress,Selected Topics in Federal Health Statistics, GPOstock No. 052-003 -00683-9 (Washington, D. C.:U.S. Government Printing Office, June 1979).Office of Technology Assessment, U.S. Congress,Technological Innovation and Health, Safety,and Environmental Regulations, in press, 1982.Okie, S., “Hard To Put Price on Heart Surgery’sValue,” The Washington Post, Apr. 27, 1981,A13.Palmour, R. M., and Ervin, F. R., “Biochemicaland Behavioral Studies of a Peptide Derived Fromthe Dialysate of Psychiatric Patients, ” presentedat the 24th Annual Meeting of the American Soci-ety for Artificial Internal Organs, April 1978.Patton, M. Q., Qualitative Evaluation Methods(Beverly Hills, Calif.: Sage Publications, 1979).Perry, S., and Kalberer, J. T., Jr., “The NIH Con-sensus-Development Program and the Assess-ment of Health Technologies: The First TwoYears,” N. Eng. J. Med. 303:169, 1980.

288.

289.

290.

291.

292

293,

294.

295.

296.

297.

298.

299.

300.

301.

302.

Peterson, R. D., and MacPhee, C. R., EconomicOrganization in Medical Equipment and Supply(Lexington, Mass.: D. C. Heath & Co., 1973).Pete, R., et al., “Design and Analysis of Random-ized Clinical Trials Requiring Prolonged Obser-vation of Each Patient: 1. Introduction and De-sign,” Br. J. Cancer 34:585, 1976.Pickup, J. C., and Keen, H., “Continuous Sub-cutaneous Insulin Infusion: A Developing Toolin Diabetes Research, ” Diabetologia 18:1, 1980.Pickup, J. C., et al., “Continuous SubcutaneousInsulin Infusion: Improved Blood-Glucose and In-termediary-Metabolite Control in Diabetics, ”Lancet 1:1255, 1979.Pillemer, D. B., and Light, R. J., “synthesizingOutcomes: How To Use Research Evidence FromMany Studies, ” Harvard Educ. Rev. 50:176,1980.Policy Research, Inc., Baltimore, Md., MedicalPractice Information Demonstration Project,final report to the Office of the Assistant Secre-tary for Health, Department of Health, Educa-tion, and WeIfare, contract No. 282-77-0068 GS,December 1979.Posavac, E. J., “Evaluations of Patient EducationPrograms: A Meta-Analysis, ” Eva]. Health Pro-fess. 3:47, 1980.Powledge, T. M., and Fletcher, J., “Guidelinesfor the Ethical, Social and Legal Issues in PrenatalDiagnosis,” N. Eng. J. Med. 300:168, 1979.President’s Commission for the Study of EthicalProblems in Medicine and Biomedical and Behav-ioral Research, Dfining Death: Medical, Legaland Ethical Issues in the Determination of Bath(Washington, D. C.: U.S. Government PrintingOffice, July 1981).Preston, T. A., Coronary Artery Surgery: A Crit-ical Review (New York: Raven, 1977),Quinn, J. G., “Why Executives Think Short, ”Newsweek, July 13, 1981.Radzius, J. R., “FDA Regulation of Drugs,Devices, ” AORN J. 24:254, 1976.Ramroth, D. N. (cd.), Proceedings From theMedical and Scientific Section Meeting on Get-ting New Products to the Market (Washington,D. C.: Health Industry Manufacturers Associa-tion, 1979).Rapoport, J., “Diffusion of Technological inno-vation Among Non-Profit Firms: A Case Study

of Radioisotopes in U.S. Hospitals, ” J, Econ. BUS.301:108, 1 9 7 8 .Registry for Research on Hormonal Transplacen-


303.

304.

305.

306.

307.

308.

309.

310.

311.

312.

313.

314.

315.

316.

317.

318.

tal Carcinogenesis, 1978 Newsletter, Chicago,n.d.Registry for Research on Hormonal Transplacen-tal Carcinogenesis, 1979-80 Newsletter, Chicago,n.d.Reich, W. T. (cd.), Encyclopedia of Bioethics(New York: Free Press, 1978).Reiss, J. B., “Issues in the Cost and Regulationof New Medical Technologies and Procedures:Heart Transplants as a Case Study,” presentedto the Harvard-MIT Conference on MedicalTechnology, Boston, Apr. 13-16, 1980.Reiss, J. B., “Technology Assessment and Pay-ing for Medical Services Provided Patients, ” pre-pared for the Office of Technology Assessment,U.S. Congress, Washington, D. C., April 1981.Relman, A. S., “Assessment of Medical Prac-tices,” IV. Eng. J. Med. 303(3):153, 1980.Rennie, R. A., “Consensus Statements, ” N. Ehg.J Med. 303:153, 1980.Research News, The Diabetes Research andTraining Center: New Assault on an Old Enemy(Ann Arbor: University of Michigan, May,1979).Rettig, R. A., “The Federal Government andMedical Technology: Crossing Policy and Man-agement Thresholds, ” PoZicy Sci. 11:343, 1980.Riecken, H. W., and Boruch, R. F. (eds. ), Socialfiperimentation: A Method for Planning andEvacuating Social Intervention (New York:Academic Press, 1975).Rindskopf, D., “Using Structural Equation Mod-els To Analyze Nonexperimental Data, ” in Re-analyzing Program Evaluations: Policies andPractices for Secondary halysis of SociaZ andEducational Programs, R. F. Boruch, et al. (eds. )(San Francisco: Jossey-Bass, in press).Roberts, E. B., “Influences on Innovation: Extrap-olations to Biomedical Technology, ” in Biomedi-cal Innovation, E. B. Roberts, et al. (eds. ) (Cam-bridge, Mass.: MIT Press, 1981).Robinson, J. A., “Coronary Care Unit VersusHospital Mortality in Acute Myocardial Infarc-tion,” Israel J. Med. Sci. 5:772, 1969.Rodak, J., National Center for Health Care Tech-nology, Rockville, Md., personal communica-tion, December 1980.Roe, B. B., “The UCR Boondoggle: A DeathKnell for Private Practice?” N. Eng. J. Med.305(1):41, 1981.Rogers, E. M., and Shoemaker, F. F., Communi-cation of Innovations: A Cross-Cultural Ap-proach, 2d ed. (New York: Free Press, 1971).Romm, F. J., and Hulka, B. S., “Developing Cri-teria for Quality of Care Assessment: Effect of

319.

320.

321.

322.

323.

324.

325.

326.

327.

328.

329<

330.

331,

332s

333.

334.

335.

336,

337.

the Delphi Technique, “ HeaZth Serv. Res. 14:309,1979.Roos, L. L., Jr., et al., “Assessing the Impact ofTonsillectomies, ” Med. Care 16(6):502, 1978.Roos, L. L., Jr., et al., “Using AdministrativeData Banks for Research and Evaluation: A CaseStudy,” EvaJuat. Q. 3(2):236, 1979,Roosf N. P., “Who Should Do the Surgery? Ton-sillectomy-Adenoidectomy in One CanadianProvince,” Inquiry 16(1):73, 1979.Roos, N. P., et al., “A New Audit Procedure Ap-plied to an Old Question: Is the Frequency ofT&A Justified?” Med. Care 15(1):1, 1977.Rosati, R., et al., Expansion, Ektension, and EvaLuation of a C~nical Databank for Coronary Ar-tery Disease, contract No. HRA 230-76-0300,submitted to the National Center for Health Serv-ices Research, Hyattsville, Md., August 1980.Rosenthal, R., “Combining Results of Independ-ent Studies,” Psychol. Bull. 85:185, 1978.Rosenthal, R., fiperimenter Effects in BehavioralResearch (New York: Irvington, 1976).Ross, R. S., “Ischemic Heart Disease: An Over-view,” Am. J. Cardiol. 38:496, 1975.Rothwell, R., and Zegveld W., “The Role ofSmall Manufacturing Enterprises in Innovation, ”P. Innov. 2(1):3, 1979.Rubenstein, L., et al., “Quality of Care Assess-ment by Process and Outcome Scoring, ” Ann. Ln-tern. Med. 86:617, 1977.Ruffin, J. M., et al., “A Co-Operative Double-Blind Evaluation of Gastric Freezing in the Treat-ment of Duodenal Ulcer, ” N. Ehg. J. Med. 281:16,1969.Runyon, R. P., and Haber, A., Fundamentals ofBehaviora~ Statistics, 3d ed. (Reading, Mass.:Addison-Wesley, 1976).Russell, L. B., Technology in Hospitals: MedicalAdvances and Their Diffusion (Washington,D. C.: The Brookings Institution, 1979).Ryder, H. W., letter to the editor, N. Ehg. J. Med.296:45, 1977.Salkever, D., and Bice, T., Hospital Certificate-of-Need Controls: Impact on Investment, Costs,and Use (Washington, D. C.: American EnterpriseInstitute for Public Policy Research, 1979).Salkever, D., and Bice, T., “The Impact ofCertificate-of-Need Controls on Hospital Invest-ment,” Milbank Mere. Fund Q. 54:185, 1976.Saxe, L., and Fine, M., Social Experiments:Methods for Design and Evaluation (BeverlyHills, Calif.: Sage Publications, 1981).Schoenbaum, S. C., et al., “The Swine-InfluenzaDecision, ” N. Eng. J. Med. 295:759, 1976.Schor, S., and Karten, I., “Statistical Evaluation

References ● 217

338.

339.

340.

341.

342.

343.

344.

345.

346.

347.

348.

349<

350.

351.

352.

of Medical Journal Manuscripts, ” ~. A. M A.195:145, 1966.Schulz, S. C., et al., “Dialysis in Schizophrenia:A Double-Blind Evaluation, ” Science 211:1066,1981.Scriven, M., “The Methodology of Evaluation, ”in Perspectives of Curriculum Evaluation, R. W.Tyler, et al. (eds. ) (Chicago: Rand McNally,1967).Sechrest, L., et al,, “Introduction-Some Ne-glected Problems in Evaluation Research:Strength and Integrity of Treatments” in Evalua-tion Studies Review Annual, vol. 4, L. Sechrest,et a]. (eds. ) (Beverly Hills, Calif.: Sage Publica-tions, 1979).Shapiro, S., “Evidence on Screening for BreastCancer From a Randomized Trial, ” cancer39:2772, 1977.Sherman, M. A., “Mobile Intensive Care Units:An Evaluation of Effectiveness, ” J, A.M.A.241:1899, 1979.Sherman, M. A., et al., “Threats to the Validityof Emergency Medical Services Evaluation, ”Med. Care 17:127, 1979.Shields, S., Hospital Utilization Plan, Pittsburgh,Pa., personal communication, May 8, 1981.Shrader-Frechette, A., “Technology Assessmentas Applied Philosophy of Science, ” Sci. TechnoZ.Human Values 6:33, 1980.Silverman, W. A., “The Lesson of RetrolentalFibroplasia, ” Sci. Am. 236(6):100, 1977.Silvers, J. B., “Competitive Economics of HealthCare Providers in the 1980’s: The Case of theLarge National Clinics, ” presented at the ProjectHOPE conference, “The Role of Major MedicalClinics in Meeting Future National HealthNeeds,” Millwood, Va., Mar. 11-13, 1981.Singer, D., et al., “Unexpected Readmission tothe Coronary Care Unit During Recovery FromAcute Myocardial Infarction, ” N. Eng. J Med.,304(11):625, 1981.Singer, I. D., Washington, D. C., “Notes onFederal Efforts in Ethics and Health Policy, ” per-sonal communication, 1981.Siperstein, M. D., et al., “Control of BloodGlucose and Diabetic Vascular Disease, ” N. Eng.J Med. 296:1060, 1977.Skolnick, M., “The Utah Genealogical Data Base:A Resource for Genetic Epidemiology, ” in Ban-bury Report 4: Cancer Incidence in LHinedPop-ulations (Cold Spring Harbor, N. Y.: Cold SpringHarbor Laboratory, 1980).Skolnick, M., et al., “A Computerized FamilyHistory Data Base System, ” Sociol. & Soc. Res.,63(3):507, 1979.

353,

354,

355

356,

357,

358,

359.

360.

361.

362.

363.

364.

365.

366.

367,

368.

369,

Smith, M. L,, and Glass, G. V., “Meta-Analysisof Psychotherapy Outcome Studies, ” Am.psycho]. 32:752, 1977.Smith, M. L., et al., The Benefits of Psychother-apy (Baltimore: Johns Hopkins University Press,1981).Smithson, L., Structure of the U.S. MedicalSupply, Z@ipment, andlkices Industries (Stan-ford, Calif.: SRI International, 1979).Sonnenfeld, J. A., “The Blue Cross and BlueShield Associations’ Medical Necessity Program, ”VoZunt. Effort Q. 3(2):5, 1981.Speer, D. C., and Tapp, J. C., “Evaluation ofMental Heahh Service Effectiveness: A “Start Up”Model for Established Programs, ” Am. ). Ortho-psych. 46:217, 1976.Spodick, D. H., “The Surgical Mystique and theDouble Standard: Controlled Trials of Medicaland Surgical Therapy for Cardiac Disease, ” Am.Heart J 85:579, 1973.Stein, D. J., “Manufacturers of Medical DevicesJoin the Chorus of Regulatory Critics, ” NationalJ 12(38):1556, 1 9 8 0 .Stinson, E. R., and Mueller, D. A., “Survey ofHealth Professionals’ Information Habits andNeeds: Conducted Through Personal Interviews, ”J. A.M.A. 243(2):140, 1980.Tamborlane, W. V., et al., “Restoration of Nor-mal Lipid and Amino Acid Metabolism in Dia-betic Patients Treated With a Portable Insulin-Infusion Pump, ” Lancet 1:1258, 1979.Tattersall, R. B., “Home Blood Glucose Monitor-ing,” DiabetoZogia 16:71, 1979.Temple, R., and Pledger, G. W., “The FDA’sCritique of the Anturane Reinfarction Trial, ” N.Eng. J Med. 303:1488, 1980.Thacker, S., Center for Disease Control, Atlan-ta, Ga., personal communication, April 1981.Thibault, G. E., et al., “Medical Intensive Care:Indications, Interventions, and Outcomes,” N.Ertg. J Med., 302(17):938, 1980.Thomas, L., Lives of a CeZl (New York: BantonBooks, Inc., 1975).Thompson, M. S., and Cohen, A. B., “DecisionAnalysis: Electronic Fetal Monitoring,” in A4eth-ods of Evaluating Health Services, P. M. Wort-man (cd. ) (Beverly Hills, Calif.: Sage Publica-tions, 1981).Thornell, C. A., Comparison of Strategies for theDevelopment of Process Measures in EmergencyMedical Services, NCHSR Research SummarySeries (Hyattsville, Md.: National Center forHealth Services Research, 1981).Tilson, D., et al., “The Federal Interest, ” in TheDiffusion of Medical Technology, G. Gordon and


370<

371.

372.

373.

374.

375.

376.

377.

378.

379.

380.

381.

382.

383.

384.

385

386

G. L. Fisher (eds. ) (Cambridge, Mass.: Ballinger,1975).Toffler, A., The Third Wave (New York: WilliamMorrow & Co., 1980).Tosteson, D. C., “Lithium and Mania, ” %i. Am.244:164, 1981.Towery, O. B., and Perry, S., “The ScientificBasis for Coverage Decisions by Third-Party Pay-ers, J. A.M.A. 245(1):59, 1981.Tribe, H., “Technology Assessment and theFourth Discontinuity: The Limits of Instrumen-tal Rationality, ” S. CaZil Law Rev. 46:617, 1973.Tversky, A., and Kahneman, D., “JudgmentUnder Uncertainty: Heuristics and Biases,”science 185:620, 1974.Utterback, J. M., “Innovation in Industry and theDiffusion of Technology,” Science 183:620, 1974.Utterback, J. M., and Abernathy, W. J., “A Dy-namic Model of Process and Product Innova-tion, ” bega 3(6):639, 1975.Veatch, R. M., Bethesda, Md., “Consensus De-velopment in Medical Technology: Problems forPlanning a Consensus Conference on CoronaryArtery Bypass Surgery, ” unpublished, 1980.Veatch, R. M., and Branson, R. (eds.), Ethics andHealth Policy (Cambridge, Mass.: Ballinger,1976).Wagemaker, H., Jr., and Cade, R., “The Use ofHemodialysis in Chronic Schizophrenia, ” Am. J.Psychiatry 134:684, 1977.Wagner, J. L., “Toward a Research Agenda onMedical Technology,” in Medical Technology,publication No. (PHS) 19-3254 (Washington,D. C.: U.S. Government Printing Office, 1979).Wangensteen, O. H., et al., “Achieving ‘Physio-logical Gastrectomy’ by Gastric Freezing, ”J, A.M.A. 180:95, 1962.Warner, K. E., “A ‘Desperation-Reaction’ Modelof Medical Diffusion, ” Health Serv. Res. 10:369,1975.Warner, K. E., and Luce, B. R., Cost-Benefit andCost-Effectiveness Analysis in Health Care: Prin-ciples, Practice, and Potential (Ann Arbor,Mich.: Health Administration Press, Universityof Michigan, in press).Warren, K. S., (cd.), Coping With BiomedicalLiterature (New York: Praeger, 1981).Watkins, R,, Group Health Cooperative, Seat-tle, Wash., personal communication, May 6,1981.Weinstein, M. C., and Fineberg, H. V., ClinicalDecision Analysis (Philadelphia: W. B. Saunders,1980).

387.

388.

389

390!

391.

392.

393.

394.

395.

396.

397,

398.

399.

400.

401.

402.

Weiss, D. H., Evacuation Research: Methods forAssessing Program Effectiveness (EnglewoodCliffs, N. J.: Prentice-Hall, 1972).Wennberg, J. E., “Commentary: Physician Un-certainty, Specialty, Idealogy, and a SecondOpinion to Tonsillectomy,” Pediatrics 59:952,1977.Wennberg, J. E., et al., “The Need for Assessingthe Outcome of Common Medical Practices, ”Ann. Rev. Public Health 1:277, 1980.Wennberg, J. E., et al., “Some Uses of ClaimsData for the Analysis of Surgical Practicesf ” inProceedings of the 18th National Public HealthConference on Records and Statistics, DHHSpublication No. (PHS) 81-1214, August 1980.Whittaker, G. F., “The Evaluation of a MergedHospital System: Application of the InterruptedTime-Series Design,” Evaluat. Rev. 5:68, 1981.Willems, J., “The Relationship Between the Dif-fusion of Medical Technology and the Organiza-tion and Economics of Health Care Delivery, inMedicaZ Technology, J. Wagner (cd.), DHEWpublication No. (PHS) 79-3254, 1979.Williams, M. E., University of Illinois, Cham-paign-Urbana, 111., personal communication,March 1981.Williams, M. E., et al. (cd.), Computer ReadableData Bases: A Compendium and Source Book(Washington, D. C.: American Society for Infor-mation Science, 1979).Williams, M. E., “Database and Online Statisticsfor 1979,” ASIS Bulletin 7(2):27, 1980.Williamson, J. W., Johns Hopkins University,“Reformation of Project Goals, ” unpublishedpaper, 1981.Wolf, C., Jr., Ethics and Policy Analysis (SantaMonica, Calif.: Rand Corp., February 1980).Wolf, S., “High Time for Synthesis, ” Forum onMedicine 3:502, 1980.Worthen, D. B., “Prescribing Influences: AnOverview,” Br. J. Med. Educ. 7:109, 1973.Wortman, P. M., “Consensus Development, ” inMethods for Evaluating Health Services, P. M.Wortman (cd.) (Beverly Hills, Calif.: Sage Pub-lications, 1981).Wortman, P. M., “Randomized Clinical Trials, ”in Method for Evaluating Health Services, P. M.Wortman (cd. ) (Beverly Hills, Calif.: Sage Pub-lications , 1981).Wortman, P. M., et al., “The First Year of theEducation Voucher Demonstration: A SecondaryAnalysis of Student Achievement Test Scores, ”Eval. Q. 2:193, 1978.

403.

404.

405.

Wylie, J. H., et al., “Effect of Cimetidine on Sur- 406.gery for Duodenal Ulcer, ” Lancet 1:1307, 1981.Young, D. A., “Communications Linking ClinicalResearch and Clinical Practice, ” in Biornedica~ ~- 407.novation, E. B. Roberts, et al. (eds. ) (Cambridge,Mass.: MIT Press, 1981).Zaltman, G., et al., Innovations and Organiza-tions (New York: John Wiley & Sons, 1973).

References . 219-- ——.— — .——

Zimmerman, H., Rhode Island Health ServicesResearch, Inc., Providence, R. I., personal com-munication, Apr. 17, 1981.Zwick, D., Bethesda, Md., personal communica-tion, December 1981.

Indexabortion, 48ananencephaly, 48arthroscopy, 82artificial heart, 48-49, 75assessment

clinical trials in, 130-137compendium of bibliographic data bases for (table),

121-126consensus development, 64, 100, 144-145cost-benefit analysis in, 11, 39cost-effectiveness analysis in, 11, 14, 39, 40, 41, 46,

60-62, 67critique of current system for, 91-102data sources for, 24-27dissemination of information on, 101-102of economic effects, 24encouragement of private sector leadership in, 108experimental studies in, 34-35group decision methods in, 62-66, 93of health effects, 23, 33-39identification of medical technologies needing, 94-97information for, 23-30interpretation of studies, 128-137integrat( d system of, 105literature review in, 57-58methodological considerations in, 127-149objective of, 91of performance standards, 23-24Process phases, 94-102proposed organization for, 106-107relationship to innovation, 146-149of social effects, 24social values in, 45-53statistical data sources for (table), 113-121synthesis activities in, 57-60, 100-101, 137-146testing phase, 98-100uncontrolled study designs in, 136-137validity of research, 128-129

Bibliographic Retrieval Service, 28BIOSIS, 28Blue Cross/Blue Shield, 7, 26, 81, 92

Medical Necessity Project, 78, 98Blue Cross of Massachusetts, 78Blue Shield of California, 78, 82

Ambulatory Surgery Project, 78Medical Policy Committee, 78

Brook, Robert, 192Bunker, John, 192

caesarian delivery, 52, 129California Podiatry Association, 78cancer

chemotherapy, 75endometrial, 38Halsted radical surgery, 129screening, 75

case-control study, 37-38case studies, 10-11, 128Centers for Disease Control (CDC), 13, 51, 85Central Intelligence Agency (CIA), 13, 51chemotherapy, 37, 75clearinghouses, 29-30Cleveland Clinic, 77clinical trials, 131-137

cohort design in, 35-36, 132historical controls, 134-135matching in, 132-133retrospective case-control studies, 133-134time-series design, 135-136

cobalt therapy, 83Commission on Professional and Hospital Activities

(CPHA), 26-27Community Services Administration, 29computed tomography (see CT scanning)Congress

acts of (see legislation)House Committee on Energy and Commerce, 1?3options for (see Policy options), 18Senate Committee on Labor and Human Resources, 193

consensus development, 63construct validity, 129, 139coronary artery bypass graft surgery, 52, 129, 131coronary care units (CCUS), 37Coronary Drug Project, 1-30cost-effectiveness analysis (CEA), 61costs vs. prices, 40-41CT scanning, 40, 46, 78, 84

data bases, 25, 78Basic Vital Statistics, 113Bioethicdine (table), 121BIOSIS Previews (table), 121Blue Cross/Blue Shield Systems (table), 117Boston Collaborative Drug Surveillance Program (table), 119Boston University Medical Center Drug Epidemiology Unit

(table), 119Bureau of Labor Statistics Annual Occupational Injuries

and Illness Survey (table), 117Cambridge Scientific Abstracts (table), 122, 125CANCERLIT (table), 121CANCERPROJ (table), 121Centers for Disease Control Birth Defects Monitoring

System (table), 114Centers for Disease Control Hepatic Angiosarcoma

Surveillance Finding Effort (table), 114Centers for Disease Control National Disease Surveillance

Program (table), 113Centers for Disease Control Smokey Nuclear Test Cohort

Study (table), 114Chemical Abstracts Service (table), 121Child Abuse and Neglect (table), 122CLINPROT (table), 122compendium of bibliographic (table), 121-126

223


Comprehensive Dissertation Index (table), 122Conference Papers Index (table), 122Department of Commerce Census of the Population

(table), 117Department of Defense Civilian Health and Medical

Program of the Uniformed Services (table), 117Department of Defense TRI-Service Medical Information

Systems (table), 117DES Vaginal Cancer Registry (table), 118DRUGINFO and Alcohol Use/Abuse (table), 122Duke University Cardiovascular Data Bank (table), 118Dunedin Program (table), 119ENVIRONLINE (table), 122EPILEPSYLINE (table), 122Excerpta Medica EMBASE (table), 122Foods ADLIBRA (table), 122Food and Drug Administration Adverse Drug Reaction

Spontaneous Recording System (table), 114Food and Drug Administration Boston Collaborative Drug

Surveillance Program (table), 119Food and Drug Administration for Implanted Cardiac

Pacemakers (table), 11sFood and Drug Administration Medically Oriented Data

System (table), 115Food and Drug Administration Nationwide Evaluation of

X-Ray Trends (table), 115Food and Drug Administration National Registry for

Drug-Induced Ocular Side Effects (table), 115Food and Drug Administration Poisoning Control Case

Registry (table), 114Food and Drug Administration Registry of Adverse

Reactions to Contrast Media (table), 115Food and Drug Administration Registry of Dermatological

Reactions to Drugs (table), 114Food and Drug Administration Registry of Tissue

Reactions to Drugs (table), 114Health Index Info (table), 122Health Planning & Admin, 28, 29Health Planning and Administration data base (table), 122HISTLINE (table), 122Hospital Discharge Data Systems (table), 118International Pharmaceutical Abstracts (table), 122IRL Life Sciences Collection (table), 122Johns Hopkins University POPLINE (table), 122Kaiser-Permanente Health Services Research Center

(Oregon Region) System (table), 120Laboratory Animal Data Bank (table), 122Manitoba Health Services Commission Databank (table),

120Massachusetts General Hospital Computer-Stored

Ambulatory Record System (table), 118Massachusetts General Hospital Intensive Care Databank

(table), 119Medicare Claims File (table), 116Medicaid Management Information System (table), 116MEDLINE (table), 122MEDOC (table), 122Nat iona l Academy of Dermato logy Reg is t ry o f

Dermatological Reactions to Drugs (table), 114

National Cancer Institute Surveillance, Epidemiology, andEnd Results Program (table), 115

National Clearinghouse for Mental Health Information(table), 122

National Health Interview Survey (table), 113National Health and Nutrition Examination Survey

(table), 113National Hospital Discharge Survey (table), 113NCHS Basic Vital Statistics (table), 113NCHSR Computer-Stored Ambulatory Record System

(table), 118NCHSR/Duke University Cardiovascular Data Bank

(table), 118NCHS National Ambulatory Medical Care Survey

(table), 113NCHS National Death Index (table), 113NCHS National Fetal Mortality Survey (table), 113NCHS National Health Interview Survey (table), 113NCHS National Health and Nutrition Examination Survey

(table), 113NCHS National Hospital Discharge Survey (table), 113NCHS National Natality Index (table), 113NCHSR Problem Oriented Medical Information Systems

(table), 118New York Hospital databank on patients in coma for

causes other than head injury (table), 118NHLBI Coronary Artery Surgical Study (table), 116NHLBI Framingham Heart Study (table), 115NHLBI Hypertension Detection Second Followup Program

(table), 115NHLBI Multirisk Factor Intervention Trial (table), 115NHLBI Percutaneous Transluminal Coronary Angioplasty

Registry (table), 116NIAAA Alcoholism Program Monitoring System

(table), 114NIAMDD National Diabetes Data Group (table), 116NIDA Client Oriented Data Acquisition Process

(table), 114NIDA Drug Abuse Warning Network (table), 114NIDA National Drug Abuse Treatment Utilization Survey

(table), 114NIOSH Registry of Toxic Effects of Chemical Substances

(table), 125Olmstead County, Minnesota System (table), 119Pediatric Drug Surveillance Program (table), 119Pittsburgh Diabetes Registry (table), 118PRE-MED (table), 125PSRO Hospital Discharge Data Set (table), 116Rhode Island Diabetes Registry (table), 118Rhode Island Health Services Research, Inc. SEARCH

(table), 119RINGDOC (table), 125Science Citation Index SCISEARCH (table), 125Seattle Group Health Cooperative System (table), 119Seattle Heart Watch (table), 120SERLINE (table), 125SOCIAL SCISEARCH data base (table), 125SS1 Annual Disability Determination/Social Securi~y

Income Extract (table), 117

Index ● 225

SS1 Continuous Disability History Survey (table), 117SSIE (table), 126SS1 Leed File (table), 117statistical (table), 113-120Toxicology Data Bank (table), 126TOXLINE (table), 126University of North Carolina Population Bibliography

(table), 124University of Vermont Problem Oriented Medical

Information Systems (table), 118Utah Genealogical Data Base (table), 116VETDOC (table), 126Veterans Administration Compensation and Pension

System (table), 117Veterans Administration Patient Treatment File (table), 117

decisionmaking, 66-67Delphi technique, 15, 65, 66, 93, 145-146

features of, 62Department of Agriculture, 29Department of Commerce, 29, 30Department of Defense (DOD), 13, 30, 51, 105Department of Education, 29Department of Energy (DOE), 29, 30Department of Health, Education, and Welfare (see

Department of Health and Human Services)Department of Health and Human Services (DHHS), 9, 16,

18, 25, 29, 80, 85, 96, 108, 109Ethics Advisory Board, 13, 50, 51Technology Coordinating Committee, 9, 84, 107

Department of Housing and Urban Development, 29Department of Justice, 29Department of Labor, 29Department of Transportation (DOT), 29DHEW (see Department of Health and Human Services)DHHS (see Department of Health and Human Services)DIALOG Information Service, Inc., 28discounting, 41DOE (see Department of Energy)DOT (see Department of Transportation)drugs, 9, 71-72

advertising of, 86adverse reactions to, 98Federal regulation of, 78-79investigational new (IND), 79new drug application (NDA), 9, 12, 79Phase I studies of, 79Phase II studies of, 79Phase 111 studies of, 79surveillance of, 36, 97-98

Duke University, 25

electroencephalograph, 83Emergency Care Research Institute (ECRI), 78endoscopy, 82end-stage renal disease (ESRD), 48End-Stage Renal Disease Program, 48ESRD (see end-stage renal disease)estrogen, 38

Excerpta Medica, 28external validity, 129, 139

FDA (see Food and Drug Administration)fetal monitoring, 57, 129Food and Drug Administration (FDA), 3, 7, 8, 9, 17, 72,

74, 78, 79, 80, 85, 87, 88, 92, 94, 95, 96, 105, 127regulation of drugs and devices by, 77, 99reporting system for adverse drug reactions, 98role in medical technology assessment, 12-13surveillance of drugs, 97-98, 98

Fowles, Jinnet, 192

gastric freezing, 4, 130, 167g]ossary, 198-202group decision methods, 93, 144-146

advantages and disadvantages of, 65-66

Hartford Foundation, 78HCFA (see Health Care Financing Administration)Health Care Financing Administration (HCFA), 7,9, 17, 18,

25, 48, 52, 74, 75, 84, 87, 88, 92, 94, 95, 96, 97, 98,99, 100, 101, 105, 106, 109

Health Standards and Quality Bureau, 77Office of Coverage Policy, 77Office of Research and Demonstrations, 13, 76, 77, 9.5proposal to allow reimbursement for experimental

technologies, 107-108research programs of, 76-77role in medical technology assessment, 9

health maintenance organizations (HMOS), 80Health Resources Administration (HRA)

Bureau of Health Planning, 29, 30hemodialysis and kidney transplant surgery

case study of, 175-184

informed consent, 49innovation

case studies of medical, 167-184characteristics of successful, 151-152definitions of, 150process for drugs and devices, 152-155process for medical and surgical procedures, 155-157public accountability in, 157-165research on, 150-151

Institute for Health Care Evaluation, 19model of, 185-191proposal for, 107

insurance claims records, 25internal validity, 138-139

Johns Hopkins Hospital, 78

W. K. Kellogg Foundation, 26

legislationAlcohol Abuse and Alcoholism Act, 80Community Mental Health Centers Act, 80

226 ● Strategies for Medical Technology Assessment———— ———.

creation of the National Center for HeaIth CareTechnology (NCHCT), 106

Food, Drug, and Cosmetic Act, 79, 93Freedom of Information Act, 51Medical Device Amendments (to Food, Drug, and

Cosmetic Act), 9, 79, 127National Health Planning and Resources Development

Act, 13, 80Public Health Services Act, 80Social Security Act, 6, 80, 81State certificate-of-need laws, 80

Library of CongressNational Referral Center, 29

Lohr, Kathleen, 192

macroallocation, 49mammary artery ligation, 130Manitoba Health Services Commission, 36maternal serum alpha-fetoprotein (MSAFP), 48, 96

case study of, 174-175Mayo Clinic, 37, 77Medicaid, 26, 48, 76, 80, 81, 83, 98-medical devices, 72

Federal regulation of, 79-80Medical Literature Analysis and Retrieval System (see

MEDLARS)medical technologies

assessment strategies for, 3-4, 6, 92-93capita] investment in, 161conceptual framework for assessing, 7cost-effectivness of, 52definition of, 3development, diffusion, and use of, 71-88economic effects of, 11-12, 14-15, 52effectiveness of, 52efficacy of, 33ethical issues in, 48-49, 52Federal role in assessing, 4, 6health effects of, 10-11, 14-15, 33-39health professionals and, sidentification of assessment needs, 8-10, 16-17industry, 5-6, 99information dissemination on, 15-16, 17-18, 84-85, 85-86innovation process for, 71-74, 150-166legal implications of, 52marginal valuation of, 39-40market for, 6medical devices, 72, 159-160medical and surgical procedures, 73, 99, 160-161opportunity cost of, 39pharmaceutical industry, 71-72public concern over, 4-5regulation of, 86-87, 158-161reimbursement policy effects on, 81-84, 161-165research findings on, 14safety of, 33, 52social effects of, 12, 14-15, 47-50, 52testing of, 10-13, 17value premises of assessment of, 45-47

Medicare, 4, 6, 7, 26, 52, 75, 76, 80, 81, 83MEDLARS, 18, 28, 29MEDLINE, 28

subject coverage of, 101menisectomy, 82meta-analysis, 58-59, 128, 142-143methotrexate, 37MSAFP (see maternal serum alpha-fetoprotein)multiple sclerosis, 78Multiple Sclerosis Foundation, 78

National Academy of Sciences (NAS)Institute of Medicine, 106

National Cancer Institute (NCI), 30Surveillance, Epidemiology, and End Results (SEER)

program, 26National Center for Health Care Technology (NCHCT), 9,

13, 17, 18, 50, 52, 63, 75, 77, 84, 85, 87, 94, 95, 96,98, 108, 110, 127

Clinical Data Acquisition Plan, 99health care research programs of, 76proposal to refund, 107role in medical technology assessment, 9, 13

National Center for Health Services Research (NCHSR), 9,17, 18, 25, 75, 84, 95, 97, 99-100, 105, 109

health services research programs of, 75-76User Liaison Program, 15, 84

National Center for Health Statistics (NCHS), 24-25, 27National Commission, 13, 50-51National Health Planning Information Center, 29National Heart and Lung Institute (see National Heart, Lung,

and Blood Institute)National Heart, Lung, and Blood Institute, 26, 49National Institutes of Health (NIH), 9, 17, 18, 25, 28, 51,

57, 63, 65, 66, 74, 77, 88, 92, 95, 96, 97, 99, 101,105, 109

basic research funding by, 74-75clinical trials supported by (table), 75consensus development in, 144-145consensus development meetings of (table), 64Office for Medical Applications of Research (OMAR), 16,

63, 85, 94, 98, 100, 109role in medical technology assessment, 9, 13

National Library of Medicine (NLM), 15, 16, 28, 29, 30,65, 109

data bases of, 18, 28, 101, 102expansion of data base to include more Governmental

research reports, 110Hepatitis Knowledge Base, 63-64, 100-101, 102L i s t e r H i l l N a t i o n a l C e n t e r f o r Biomedica]

Communications, 63Nationa] Science Foundation (NSF), 13, 29, 51National Technical Information Service (NTIS), 16, 28,

30, 102data base of (table), 122

NCHSR (see National Center for Health Services Research)NIH (see National Institutes of Health)NLM (see National Library of Medicine)nominal group technique, 62-63

Index . 227

non-randomized clinical trials, 127NSF (see National Science Foundation)NTIS (see National Technical Information Service)

observational studies, 35-38Office of Science and Technology (see White House Office

of Science and Technologyoffice of Technology Assessment (OTA), 3, 14, 16, 19, 24,

25, 30, 33, 39, 46, 47, 57, 60, 61, 67, 94, 109, 129acknowledgments to contributors to this report, 195-197conclusions on adequacy of testing for safety and

efficacy, 100conclusions on communication of information on medical

technologies, 102conclusions on identification of medical technologies, 96,

97, 98conclusions on synthesis phase of assessment, 101functions of, 50health-related reports of, 13methods used in this study, 192-194other volumes produced by this study, 193-194

OlvlAR (see under National Institutes of Health)osteosarcoma, 37OTA (see Office of Technology Assessment)overhead costs, 40

Pap test program, 40peptic ulcer, 4percutaneous transluminal coronary angioplasty (PTCA),

26, 87case study of, 168-174

Peter Bent Brigham Hospital, 78phakoemulsification, 82Phase I studies, 12, 79Phase 11 studies, 12, 79Phase 111 studies, 12, 79plasmapheresis, 78policy options, 18, 105-110

for congressional oversight, 108-110for coordination of technology assessment, 110for dissemination activities, 109-110for education of researchers, 108for identifying medical technologies for assessment, 109]egis]ative, 106-109organizational, 106-107for private sector oversight, 108for research funding, 107-108for synthesizing research information and group

decisionmaking activities, 109for testing medical technologies, 109

policy-related activities, 50-52, 74-86President’s Commission, 13, 50-52prices vs. costs, 40-41

Professional Standards Review Organizations (PSROS), 13,80, 92, 95, 97, 98, 101, 105, 106, 109

Project Share, 29-30prospective studies, 10PSROS (see Professional Standards Review Organizations)psychotherapy, s9, 73Public Health Service, 25, 77, 96, 107, 109

Health Services Research Coordinating Committee, 84

radiology, 52, 83Rand Corporation, 62randomized clinical trials (RCT), 10, 11, 34-35, 39, 57, 74,

78, 91, 93, 105, 106, 109, 127, 128attributes of, 130-131basis for conducting, 149

RCT (see randomized clinical trials)Reiss, John, 192research and development (R&D), 11, 15

costs of, 4 0in development, diffusion, and use of medical

technologies, 74-78Federal activities in, 75-77private sector activities in, 77-78protection of human subjects, 50-52

research (see research and development)Royal Perth Hospital, 37

schizophreniahemodialysis treatment of, 167-168

SEER program (see under National Cancer Institute)Simpson’s paradox, 58Southwestern Michigan Hospital Council, 27spina bifida, 48Stanford University, 25statistical conclusion validity, 34, 129, 139synovectomy, 82System Development Corp., 28

third-party payers, 73, 81-84, 86-87tomography (see CT scanning)Toxicology Information Online (see TOXLINE)TOXLINE, 28

Veterans Administration, 13, 51, 105, 130vital statistics, 25voting method, 58, 141

Wennberg, John, 192White House Office of Science and Technology, 13, S1Williamson, John, 192Winkler, John, 192Woolf, Patricia, 192Wortman, Paul, 192

0

Strategies for Medical Technology Assessment

Documents

Transcript of Strategies for Medical Technology Assessment