Using Health Research in Policy and Practice:

Using Health Research in Policy and Practice:Case Studies from Nine Countries

Milbank Memorial FundAcademyHealth

by Ray Moynihan

©2004 Milbank Memorial Fund

Milbank Memorial Fund

645 Madison Avenue

New York, NY 10022

All rights reserved. No part of this

publication may be reproduced, stored in

any retrieval system, or transmitted in any

form or by any means, electronic, mechanical,

photocopying, recording, or otherwise without

prior permission.

The Milbank Memorial Fund is an endowed

operating foundation that engages in

nonpartisan analysis, study, research, and

communication on significant issues in health

policy. In the Fund’s own publications, in

reports or books it publishes with other

organizations, and in articles it commissions

for publication by other organizations, the

Fund endeavors to maintain the highest

standards for accuracy and fairness.

Statements by individual authors, however, do

not necessarily reflect opinions or factual

determinations of the Fund.

Printed in the United States of America.

ISBN 1-887748-60-1

Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .v

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vi

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2

Bangladesh: Narrowing the Gap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

Canada: Reconnecting Nurses and Patients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

Chile: Evidence for Equity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

England and Wales: Evidence Is Not Enough . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

Russia: Evidence-Friendly System Saves Lives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

South Africa: AIDS Survey Hits Home Run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15

Thailand: Evidence for Action on Universal Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17

United States: Best Evidence to the Bedside . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

The World Health Organization’s Health Evidence Network . . . . . . . . . . . . . . . . . . . . . . . . . . .22

Common Themes: Communication, Integration, Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . .24

Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

The Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35

T A B L E O F C O N T E N T S

Milbank Memorial Fundv

Many policymakers and clinicians take professional pleasure in grounding their decisions in the best

available evidence, especially when they have confidence in that evidence and access to it when they need

it. Similarly, many researchers are eager for their work to be useful, especially when they are only

required to communicate it in journal articles and on a schedule that they choose.

In this report, Ray Moynihan, an experienced journalist, describes collaboration between

researchers and users of their skills and findings in nine countries. Moynihan introduces the report by

briefly recapitulating controversial issues in the application of knowledge derived from research to

policy and practice. He concludes by describing common themes in the nine stories.

Moynihan began his reporting of the nine stories at an international conference in the fall of 2003

that was sponsored by AcademyHealth and the Agency for Healthcare Research and Quality. The Milbank

Memorial Fund commissioned him to roam the conference in search of stories about how research

informs policy and practice. He completed his reporting after the conference.

Many people helped to make these stories by providing, interpreting, and contextualizing evidence

and then making decisions based in part on it. We thank the persons named in each of the nine stories,

each of whom had an opportunity to comment on what Moynihan wrote, as well as the reviewers who

commented on the report as a whole.

Daniel M. Fox

President

Milbank Memorial Fund

David Helms

President and CEO

AcademyHealth

F O R E W O R D

viAcademyHealth

The following persons were interviewed for this report and/or reviewed it in draft. They are listed in the

position they held at the time of their participation.

Jordi Alonso, Unit Institut Municipal d’Investipaun Media, University of Barcelona, Spain; Donald M.

Berwick, Chief Executive Officer, Institute for Healthcare Improvement, Boston; Gabriel Carasquilla,

Doctor, University of Valle, Colombia; Timothy S. Carey, Professor of Medicine, University of North

Carolina, Chapel Hill; Mushtaque Chowdhury, Visiting Professor, Columbia University, New York;

Carolyn Clancy, Director, Agency for Healthcare Research and Quality, Rockville, Md.; Karen Davis,

President, The Commonwealth Fund, New York; Andrés de Francisco, Deputy Executive Director, Global

Forum for Health Research, World Health Organization, Geneva; Richard Deyo, Professor, University of

Washington, Seattle; Mary Ferguson-Paré, Chief Nursing Executive, University Health Network, Canada;

Kathleen M. Foley, Chief, Pain Service, Department of Neurology, Memorial Sloan-Kettering Cancer

Center, New York; Miguel Gonzalez-Block, Programme Manager, Alliance for Health Policy and Systems

Research, World Health Organization, Geneva; David Helms, President and CEO, AcademyHealth,

Washington, D.C.; Liliana Jadue Hund, Instituto de Epidemiología y Políticas Públicas, Facultad de

Ciencias de la Salud, Universidad del Desarrollo, Chile; Egon Jonsson, Leader, Health Evidence Network,

World Health Organization, Copenhagen; John Lavis, Associate Professor, McMaster University,

Hamilton, Ontario, Canada; Steven Lewis, President, Access Consulting Ltd., Saskatoon, Canada; Peter

Littlejohns, Clinical Director, National Institute for Clinical Excellence, London; Jonathan Lomas,

Executive Director, Canadian Health Services Research Foundation, Ottawa; Princeton N. Lyman,

Senior Fellow, Director, Africa Policy Studies, Council on Foreign Relations, Washington, D.C.; Rashad

Massoud, Director, Quality and Performance Institute, University Research Company, Washington, D.C.;

Andrew D. Oxman, Director, Department of Health Services Research, Norwegian Directorate for Health

and Social Welfare, Oslo; Tikki Pang, Director, Research Policy and Cooperation, World Health

Organization, Geneva; Jonathan Perlin, Deputy Under Secretary for Health, Department of Veterans

Affairs, Washington, D.C.; Patricia Pittman, Director for International Initiatives, AcademyHealth,

Washington, D.C.; Hernan Sandoval, Executive Secretary, National Committee for Health Reform, Chile;

Kenneth I. Shine, Executive Vice Chancellor for Health Affairs, University of Texas; Olive Shisana,

Executive Director, Human Sciences Research Council, South Africa; Ronald J. Stamp, Institute of

Medicine, Health and Social Care, University of Portsmouth, United Kingdom; William Tierney, Chief of

General Internal Medicine and Geriatrics, Indiana University; Suwit Wilbulproprasert, Deputy

Permanent Secretary, Ministry of Public Health, Thailand.

The author would like to thank those who assisted with note-taking and other arrangements at the

Washington, D.C., conference: Anne Gauthier, Ethan Joselow, Tamar Klaiman, Patricia Pittman,

Deborah Queenan, Deborah Rogal, Kari Root, and Wendy Valentine.

A C K N O W L E D G M E N T S

Milbank Memorial Fund1

As the move toward an evidence-based approach to health care has gathered momentum over the past

two decades, increased attention has been paid to strategies that try to ensure that knowledge gained

from the best evidence is actually used in practice. This move to enhance the utility of research involves

both making research evidence more usable and improving the capacity of management, policy, and

clinical decision makers to use it.

Using evidence to improve health services has been the focus of a series of five international

conferences held over the past decade, the most recent one in Washington, D.C., in September 2003.

Drawing on presentations at that conference, this report presents nine short case studies from around

the world, where very different sorts of research evidence are being used in very different policy and

practice settings. The purpose of these case studies is not to identify “proven” translation strategies but

to offer insights into the ways different people are trying to better use scientific research to improve

human health.

Several common themes arise from the insights offered. Communication between those who

produce and those who use research is critical, though close relationships between researchers and

decision makers raise questions about the blurring of roles and potential impacts on scientific

autonomy. Integration of research, research utilization, and routine practice is a key goal within health

care organizations and systems, though rarely achieved. Evaluation of efforts to better use evidence is

crucial yet often poorly done, reducing confidence that human health is actually being improved.

E X E C U T I V E S U M M A R Y

2AcademyHealth

One way to fight malaria, a disease that afflicts half a billion people, is to “promote the role of science in

decision making,” according to a recent report of the Global Health Council. “By fully implementing

what we know today, more lives can be saved and the health of millions can be improved.” The

organization’s president reiterated the message: “Health decision makers must be at the forefront of

putting research into action if the promise of a world in which malaria does not dictate the lives of

millions is to be fulfilled.”

The prescription—to better use existing knowledge—is by now as familiar as it is urgent. In the Lancet

in 2003, an international working group on child mortality declared: “Application of what we know can

reduce child mortality by two-thirds. . . . [T]he capacity of countries to obtain and use information to

support child health programmes will be a determining factor in reducing child mortality.”

Using scientific evidence to improve health services has been the theme of a series of five

international conferences held over the past decade. The most recent one, held in Washington, D.C., in

September 2003 and jointly hosted by AcademyHealth and the Agency for Healthcare Research and

Quality, focused on how nations organize their health services research and how that research can be

used to improve clinical practice, management strategies, policymaking, and health outcomes. The

presentations made at that conference and discussions with participants form the basis of the short case

studies covered in this report.

As the move toward an evidence-based approach to health care has gathered momentum over the

past two decades, a broadening of its focus has occurred. Along with generating and disseminating good-

quality evidence about what works and what does not, increased attention is being paid to strategies that

try to ensure that the knowledge gained from the best evidence is actually used in practice. This move to

enhance the utility of health services research means both making research evidence more usable and

improving the capacity of management, policy, and clinical decision makers to use it.

The current debates about using evidence in practice range across many different fields of study

and vast academic terrain. “It’s a community, a trailer park of different disciplines,” mused Institute for

Healthcare Improvement chief executive officer Donald Berwick during the Washington conference,

“but progress occurs at the intersections.” The nonprofit Institute for Healthcare Improvement is one

of the groups that is pioneering strategies to integrate the best evidence into routine practice within

health care organizations. One of the strategies involves setting up “collaboratives,” whereby different

organizations and researchers come together to pool expertise in order to try and quickly achieve major

improvements in the use of best evidence and ultimately the quality of care. A recent example involved

22 organizations working together to better use the evidence about the appropriate care of lower back

pain, with 25 percent able to show some form of improvement within 12 months.

Attempting to render an overview or bring coherence to these fragmented theoretical discussions

within the “trailer park” is beyond the scope of this report. Instead, it concentrates on nine case studies

from around the world where evidence of some kind is being used in practice and progress of some kind is

thought to be occurring. Before moving to the case studies, however, a few threads of discussion arising

from both the conference and the relevant scientific literature are worth noting by way of background.

I N T R O D U C T I O N


According to a recent systematic review of the literature about how policymakers use evidence, one

of the most commonly discussed theoretical issues in that literature is what is meant by the term use. The

most common categorization suggests three different meanings—that is, a rough consensus exists in the

literature that there are three different ways in which evidence is used. First, there is direct or

instrumental use of evidence, where relevant research results are seen to be directly affecting policy

decisions. Second, there is selective or legitimating use of evidence, where evidence is used to legitimate

or sustain a predetermined position already taken by policymakers. Third, there is enlightening use of

evidence that can help enrich or deepen understanding of a problem. As we will see, evidence is used in

all three ways throughout many of the case studies.

Another important question concerns what is meant by the term evidence. Commonly, we refer to

evidence as the results of scientific studies, but as the case studies will show, many other forms of

information and knowledge, as well as values, judgments, experiences, and intuition, routinely play a

role in health care decision making, often trumping the “pure” research evidence in terms of

importance and utility. Moreover, as researchers have applied more scrutiny to the strength and quality

of “pure” evidence in recent years, more and more of its weaknesses have been revealed. It is now widely

accepted that much of the “scientific” evidence in health care is of extremely poor quality, biased in

favor of professional or commercial sponsors, producing misleading and unreliable results that

overstate the benefits and play down the harms of many health care interventions. The way forward

appears to involve decision makers relying, to whatever extent is possible, on the best of the available

evidence—based on systematic reviews of studies with designs that attempt to minimize bias. Likewise, a

decision-making process that is as open as possible about the conflicts of interest of its participants is

seen as a desirable model, one being actively pursued in England and Wales by the National Institute for

Clinical Excellence (which is described below).

One of the basic tenets of the debate about how better to move research into practice is that

personal relationships and trust will help facilitate that move, through direct communication between

researchers and decision makers and via knowledge-broker intermediaries. While this strategy is widely

accepted and highly valued, it has sparked some important questions that were explored during

discussions and presentations at the Washington conference. Are the webs of influence in health care

that affect decision making in fact much broader than the researcher-broker-policymaker circuit

involving a host of other key actors and public opinion as well? Is it not practically impossible for

decision makers to have personal relationships and direct communications with every health services

researcher? Does a close relationship between researcher and policymaker come with any cost to the

researcher’s autonomy? Although there are no clear answers, these questions appear to be provoking

valuable discussions, which echo through a number of the following case studies.

Attempting to define and delineate the roles of researcher and decision maker provokes more

fascinating tensions in discussions about translating research into practice. At the Washington

conference plenary session on national policies for organizing research, Thai policymaker Suwit

Wilbulproprasert and Kenneth Shine, an American researcher who is also past president of the Institute

4AcademyHealth

of Medicine of the National Academies of Science, advanced quite different perspectives on the nature

of those roles. Using the context of his nation’s recent move toward universal health insurance—covered

later in this report as a case study—Wilbulproprasert argued strongly that people who combined the roles

of researcher and decision maker/advocate were more effective in facilitating the translation of

evidence into practice. By way of contrast, Shine suggested that such a blurring of roles could

undermine the credibility of the scientist/researcher.

A final tension worth noting is the one between the local and the global, the center and the

periphery. Whereas a sense exists that evidence is often seen as “global” in nature, putting that evidence

into practice is regarded as a local affair. For example, the literature about using evidence to improve

health care quality stresses the importance of individual health care organizations taking the centrally

produced evidence or recommendations for best practice and “reinventing” them at the local level.

Similarly, one of the most common refrains in these debates—and from the players involved in the

following case studies—is that genuine commitment from local leadership is absolutely critical to the

success of any efforts to put evidence into practice. Others, including University of Melbourne

researcher Lenore Manderson, have pointed out that “global” evidence often arises from the

experiences of the richest countries, which can limit both the relevance and success of translational

efforts in resource-poor or culturally different settings.

In an attempt to reflect global diversity, these nine case studies vary widely, featuring different sorts

of research evidence being used by different kinds of decision makers in a variety of ways in a range of

contexts. The one clear and important limitation of this report is that all the case studies are portrayed in

a positive light, featuring enthusiastic testimony from key protagonists. Moreover, almost all the stories

and evidence advanced here to illustrate the importance of translation and implementation efforts rely

on opinions, weak study designs, or unpublished conference presentations, though in addition peer-

reviewed literature is relied on where available. Hence, the enthusiasm of the protagonists must be

weighed against weak evaluation methods and uncertainty that health outcomes are actually being

improved. However, the purpose of these case studies is not to demonstrate “proven” translation

strategies but to offer brief insights into the ways different people around the world are trying to better

use scientific research to improve human health.


Summary: Between 1980 and 1990 a nongovernmental organization in Bangladesh educated more than 12

million families about a simple, inexpensive treatment for diarrheal diseases, narrowing the gap between

knowledge and use. Concurrent evaluation of the project helped refine and improve the process.

Mixing water with sugar and salt to fight deadly diarrhea was “potentially the most important medical

advance of this century,” according to a Lancet editorial in 1978. This simple, inexpensive solution—

often called oral rehydration therapy, or ORT—has been credited with saving a million lives a year in the

developing world. Current estimates suggest that more than a million children still die annually from

diarrheal diseases, a leading cause of childhood illness and death. While improved water supplies,

nutrition, and development opportunities are among a host of urgent global priorities, the importance

of better using our knowledge about ORT has long been widely accepted.

In the late 1970s in Bangladesh, a nongovernmental organization called the Bangladesh Rural

Advancement Committee embarked on a plan to educate every household about oral rehydration

therapy. Between 1980 and 1990 the organization’s trained field force went home by home and delivered

face-to-face teaching to more than 12 million mothers. During a typical session lasting 20 to 30 minutes,

often involving a demonstration with a child with diarrhea, women learned how to prepare the

homemade solution using “a pinch of salt, a fistful of gur (brown sugar) and a half a litre of water.”

Speaking about what he saw as the success of the program, Mushtaque Chowdhury, former director of

research at the Bangladesh Rural Advancement Committee, told the Washington conference plenary

session on using evidence to improve health services management that oral rehydration therapy “is now

a part of the Bangladeshi culture.”

Demonstrating the importance of an “organizational commitment to research,” Chowdhury said

his Research and Education Division was responsible for evaluating and modifying the mass campaign

to implement the knowledge about oral rehydration therapy. Discovering early on that mortality data

were impractical to collect (in part because of the lack of birth and mortality statistics and difficulty

assigning cause of death), the organization focused on measuring the process—specifically, the retention

of knowledge by women and the actual use of the therapy in the home.

An important modification introduced early in the program involved simplifying the formula from

“two scoops” of sugar to “one fistful,” so as to synchronize the number with “one pinch” of salt and

reduce the possibility for confusion. Despite the modification, the first evaluation efforts discovered

surprisingly low rates of uptake, according to Chowdhury, currently a visiting professor at Columbia

University in New York. Searching for explanations, researchers discovered a lack of trust in the therapy

among women, in part related to low male involvement in the program. Armed with this and other key

research findings, implementation strategies were changed and expanded to include other broader

educational activities, and significantly higher rates of knowledge retention and usage were

subsequently achieved.

A study published in Health Policy and Planning found evidence that some form of oral

rehydration solution was being used in around 50 percent of diarrhea cases in Bangladesh, and that

B A N G L A D E S H : N A R R O W I N G T H E G A P

6AcademyHealth

more than 70 percent of women could prepare a “safe and effective” solution even a decade after their

household had been visited by the field-force workers. “The gap between knowledge of ORT and its use

has narrowed considerably,” the study authors wrote. Based on a survey of 9,000 households that had

been previously targeted, the authors also found some evidence that the knowledge about oral

rehydration therapy was being successfully transmitted from mother to child.

As with any technology, there is uncertainty and controversy surrounding both the benefits and

potential harms of ORT. The Health Policy and Planning authors cite another study, which found that

following the mass education campaign, only 25 percent of mothers were able to make a “correct”

solution. However, in that study, assessors used subjective criteria to analyze the prepared solutions,

rather than the chemical laboratory analysis used in the original evaluation. Whatever the success in

terms of education, major questions remain about the actual health impact of the program because

evaluation efforts were focused on processes—or “surrogate outcomes”—rather than on health outcomes.

There is also an important debate about the role of food in rehydration therapy and the type of

diarrhea for which homemade or prepackaged solutions are most effective, with some authors

suggesting that these solutions have only a limited role in nonhospital cases of diarrhea.

Notwithstanding that debate, the Bangladesh Rural Advancement Committee home-to-home program

has been held up as a global model for communicating lifesaving health information. For Chowdhury,

the experience also stands out as an example of the effective use of scientific knowledge for action.

“Research can justifiably claim credit for its part in this.”

Insights: Research evaluating initial implementation strategies can lead to refinements and improvements

in strategies.


Summary: Evidence suggests that improving nursing workloads and staffing levels can enhance nurse

satisfaction and patient outcomes. Health care organizations in Canada are aggressively attempting to

translate this evidence into practice. Some public hospitals are evaluating the impact on patient outcomes of

giving nurses more time to learn, teach, and bring research into practice.

Addressing the nursing crisis in the health systems of many nations, Institute for Healthcare

Improvement’s Don Berwick told the Washington conference that “joy” must come back to the health

care workplace, because “high quality cannot come from a demoralized workforce.” In Canada, federal

government figures show that of 48 occupations, nurses have the highest rates of “illness and injury-

related absenteeism” of any occupational group. Every week, almost one in ten nurses has some absence

from work, a rate almost double that of the average full-time Canadian worker.

Alongside the growing evidence of absenteeism, unwanted overtime, and high levels of costly nurse

turnover is strong evidence that the quality of nursing services has a direct impact on the health of

patients. At a conference session in Washington devoted entirely to nursing, researchers presented

evidence that patients were significantly less likely to die after a heart attack if they were in a hospital

with higher numbers of registered nurses. Similarly, patients undergoing stomach surgery were

significantly less likely to experience major complications if they were in intensive care units with higher

nurse staffing levels.

In Canada a concerted effort is being made to take all this research evidence and translate it into

practice to simultaneously improve both the nursing workplace and patient outcomes. One person

directly engaged in that translation effort is Mary Ferguson-Paré, the chief nursing executive at the

publicly funded University Health Network, with three hospitals and 3,000 nurses in Toronto, Ontario:

“I am on a path trying to take the research evidence, put it in practice, and then test it.” Apart from using

and testing best-practice nursing guidelines, Ferguson-Paré is also running two innovative

demonstration projects in 2004 with high-level support from within her own organization and

encouragement from within the federal and provincial governments. “Senior management,” she says,

“is really committed to us taking the evidence and acting on it, and the organization has the capacity to

move research findings into practice.”

The first of the projects involves three nursing units in teaching hospitals across Ontario making

slight increases to their nursing capacity on every shift. The plan grew out of a recent study at University

Health Network in which nurses were asked how much extra time they would need per shift to satisfy

patient needs adequately. The result of that study suggested nursing units should increase nurse staffing

levels by the equivalent of an extra 30 minutes per nurse per shift. Nurses in these units will stick to their

traditional shift lengths, so overall the amount of nursing staff will increase slightly, as will the ratio of

nurses to patients. The project will run for 12 months, and researchers will track rates of absenteeism,

adverse events, nurse turnover, and patient satisfaction.

The second of the projects involves a controlled study of a new “innovation unit.” Starting in early

2004, the 30 or so nurses of the orthopedic-rheumatology unit of Toronto Western Hospital will have to

C A N A D A :

R E C O N N E C T I N G N U R S E S A N D P A T I E N T S

8AcademyHealth

spend only 80 percent of their employment time on clinical work, taking the other 20 percent as release

time to be spent teaching, learning, and translating evidence into practice. The idea is to moderate the

nurses’ workloads and increase their opportunities for continuous learning. “We want to improve the

capacity of registered nurses to connect with patients and provide care,” says Ferguson-Paré, who is a co-

investigator of the controlled study. It will measure a host of indicators, including staff and patient

satisfaction and genuine health outcomes, including the number of patient falls and the rate of

medication errors.

An important part of the context for these projects is the wider work of the Canadian Health

Services Research Foundation and the Canadian Institutes for Health Research, organizations

promoting the translation of evidence into practice, within nursing and more broadly across health care.

One of the key academic advisers on the translational activities is the University of Toronto’s Linda

O’Brien-Pallas, a recipient of one of the recently created decade-long Canadian research chairs, a

program designed to build both research capacity and utility, co-funded with public resources that come

through both organizations. Based at the University of Toronto, O’Brien-Pallas has estimated that

nursing turnover costs more than US$20,000 per nurse, and that, in total, high turnover, absenteeism,

and the use of agency staff may be costing Canada C$1.5 billion a year.

One of the desired outcomes from the current demonstration projects is that, although they require

a bigger initial investment (extra nursing staff costs money), over time the projects will save resources or

at least break even. One of the biggest challenges in the translation work for Ferguson-Paré is finding

that extra money up front, which often calls for what she terms a “leap of faith” on the part of senior

managers and funding agencies. “It really requires outstanding leadership in health care organizations

to be scanning the environment, listening to the new research, and turning that into the ‘how we do’ the

‘what we do.’”

Insights: Organizational leadership needs to be committed to translating evidence into practice; investing

up front in costly translation exercises and their evaluation may bring long-term rewards.


Summary: Evidence about health inequity in Chile is indirectly affecting the current reforms of the national

health care system and enhancing the importance of primary health care networks. An innovative team of

researchers and policymakers called the Chilean Health Equity Gauge has been playing a role.

“Scientific knowledge has been one of the basic pillars of our proposal for reform, especially the

knowledge about equity,” according to National Committee for Health Reform executive secretary

Hernan Sandoval, a chief health adviser to Chilean president Ricardo Lagos. “In discussions about what

to do, everyone agreed we had to reform, but there was much disagreement on what to do. The only

subject that nobody could contest was inequity.”

In recent decades Chile has experienced big increases in economic growth, decreases in poverty,

and improvements in employment, education, and health, yet all groups within society have not

benefited equally from these advances. Infant mortality rates in Chile remain five times higher for

mothers with a basic education, compared with mothers who have completed secondary school.

Likewise, infant mortality is more than ten times higher in southern, rural areas compared with wealthy

parts of Santiago.

Such evidence about health inequity is having an impact on health policy in Chile, according to

Sandoval. This is in no small part due to the work of a small team of researcher/advocates called the

Chilean Health Equity Gauge, a group actively involved in the reform process. The team in Chile is part

of an international network focused on monitoring national inequities in health and health care

services and committed to closing the gap in health status between different socioeconomic groups.

The work of the Equity Gauge includes an innovative fusion of research, advocacy, and direct

participation with stakeholders, and it has been described as an “extraordinarily effective alliance,”

particularly in terms of building relationships between researchers and key decision makers. As part of

its work, the group has produced and disseminated several reports on the problem of health inequities in

Chile, as well as potential solutions. “The combination of advocacy and knowledge is very powerful,”

says Sandoval, who has worked with the group since 1999. A key achievement of the Equity Gauge was

helping to develop and write specific health objectives for the Chilean government for the 2000–2010

decade. It has also been officially commissioned to monitor and disseminate data on health inequities as

the government attempts to close the health status gap between different socioeconomic groups in Chile.

Asked whether individual pieces of research evidence had been utilized directly in specific policy

decisions, Sandoval said the impact of the evidence about health inequity was more cumulative and

indirect—what might be called an “enlightening” use of evidence. Describing an interdependence

between research and policy, he pointed to the fact that one of the major slogans of the Lagos party in

Chile is “Desarrollo con Equidad” (“Development with Equity”).

One practical strategy for reducing health inequity being used in Chile’s current reforms is a

dramatic boost to the role and funding of primary health care. In 2000, at the time of the election of the

Lagos government, primary care in Chile received just 12 percent of the health care budget and was still

widely viewed as a “second-class system,” mainly there to service the poor—essentially the poor cousin of

C H I L E : E V I D E N C E F O R E Q U I T Y

10AcademyHealth

the hospital and specialist sector. Yet in just four years, primary care’s share of the national health

budget has jumped to 18 percent, and there are plans to increase that to perhaps as much as 30 percent

with ambitious programs involving 60 new facilities across the nation.

Reformers such as Sandoval who believe social determinants play a key part in health see the new

role of primary care as a critical interface between society and the medical system, between “public

action for health” and “specific health services,” as he puts it. The reforms are about enhancing the role

of primary care, integrating it better with other parts of the system, and trying to change public

perceptions about health and health care services. “One of the big problems is people still think they are

going to find health solutions with specialists or in hospitals. We have to show that improving primary

care can improve health.”

A senior health aide to President Salvador Allende in the early 1970s, Sandoval is enthusiastic about

the current prospects for health reform in Chile and the use of evidence in the process. In February 2004

Sandoval attended a seminar in London, sharing the Chilean experiences of reform with policymakers

in the National Health Service. Adding a historical perspective to the current changes, he explains that,

in his view, although many of Chile’s physicians and their representative organizations had fought hard

for an end to dictatorship and the return of democratic political reform, the push for fundamental

reform of the health system took a backseat—until now.

Insights: Fusing research, advocacy, and stakeholder participation can create effective alliances. Research

evidence can indirectly affect decision making by “enlightening” the process.


Summary: The National Institute for Clinical Excellence is translating evidence into practice by producing

guidance for managers, professionals, and consumers on effectiveness and cost-effectiveness of health care

interventions. Yet other forms of information and knowledge also influence decisions about cost-

effectiveness. Decision making about a new, high-profile cancer drug provides a good example of where the

evidence is seen as being not enough.

The National Institute for Clinical Excellence (NICE) has emerged in recent years as a strong,

international example of an organization dedicated to translating evidence into practice. Using

evidence-based methods, NICE facilitates systematic reviews of research about health care

interventions and produces usable guidance on those interventions for people across England

and Wales. No matter whether a “guidance” finds for or against a new treatment, clinicians

and other health professionals are expected to take it into account, though they are not strictly

bound by it.

The aim is to help health care managers, health professionals, and consumers make better-

informed decisions about how well different treatments, therapies, tests, and devices might work, and

which ones give the best value for the money. Yet as senior executives from the institute wrote in

Lancet Oncology in 2003, “Although NICE’s approach to decision-making is evidence based, pure

research evidence alone is not sufficient to make judgements.”

The official “guidance” NICE recently produced on the controversial new leukemia drug

imatinib (sold under its brand names, Glivec and Gleevec) provides a fascinating insight into the way

evidence can interact with other forms of knowledge on the path to implementation. As the NICE

executives candidly elaborated in commenting on the institute’s imatinib guidance, “Although the

guidance produced is firmly based in published evidence, it is also considerably influenced by

information from numerous other stakeholders including professional organizations, patient/carer

groups, and the pharmaceutical industry.”

Appraising the effectiveness and cost-effectiveness of cancer drugs can be particularly difficult,

according to Peter Littlejohns, clinical director of NICE, because there is often a lack of good-quality

data from long-term randomized, controlled clinical trials. With imatinib, members of the NICE

appraisal committee were forced to make decisions about the drug’s cost-effectiveness in the absence

of good, long-term data and on the basis of “surrogate” outcomes, rather than on the basis of evidence

demonstrating proven advantages in morbidity or mortality.

After releasing a very negative preliminary finding for public consultation on this much-

publicized drug, the committee was persuaded to recommend in favor of the drug’s use. In a section

of their guidance titled “Consideration of the Evidence,” the committee wrote that apart from

reviewing additional updated scientific data, which helped strengthen the link between surrogate

outcomes and clinically meaningful benefits, they also considered “evidence on the nature of the

condition and the value placed on the benefits of imatinib from people with CML [chronic myeloid

leukemia], those who represent them, and clinical experts.”

E N G L A N D A N D W A L E S :

E V I D E N C E I S N O T E N O U G H

12AcademyHealth

“There is no algorithm for this decision making,” observes Littlejohns. A former clinician and

professor of public health turned policymaker, Littlejohns stresses that making judgments and

recommendations about cost-effectiveness necessitates weighing different assumptions and opinions,

and that the same evidence base can be read very differently, depending on the perspective you bring to

the reading. Asked whether NICE processes make decisions vulnerable to political or commercial

pressures from vested interests, Littlejohns acknowledges that the public consultation procedures allow

for “a form of lobbying.” He argues, however, that this is legitimate, as long as it is a comprehensive

consultation and all relevant conflicts of interest are made public as part of the process.

As it happens, part of the reason NICE was created was because different health authorities

across England and Wales were looking at the same evidence and making different decisions about

which services or interventions to fund. For example, some local authorities supported the use of a

drug class called taxanes for breast cancers, while others did not. Now if NICE makes a positive

recommendation in its national “guidance”—as it did with imatinib—a local authority has to make

funds available for that specific intervention within three months, even if that cuts across other

priorities deemed more important at the local level.

“There will always be a tension between the center and the periphery,” says Littlejohns. “If you

want a health service where there is equal access to services, there is a reduction in local autonomy

that you have to live with.”

Insights: Research evidence alone is often not enough for making cost-effectiveness judgments about health

care interventions; decision making can involve comprehensive consultation with all stakeholders, and all

conflicts of interest should be made public as part of the process.


Summary: The redesign of a system of care for sick newborns in a region of Russia produces improvements

in mortality. Evidence is used to help redesign the system, which is evaluated in five hospitals before being

implemented across 43 health care facilities in a region with a population of 2 million.

In Tver Oblast in the Russian Federation in late 1998, heath professionals and managers from local

health authorities and international organizations—including the Tver Oblast Health Authority, the

Central Public Health Research Institute in Moscow, the Russian Ministry of Health, the U.S. Agency

for Healthcare Research and Quality, and the U.S. Agency for International Development’s Global

Quality Assurance Project implemented by University Research Co. LLC—decided to embark on

improving the care for newborns suffering respiratory distress syndrome, a condition in which a

baby’s lungs do not function properly, making breathing extremely difficult. Baseline data taken

before any changes were implemented suggested that almost half of the neonates arriving at local

health care facilities with this condition were suffering hypothermia, and many of them were dying in

their first week of life.

Those health professionals involved with providing the care worked with others to redesign the

management of this clinical condition across five hospitals in the city of Tver, with an emphasis on

finding and using the best available evidence. Most significant, the system of care was changed: A new

central referral center was created; a new system of transportation with ambulances was devised; and

relevant staff members at all facilities were trained in neonatal resuscitation techniques.

Simultaneously, “evidence-based clinical guidelines” were developed for all stages within the new

system. Critically, those implementing the changes helped evaluate their impact and, as an integral

part of the improvement methodology, fed the results back into the ongoing redesign of the system.

The new system was rolled out toward the end of 1999. Within three months, the percentage of

these neonates arriving at hospitals with hypothermia had fallen from more than 50 percent to almost

zero, according to University Research Company’s Rashad Massoud, in a presentation about the case

study at the Washington conference. The proportion of babies dying in the first week of life also

began to drop, but less dramatically and more slowly; nevertheless, in relative terms it fell 64 percent

over 12 months.

“What we’re trying to do is implement evidence and test different modalities for implementing

evidence, with quasi-experimental research in real time,” says Massoud, the project leader from the

United States side of the collaboration. The teams continued to collect data on the redesign through

2003, though it is as yet unpublished. “It’s showing that the gains have held.”

What was first tried on a small scale in five hospitals has since become a much larger scale-up in

implementing evidence and improving care. In 2000 the program expanded from five hospitals to

43 hospitals covering all of Tver Oblast. Tracking data between 2001 and 2003 for the whole region,

the researchers found improvements in the health of babies with respiratory distress syndrome:

Early neonatal mortality in Tver Oblast was cut in half, from 10.8 per 1,000 in 1998 to 5.3 per

1,000 in 2002.

R U S S I A :

E V I D E N C E - F R I E N D L Y S Y S T E M S AV E S L I V E S

14AcademyHealth

Asked about the lack of comparable data from a “control” region, Massoud said, “Research is not

the first priority in this project; this is a quality improvement project with multiple interventions being

tested.” Explaining that local health professionals were essentially redesigning and improving their

own systems of care, he stressed that there was no real separation between the research and the

provision of care: “The researcher is the same person as the provider.” Asked about important

insights gained through the project, he said: “One of the lessons here is that we can use evidence in

resource-constrained settings. We can do it, and it may in fact be more wasteful not to.”

Insights: Monitor and evaluate changes on a small scale first, and use evidence at every stage in the system.

Implementing evidence can work in resource-constrained settings.


Summary: A survey on HIV/AIDS in South Africa contributes to national policy change toward the virus

and its treatment. The study’s principal investigator explains the reasons behind this example of evidence

affecting policy and practice.

It’s not every day a health survey makes world headlines and then goes on to help effect changes in

national health policy. But then, unlike the South African HIV/AIDS survey released in December

2002, most surveys are not commissioned and launched by Nelson Mandela. Funded in part by the

Nelson Mandela Foundation and conducted by the South African Human Sciences Research Council,

the first community-based survey of the HIV epidemic in South Africa serves as an important example

of how evidence can interact effectively with the policy process.

At the time of the survey’s release, headlines from Johannesburg to Washington highlighted its

frightening finding that almost 6 percent of South African children were HIV-positive. But alongside

the alarm, the survey also produced some good news: Almost half the population reported that they

were changing their sexual practices in response to the epidemic. South African men and women were

changing partners less frequently and using condoms more often, suggesting the success of publicly

funded prevention messages.

However, one of the results pushed hardest by the survey investigators was the finding that more

than 95 percent of South Africans wanted to see antiretroviral drugs made available to treat people

with AIDS and to prevent mother-to-child transmission of the virus. Six months later the South

African Cabinet reversed policy and decided to roll out an antiretroviral program. Clearly not the only

factor by a long shot, and coming after years of hard work by many different groups, the Mandela

survey has been publicly credited with playing a role in this important policy turnaround.

The study’s principal investigator was Dr. Olive Shisana, executive director of the HIV/AIDS

research program at the Human Sciences Research Council. For her there were two key reasons why

the survey evidence may have proved influential on policy: the involvement of Mandela, and the way

in which key political stakeholders were briefed before the results were made public in order to

ensure an informed response. “We allowed people to interrogate the results, before we published, so

they understood the implications of the research findings,” noted Shisana.

According to Shisana in her presentation to the Washington conference in September 2003, the

team from the Human Sciences Research Council briefed the South African president’s office and the

ministers of health, education, and science and technology, as well as officials from parliamentary

committees and national and provincial departments. On the day the study findings were released,

December 5, 2002, the South African government released a one-page, prewritten statement

welcoming the survey and noting that the public expected more resources to be spent combating the

epidemic. Because they were prepared beforehand, “the government responded directly to the

findings,” Shisana says, “and the messages did not get lost in controversy.”

Shisana is the former director general of health in South Africa and is experienced in bridging

the gap between research and policy; thus, it is not so surprising that the survey had an impact. A

S O U T H A F R I C A :

A I D S S U R V E Y H I T S H O M E R U N

16AcademyHealth

week after the report’s release the nation’s deputy president was referring enthusiastically to the

findings about changes in sexual attitudes and condom use. Three months later in Parliament the

minister for education pointed to the survey finding that more women were infected with HIV than

men (17.7 percent versus 12.8 percent) and described renewed efforts to target educational prevention

messages at girls.

Most important, according to a report in the South African daily the Mail and Guardian, the

survey findings (along with the lobbying efforts of Shisana, Mandela, and others targeting government

and party officials) played a role in the “Cabinet U-turn on antiretroviral drug treatment for

HIV/AIDS.” According to the newspaper, “In a dramatic reversal after years of dithering . . . [the]

government decided to ask the Department of Health ‘as a matter of urgency’ to develop a detailed

operational plan on an antiretroviral treatment programme. Its decision was welcomed by the United

Nations last week.” Shisana adds that pressure from groups such as the Treatment Action Campaign,

the Congress of South African Trade Union Organisations, churches, and many individuals also

helped force the policy turnaround.

In November 2003 the South African Cabinet finally approved an operational plan for

comprehensive treatment and care, essentially rolling out a national antiretroviral program with

estimates that 50,000 people could be treated in the first year. At present, newspaper reports suggest

that major delays have occurred in that planned rollout.

Insights: Influential and respected people can enhance impact by disseminating research findings to key

decision makers before publication to prevent defensive reactions and encourage use of evidence, and by

combining bad news about the extent of problems with good news about potential solutions, which may

increase decision makers’ willingness to listen.


Summary: Evidence from health services researchers has played a role in Thailand’s recent move to

universal coverage. A senior government decision maker talks candidly about the interdependence between

research and policymaking.

While many health services researchers and policymakers in the United States continue to talk about

the benefits of universal coverage, their contemporaries in Thailand have gone ahead and helped make

it happen. When a new national government was elected in 2001 with a commitment to universal

coverage, less than 70 percent of the Thai population had health insurance. Three years later, 95

percent have coverage, and the remaining 3 million people as yet uncovered have the right to it.

The new system offers a comprehensive benefit package covering acute, chronic, and preventive

services. It is financed mainly through taxation though supplemented with a co-payment of 30 baht

(about 75 cents) per medical visit from which the poor, children, and elderly are exempt. Health

services research in Thailand and those who produce it have been key to both the development and

implementation of the new system, known popularly as the “30 Baht Scheme.”

According to Dr. Siriwan Pitayarangsarit, who recently completed her doctoral thesis on the

system, the concept of universal coverage was first communicated to the (ultimately successful) Thai

Rak Thai political party by researchers during the election campaign of 2000. This happened first

through personal contacts and then via a brief paper from a group of researchers within the Ministry of

Public Health. Essentially a synthesis of international and national studies that discussed the feasibility

of universal coverage, the paper was later disseminated in the form of a concise 24-page booklet to help

build support for the reforms among political parties and nongovernmental organizations.

“There is no better time for health services researchers and social movements than the

preelection period,” says Suwit Wilbulproprasert, the senior Thai civil servant with direct

responsibility for financial management of the new system and its evidence-based implementation. For

Wilbulproprasert, while the brief 2000 paper on universal health care was significant, the more

important context was the web of personal relationships between members of the health policy “elite”

and members of the aspirant political party—relationships that date back three decades to days of

shared student activism.

Following the election of the new government in 2001, research to help implement universal

coverage was done both inside and outside the Health Ministry, with members of the academy

recruited to support the internal policy processes. Outside research comes primarily from the Health

Systems Research Institute, an independent public research management agency.

As a priority, research was required to work out exactly who was eligible for the 30 Baht Scheme

(which complements two existing insurance programs in Thailand) and to determine the per capita

cost to government of the new system. More broadly, researchers began evaluating different aspects of

the 30 Baht Scheme and feeding back results to facilitate further adjustments and evaluation. “We do

not separate the research arm and the policy arm—we work closely together,” says Wilbulproprasert,

describing a blurring of the role of researcher and policymaker: “Many of our policymakers are not

T H A I L A N D : E V I D E N C E F O R A C T I O N

O N U N I V E R S A L C O V E R A G E

18AcademyHealth

totally nonacademic, but most of the research they do is more practical, more ready to use. On the

other hand, many academic researchers also become policy advocates at the same time—not all of

them—but the ones that do, produce research that is more effective. I am not requesting [that] every

researcher becomes a policy advocate, but if you want to change things you have to be a policy

advocate as well, and there will be a higher chance the knowledge you generate will be used.”

Speaking during a plenary session of the Washington conference about the role of research and

researchers in Thailand’s move to universal coverage, Wilbulproprasert described the linkage between

social movements, knowledge generation, and political activity as the “triangle that moves the

mountain.” According to Wilbulproprasert, the comments reference both a widely known Chinese

story about how to move a mountain that stands in the way of social prosperity, and the work of a Thai

senior citizen, Professor Prawase Wasi. Urging researchers and policymakers alike to work with wider

social movements and the media to promote health reforms, Wilbulproprasert told the conference:

“Research should not float in the air. We need to link the sky and the earth.”

Taking a different view, the chair of that same session, Dr. Kenneth Shine, then with the RAND

Corporation, told conference participants, “To the extent you are a social advocate, you’re not a

scientist, you lose your credibility.” Asked about Shine’s comments, Wilbulproprasert said: “I agree

with him. If you are too close to a political party and let down the scientific evidence, you may lose

credibility. But you can definitely maintain your social credibility if you base your political advocacy

on scientific evidence.” Shine responds that good research can be used for both education and

informing political advocacy, but the roles should not be too blurred: “It is very important that when

researchers report their work, they do so in a form that is scientifically valid and do not include a

strong advocacy position.”

Like any major reforms, the move toward universal care in Thailand has not been smooth, with,

for example, important academic disagreements over the size of the co-payment and the per capita

cost of the 30 Baht Scheme to government. Moreover, the expansion of coverage has been tied to a

wider expansion and reform of the entire health system. In practice, more of the additional resources

are flowing to poorer provinces and less to wealthy provinces in an attempt to correct traditional

inequities. This, in turn, is causing much controversy. But according to Wilbulproprasert, the

redistribution of the new resources is also being driven by a large amount of evidence about the

previous maldistribution of facilities and human resources.

Insights: Preelection periods provide good opportunities for health services researchers; people who mix

policymaking and research can produce more effective research that is more ready to use.


Summary: During the 1990s the Veterans Health Administration reengineered its system to embed best

evidence into routine practice. Evaluation of the reengineering suggests improvements in the quality of

health care. Independent observers praise the strategy as an example of embedding best evidence in

systems of care.

When it comes to talk of using evidence in practice, the folks at the Veterans Health Administration

like to describe themselves as a “measurement laboratory for the nation.” With 75,000 clinicians

working under one roof servicing almost 7 million Americans, it is certainly a very large lab.

In the mid-1990s the Veterans Health Administration embarked on a major “reengineering” of its

health system to try and improve the use of information technology, the measuring and reporting of

performance, the quality of care, and health outcomes. According to a study published in the New

England Journal of Medicine (NEJM) in 2003, that reengineering “appears to have resulted in dramatic

improvements in the quality of care provided to veterans.”

Comparing data between 1994 and 2000, the study reported statistically significant improvements

in the quality of preventive, acute, and chronic care across nine indicators used in all years. By 2000

the proportion of patients receiving appropriate care was 90 percent or greater for nine of 17 selected

quality-of-care indicators. The proportion of women screened for cervical cancer jumped from 62

percent to 93 percent; the percentage of patients offered a beta-blocker after acute myocardial

infarction rose from 70 percent to 95 percent.

“We wanted to translate the best evidence to the bedside,” says Jonathan Perlin, then deputy

under secretary for health (now acting under secretary for health) at the Department of Veterans

Affairs (VA) and one of the NEJM study authors. “We’ve had remarkable progress.”

One key part of the reform process is the Veterans Health Administration Quality

Enhancement Research Initiative, a national system for the translation of research into practice,

with a mission “to enhance the quality and outcomes of VA health care by systematically

implementing clinical research findings and evidence-based recommendations into routine clinical

practice.” As part of this initiative, multidisciplinary, condition-specific teams gather evidence on

best practice, translate that evidence into routine care processes, and then measure the impact of

those translation interventions.

Internal data (publicly available on the Web) suggest that the translation efforts may have

produced real improvements in outcomes: The substance abuse team has helped increase the number

of opioid-dependent veterans with access to methadone treatment; the heart disease team has helped

reduce the number of veterans suffering coronary events; and the mental health team has reduced the

number of patients on nonguideline doses of antipsychotics.

Director of that initiative John Demakis says the organization is keen to share what it is learning

about the “science of implementation.” At the Washington conference he outlined several important

lessons, including winning support from local opinion leaders and hospital leadership, building

performance measures into executive contracts, customizing interventions to local contexts, using

U N I T E D S T A T E S :

B E S T E V I D E N C E T O T H E B E D S I D E

20AcademyHealth

good evidence to determine best practice, evaluating as you go, responding to barriers quickly,

changing the perception of research to being integrated with practice rather than outside practice,

and participating in demonstration projects, which can evolve into routine processes.

For Jonathan Lomas, executive director of the Canadian Health Services Research Foundation,

the Veterans Health Administration reforms serve as a powerful example. Here is a large frontline

delivery organization, Lomas wrote in a recent BMJ editorial, “using health services research to

respond to the direct needs of their managers and clinicians for better information on which to

base their decisions.” Rather than handing over research to experts in universities, says Lomas, the

research agenda is set by the organization’s own needs; the research is done collaboratively by

managers, clinicians, and researchers; and the results find their way directly into practice through

practice guidelines, computer reminders, test ordering systems, disease management teams, and so

on. “For too long, implementation of health services research has been viewed as a technical

exercise in better dissemination; now is the time to highlight the importance of inter-personal links

and the need to embed exchange between applied research and practice within health service

delivery organizations.”

One of the most important limitations of this evidence of quality improvement is that the results

come from observational studies, so there is only limited confidence that the improvements are caused

by the interventions. Ironically, even with that important limitation, the evidence of quality

improvement offered here might be the strongest of any within these nine case studies of using

evidence in practice.

Another consideration is that, notwithstanding important exceptions, the nature of the quality

indicators selected within the Veterans Health Administration generally tends to reward doing more

rather than less, at a time when some evidence is suggesting much systemwide excess. And there

remains uncertainty about the meaningful benefits of even well-established procedures, including

mammography screening—one of the quality measures used by Veterans Affairs.

Asked about these issues, Perlin argued that his organization was “tackling misuse and underuse

first, because unlike other parts of the health sector, in the VA there are no incentives that reward

overutilization.” He also explained that rates for prostate-specific antigen (PSA) screening were not

measured because of scientific uncertainty surrounding its benefits, whereas rates of counseling about

PSA tests were measured. Similarly, he described controversial discussions within the organization

about the targets for controlling lipid levels. In this case, professional cardiology groups were

suggesting more aggressive reductions than another camp within the organization that he described as

the “evidence jocks.”

Reflecting on the move to better use evidence within his organization, Perlin said that because

the viability of Veterans Affairs was in question in the early 1990s, discomfort with change was less

uncomfortable than the status quo. As part of that change, he says, quality was defined in patient-

centric terms (access, satisfaction, and functional status), research was used to outline evidence-based

approaches, and electronic records were used to provide timely evidence and to generate data for


measurement against prespecified benchmarks. “VA measures not to record the past,” he says, “but to

define future performance.”

Insights: Win support for change from leadership; customize interventions to local contexts; evaluate as you

go; respond to barriers quickly; and change the perception of research to being integrated with practice

rather than outside practice.

22AcademyHealth

Summary: The World Health Organization is providing a new service—the recently created Health

Evidence Network—that asks policymakers what policy questions they have, then summarizes the best

evidence to give them brief and timely answers.

Imagine a scenario where policymakers could regularly ask questions about how to solve their

nation’s most pressing health problems—and get prompt answers from a credible global group

summarizing the best available scientific evidence. Now imagine that the service was available to

policymakers in 50 nations and that answers sometimes came back in a matter of weeks, with the

world’s best scientific evidence summarized in a single paragraph.

Based in Denmark, the World Health Organization’s new Health Evidence Network is aiming to

offer just such a service. “What we are doing is very different from what most people producing

evidence are doing,” says an enthusiastic Egon Jonsson, leader of the network since its creation in

early 2003. “Most researchers are satisfied with publishing,” he says. “Whether everything goes down

a big black hole or not, most researchers don’t care because what counts is publishing. We go to the

policymakers first and ask them what questions they have—then we go to the production plants and see

what evidence can be found.”

Already the outfit has commissioned and produced more than a dozen “synthesis reports”

answering such broad policy-relevant questions as “What are the best strategies for ensuring quality in

hospitals?” The report about this question has been downloaded from the network’s Web site more

than 9,000 times.

Typically, a question arrives from a health department or ministry, an independent expert is

commissioned to review all the relevant evidence systematically, and a ten-page report and one-page

summary are produced as quickly as possible. “When we met with a hundred policymakers during the

pilot phase, they said it’s fine, it’s wonderful that you have a one-page summary, but can’t you make it

shorter?” Jonsson laughs into the phone from his Copenhagen office. “We are seriously considering

having only a paragraph.”

A professor at the Karolinska Institute in Stockholm and a former head of the Swedish

government agency responsible for assessing health care interventions, Jonsson concedes that with

such brevity, some complexity and subtlety can be lost, but he says that researchers dealing with the

policy process routinely have to boil down their work. “I know myself from experience in Sweden, the

minister may only have eight minutes to hear about a huge 800-page assessment that I have been

working with for three years.”

Based in Europe, the service is currently available to more than 50 nations, though the questions

received are often global in relevance. For example, the Belgian Ministry of Health recently asked, “Is

there any evidence to support various suicide-prevention strategies?” Keen to get answers, the

ministry asked to see the draft report, which examined evidence from the evaluation of 30 different

suicide-prevention strategies. While no single strategy appeared to be effective, certain prevention

programs for at-risk populations appeared promising.

T H E W O R L D H E A L T H O R G A N I Z A T I O N ’ S

H E A L T H E V I D E N C E N E T W O R K


The initiative in some ways complements the processes of the international Cochrane

Collaboration and other similar groups, where researchers independently drive the agenda for

producing systematic reviews of the evidence about health care interventions. As with other forms of

health technology assessments, the Health Evidence Network synthesis reports will, when appropriate,

draw on existing systematic reviews.

For Jonsson, there is no danger in policymakers having more influence over the research agenda,

because, he argues, they currently have so little: “If people make more evidence-based decisions, it is

better for patients, it’s better for financing, it’s better for access, and it’s better for quality of care.”

Yet the network is still in its early days, and questions remain about the applicability of global

evidence to national questions, about the portability of the synthesis reports across different national

settings, and about how these reports might be maintained or updated.

Insights: Asking policymakers what they want may increase the relevance and impact of research; short

summaries of one page and even one paragraph are desirable.

24AcademyHealth

Several common themes emerge from an informal synthesis of the insights offered by this diverse

group of case studies of using evidence in practice. They are summarized here as being

communication, integration, and evaluation.

C O M M U N I C A T I O N

Several different modes of communication emerge as being important to translating evidence into

practice. First, for those advocating better use of evidence, communicating with organizational

management or system leadership is deemed highly valuable, in order to secure support,

ownership, and “buy-in” for change. This was a key lesson learned within the U.S. Veterans Health

Administration transformation and is stated as an important precondition for change within the

Canadian nursing case study. In the South African example, communicating controversial research

findings to key decision makers before publication was credited with enhancing research impact

and utilization.

Second, in a broader sense, better communication between those who produce and those who

use research evidence for decision making is widely regarded as vital. In Thailand’s move toward

universal health insurance coverage and in Chile’s move to reduce health inequities, close working

relationships between researchers and decision makers were seen as increasing the effectiveness of

research and improving policy processes. In both situations, however, that closeness raises questions

about a blurring of roles that some regard as detracting from dispassionate scientific autonomy. In

the case of the World Health Organization’s new Health Evidence Network, it is as yet too early to

assess the success of innovative research processes initiated by specific communications from senior

decision makers.

Third, some observers see great value in better communication between not only those involved

in producing and using research, but also communication between researchers, decision makers, and

the wider set of stakeholders affected by the use of that research. For the managers of the National

Institute for Clinical Excellence in England and Wales, consultation with a wide range of stakeholders,

including physicians, patients, and product manufacturers, during the technology assessment process

is seen as highly desirable. Others stress that such consultation needs to be rigorously structured,

transparent, and comprehensive if it is to inform rather than confuse or undermine evidence-based

decision making.

I N T E G R A T I O N

Integration of research, research utilization, and practice is a key goal within health care

organizations and systems, though it is rarely achieved. Within this group of case studies, the

Bangladesh Rural Advancement Committee, the U.S. Veterans Health Administration, and the health

authorities in the Tver Oblast region of Russia have all attempted such integration, though the

limitations of evaluation procedures leave uncertainty about their success. Nevertheless, those

promoting evidence-based decision making see embedding “exchange between applied research and

practice within health service delivery organizations” as an important path to improving health.

C O M M O N T H E M E S : C O M M U N I C A T I O N ,

I N T E G R A T I O N , E V A L U A T I O N


E V A L U A T I O N

Evaluation of efforts to better use evidence is crucial, yet often poorly done, reducing confidence that

human health is actually being improved in the process. In none of these case studies is there strong,

high-quality evidence from evaluation studies proving that using research in practice has improved

health outcomes. In Bangladesh there is limited “surrogate outcome” evidence that mothers retained

knowledge of simple effective solutions for a common health problem. In Russia there is before-and-

after data suggesting improvements in neonatal health from system reform, though as yet no published

data comparing the target areas with “control” areas. Similarly, in the United States, the large body of

evaluative data from the Veterans Health Administration comes from observational studies.

Ironically, while evaluation is universally regarded as an integral part of any attempt to better

translate evidence into practice, the degree of rigor applied to the evaluation process does not yet

match the enthusiasm of its proponents.

26AcademyHealth

I N T R O D U C T I O N

2 “One way to fight malaria . . . health of millions can be improved: Global Health

Council 2003.

2 “Health decision makers must be at the forefront: Global Health Council 2004.

2 In the Lancet in 2003, an international working group on child mortality declared: Bellagio Study

Group on Child Survival 2003.

2 The most recent one, held in Washington, D.C.: Many of the presentations are available at

http://www.icsbhs.org under the “Agenda” section.

2 As the move toward an evidence-based approach to health care has gathered: Observations based in

part on author’s interview with Jonathan Lomas, executive director, Canadian Health Services

Research Foundation, 2003.

2 “It’s a community, a trailer park of different disciplines: Author’s interview with Don Berwick, chief

executive officer, Institute for Healthcare Improvement, 2003.

2 The nonprofit Institute for Healthcare Improvement is: See http://www.ihi.org/collaboratives/

(accessed May 10, 2004). See also Deyo et al. 2000; Headrick 2000.

3 According to a recent systematic review of the literature: Innvaer et al. 2002.

3 Another important question concerns what is meant: Moynihan 2004.

3 One of the basic tenets of the debate: Innvaer et al. 2002.

3 While this strategy is widely accepted and highly valued: Gerlier-Forest 2003.

3 At the Washington conference plenary session: Author’s notes of conference discussions.

4 A final tension worth noting is: Berwick 2003a.

4 Others, including University of Melbourne researcher Lenore Manderson: Manderson 2003.

B A N G L A D E S H : N A R R O W I N G T H E G A P

5 Mixing water with sugar and salt to fight deadly diarrhea: Water with Sugar and Salt (editorial) 1978.

5 This simple, inexpensive solution: Chowdhury et al. 1997.

5 Current estimates suggest that more than a million children: Parashar, Bresee, and Glass 2003.

5 In the late 1970s in Bangladesh: Chowdhury et al. 1997.

5 Speaking about what he saw as the success of the program: Chowdhury 2003.

5 Demonstrating the importance of an “organizational commitment: Interview with Mushtaque

Chowdhury, visiting professor, Columbia University, 2003.

5 An important modification introduced early: Ibid.

5 A study published in Health Policy and Planning found: Chowdhury et al. 1997.

6 As with any technology, there is uncertainty and controversy: Ibid.

6 There is also an important debate about the role of food: Almroth and Latham 1995.

p.

N O T E S


6 Notwithstanding that debate, the Bangladesh Rural Advancement Committee: Black 1985.

6 For Chowdhury, the experience also stands out: Chowdhury 2003.

C A N A D A : R E C O N N E C T I N G N U R S E S A N D P A T I E N T S

7 Addressing the nursing crisis in the health systems of many nations: Berwick 2003b.

7 In Canada, federal government figures show: Office of Nursing Policy, Health Canada 2004.

7 Alongside the growing evidence of absenteeism: O’Brien-Pallas 2003.

7 At a conference session in Washington devoted entirely to nursing: Allison 2003. See also Person

et al. 2004.

7 Similarly, patients undergoing stomach surgery were: Dang 2003. See also Dang et al. 2002.

7 In Canada a concerted effort is being made: Author’s interview with Mary Ferguson-Paré, chief

nursing executive, University Health Network, 2004 (hereafter Ferguson-Paré interview).

7 The first of the projects involves three nursing units: Ibid.

7 The second of the projects involves a controlled study: Ibid.

8 An important part of the context for these projects is the wider work: O’Brien-Pallas 2003.

8 One of the desired outcomes from the current demonstration projects is: Ferguson-Paré interview.

C H I L E : E V I D E N C E F O R E Q U I T Y

9 “Scientific knowledge has been one of the basic pillars: Author’s interview with Hernan Sandoval,

executive secretary, National Committee for Health Reform, 2004 (hereafter Sandoval interview).

9 In recent decades Chile has experienced: Vega et al. 2002.

9 Likewise, infant mortality is more than ten times higher: Sandoval interview.

9 Such evidence about health inequity is having an impact:

http://www.gega.org.za/profiles/chile.php (accessed February 4, 2004).

9 The team in Chile is part of an international network: Vega et al. 2002.

9 The work of the Equity Gauge includes: Pittman 2003.

9 As part of its work, the group has produced: http://www.gega.org.za/profiles/chile.php (accessed

February 4, 2004).

9 “The combination of advocacy and knowledge is: Sandoval interview.

9 A key achievement of the Equity Gauge was: Vega and Sandoval 2003.

9 Asked whether individual pieces of research evidence had: Sandoval interview.

9 One practical strategy for reducing health inequity being used: Ibid.

10 Reformers such as Sandoval who believe social determinants play: Ibid.

10 A senior health aide to President Salvador Allende in the early 1970s: Ibid.

28AcademyHealth

E N G L A N D A N D W A L E S : E V I D E N C E I S N O T E N O U G H

11 The National Institute for Clinical Excellence (NICE) has emerged:

http://www.nice.org.uk/cat.asp?c=57705 (accessed May 10, 2004).

11 Yet as senior executives from the institute wrote in Lancet Oncology in 2003: Littlejohns, Barnett,

and Longson 2003.

11 The official “guidance” NICE recently produced:

http://www.nice.org.uk/pdf/TA70_Imatinib_fullguidance.pdf (accessed May 10, 2004).

11 As the NICE executives candidly elaborated: Littlejohns, Barnett, and Longson 2003.

11 Appraising the effectiveness and cost-effectiveness: Author’s interview with Peter Littlejohns, clinical

director, National Institute for Clinical Excellence, 2004 (hereafter Littlejohns interview).

11 With imatinib, members of the NICE appraisal committee were forced: Littlejohns, Barnett, and

Longson 2003.

11 After releasing a very negative preliminary finding:


12 “There is no algorithm for this decision making: Littlejohns interview.

12 “There will always be a tension between the center and the periphery: Littlejohns interview.

R U S S I A : E V I D E N C E - F R I E N D L Y S Y S T E M S AV E S L I V E S

13 In Tver Oblast in the Russian Federation in late 1998: The USA-Russia Joint Commission on

Economic and Technological Cooperation 2001.

13 Baseline data taken before any changes were implemented suggested: Massoud 2003.

13 The new system was rolled out toward the end of 1999: Ibid.

13 “What we we’re trying to do is implement evidence: Interview with Rashad Massoud, director,

Quality Performance Institute, 2003 (hereafter Massoud interview).

13 What was first tried on a small scale in five hospitals has: Massoud 2003.

14 Asked about the lack of comparable data from a “control” region: Massoud interview.

S O U T H A F R I C A : A I D S S U R V E Y H I T S H O M E R U N

15 It’s not every day a health survey makes world headlines: Human Sciences Research Council 2002.

15 At the time of the survey’s release: Ibid.

15 However, one of the results pushed hardest by the survey investigators was: AIDS: Ministers Revolt 2003.

15 The study’s principal investigator was Dr. Olive Shisana: Author’s interview with Olive Shisana,

executive director, Human Sciences Research Council, 2003 (hereafter Shisana interview).

15 According to Shisana in her presentation to the Washington conference: Shisana 2003.


15 On the day the study findings were released: http://www.doh.gov.za/docs/pr/2002/pr1205.html

(accessed May 10, 2004).

15 Because they were prepared beforehand: Shisana interview.

16 Most important, according to a report in the South African daily: AIDS: Ministers Revolt 2003.

16 Shisana adds that pressure from groups such as the Treatment Action Campaign: Shisana interview.

16 In November 2003 the South African Cabinet finally approved: See

http://www.aegis.com/news/re/2003/RE031116.html (accessed May 10, 2004).

16 At present, newspaper reports suggest that major delays have occurred: LaFraniere 2004.

T H A I L A N D : E V I D E N C E F O R A C T I O N O N U N I V E R S A L C O V E R A G E

17 While many health services researchers and policymakers in the United States continue: Author’s

interview with Suwit Wilbulproprasert, deputy permanent secretary, Ministry of Public Health,

2004 (hereafter Wilbulproprasert interview).

17 The new system offers a comprehensive benefit package: Pitayarangsarit 2003.

17 According to Dr. Siriwan Pitayarangsarit, who recently completed: Ibid.

17 “There is no better time for health services researchers and social movements: Suwit

Wilbulproprasert’s comments at Washington conference plenary session, in possession of author.

17 As a priority, research was required to work out exactly who was eligible: Wilbulproprasert interview.

18 Speaking during a plenary session of the Washington conference: Suwit Wilbulproprasert’s

comments at Washington conference plenary session, in possession of author.

18 Taking a different view, the chair of that same session, Dr. Kenneth Shine: Kenneth Shine’s

comments at Washington conference plenary session, in possession of author.

18 Asked about Shine’s comments, Wilbulproprasert said: Wilbulproprasert interview.

18 Shine responds that good research can be used for both education and informing political advocacy:

Author’s interview with Ken Shine, executive vice chancellor for health affairs, University

of Texas, 2004.

18 But according to Wilbulproprasert, the redistribution of the new resources is: Wilbulproprasert interview.

U N I T E D S T A T E S : B E S T E V I D E N C E T O T H E B E D S I D E

19 When it comes to talk of using evidence in practice: Personal communication with Jonathan Perlin,

deputy under secretary for health, Department of Veterans Affairs, 2004.

19 In the mid-1990s the Veterans Health Administration embarked: Jha et al. 2003.

19 Comparing data between 1994 and 2000, the study reported: Ibid.

19 “We wanted to translate the best evidence to the bedside: Author’s interview with Jonathan Perlin,

2004 (hereafter Perlin interview).

30AcademyHealth

19 One key part of the reform process is the Veterans Health Administration: For more details on this

program, see http://www1.va.gov/hsrd/publications/ (accessed May 27, 2004).

19 Internal data (publicly available on the Web) suggest: See

http://www1.va.gov/resdev/prt/QUERIQuarterly_VO5_NO1.pdf (accessed May 10, 2004).

19 Director of that initiative John Demakis says: Demakis 2003.

20 For Jonathan Lomas, executive director of the Canadian Health Services Research Foundation:

Lomas 2003.

20 One of the most important limitations of this evidence of quality improvement is: Jha et al. 2003.

20 Another consideration is that, notwithstanding important exceptions: Fisher et al. 2003a, b.

20 And there remains uncertainty about the meaningful benefits: Olsen and Gøtzsche 2001.

20 Asked about these issues, Perlin argued: Perlin interview.

20 Reflecting on the move to better use evidence within his organization: Ibid.

T H E W O R L D H E A L T H O R G A N I Z A T I O N ’ S H E A L T H E V I D E N C E N E T W O R K

22 Based in Denmark, the World Health Organization’s new Health Evidence Network is: See

http://www.who.dk/HEN/20030703_1 (accessed January 14, 2004).

22 “What we are doing is very different from what most people producing evidence are doing: Author’s

interview with Egon Jonsson, leader, Health Evidence Network, 2004 (hereafter Jonsson interview).

22 “When we met with a hundred policymakers during the pilot phase, they said: Ibid.

22 A professor at the Karolinska Institute in Stockholm and a former head: Ibid.

23 The initiative in some ways complements the processes: See http://www.cochrane.org


23 For Jonsson, there is no danger in policymakers having more influence: Jonsson interview.

C O M M O N T H E M E S : C O M M U N I C A T I O N , I N T E G R A T I O N , E V A L U A T I O N

24 Third, some observers see great value in better communication: E-mail communication from

Andrew Oxman, director, Department of Health Services Research, Norwegian Directorate for

Health and Social Welfare, 2004.

24 Nevertheless, those promoting evidence-based decision making see: Lomas 2003.


AIDS: Ministers Revolt. 2003. Mail and Guardian (Johannesburg, South Africa), August 15.

Allison, J. 2003. Presentation at Washington conference. Available at

http://www.icsbhs.org/presentations/allison.pdf (accessed May 10, 2004).

Almroth, S., and M.C. Latham. 1995. Rational Home Management of Diarrhoea. Lancet 345:709–11.

Bellagio Study Group on Child Survival. 2003. Knowledge into Action for Child Survival. Lancet

362:323–7.

Berwick, D. 2003a. Disseminating Innovations in Health Care. Journal of the American Medical

Association 289:1969–75.

Berwick, D. 2003b. Presentation at Washington conference. Available at

http://www.icsbhs.org/presentations/berwick.pdf (accessed May 10, 2004).

Black, R.E. 1985. Communication for Improved Health Services. Development Communication Report

51:1–2.

Chowdhury, M. 2003. Presentation at Washington conference. Available at

http://www.icsbhs.org/presentations/chowdhury.pdf (accessed May 10, 2004).

Chowdhury, M., F. Karim, S. Sarkar, R. Cash, and A. Bhuiya. 1997. The Status of ORT in Bangladesh:

How Widely Is It Used? Health Policy and Planning 12(1):58–66. Available at

http://heapol.oupjournals.org/cgi/reprint/12/1/58.pdf (accessed May 10, 2004).

Dang, D. 2003. Presentation at Washington conference. Available at

http://www.icsbhs.org/presentations/dang.pdf (accessed May 10, 2004).

Dang, D., M.E. Johantgen, P.J. Pronovost, M.W. Jenckes, and E.B. Bass. 2002. Postoperative

Complications: Does Intensive Care Unit Staff Nursing Make a Difference? Heart and Lung: The

Journal of Critical Care 31:219–28.

Demakis, J. 2003. Presentation at Washington conference. Available at

http://www.icsbhs.org/presentations/demakis.pdf (accessed May 10, 2004).

Deyo, R., M. Schall, D. Berwick, T. Nolan, and P. Carver. 2000. Innovations in Education and Clinical

Practice: Continuous Quality Improvement for Patients with Back Pain. Journal of General Internal

Medicine 15:647–55.

R E F E R E N C E S

32AcademyHealth

Fisher, E.S., D.E. Wennberg, T.A. Stukel, D.J. Gottlieb, F.L. Lucas, and E.L. Pinder. 2003a. The

Implications of Regional Variations in Medicare Spending, Part 1, The Content, Quality, and

Accessibility of Care. Annals of Internal Medicine 138:273–87.

Fisher, E.S., D.E. Wennberg, T.A. Stukel, D.J. Gottlieb, F.L. Lucas, and E.L. Pinder. 2003b. The

Implications of Regional Variations in Medicare Spending, Part 2, Health Outcomes and Satisfaction

with Care. Annals of Internal Medicine 138:288–98.

Gerlier-Forest, P. 2003. Presentation at Washington conference. Available at

http://www.icsbhs.org/presentations/forest.pdf (accessed May 10, 2004).

Global Health Council. 2003. Reducing Malaria’s Burden: Evidence of Effectiveness for Decision Makers,

Technical Report, 6 (December). Available at http://www.globalhealth.org/view_top.php3?id=384


Global Health Council. 2004. Reducing Malaria’s Burden: Evidence of Effectiveness for Decision Makers,

Technical Report (press release). Available at http://www.globalhealth.org/assets/press/pr-

aids_012104.pdf (accessed May 10, 2004).

Headrick, L. 2000. Commentary: Two Kinds of Knowledge to Achieve Better Care. Journal of General

Internal Medicine 15:675.

Human Sciences Research Council. 2002. The Impact of HIV/AIDS on the Health Sector: National

Survey of Health Personnel, Ambulatory and Hospitalised Patients and Health Facilities. Available at

http://www.hsrcpublishers.co.za/index.html?health_sector.html~content (accessed May 10, 2004).

Innvaer, S., V. Gunn, M. Trommald, and A. Oxman. 2002. Health Policy-Makers’ Perceptions of Their

Use of Evidence: A Systematic Review. Journal of Health Services Research and Policy 7:239–44.

Jha, A., J. Perlin, K. Kizer, and R. Dudley. 2003. Effect of the Transformation of the Veterans Affairs

Health Care System on the Quality of Care. New England Journal of Medicine 348:2218–27.

LaFraniere, S. 2004. South Africa Is Criticized for Delay in AIDS Treatment. New York Times,

February 20.

Littlejohns, P., D. Barnett, and C. Longson. 2003. The Cancer Technology Appraisal Programme of

the U.K.’s National Institute for Clinical Excellence. Lancet Oncology 4:242–50.


Lomas, J. 2003. Health Services Research (editorial). BMJ 327:1301–2.

Manderson, L. 2003. Presentation at Washington conference. Available at

http://www.icsbhs.org/presentations/manderson.pdf (accessed May 10, 2004).

Massoud, R. 2003. Presentation at Washington conference. Available at

http://www.icsbhs.org/presentations/massoud1.pdf (accessed May 10, 2004).

Moynihan, R. 2004. Evaluating Health Services: A Reporter Covers the Science of Research Synthesis.

New York: Milbank Memorial Fund.

O’Brien-Pallas, L. 2003. Presentation at Washington conference. Available at

http://www.icsbhs.org/presentations/obrienpallas.pdf (accessed May 10, 2004).

Office of Nursing Policy, Health Canada. 2004. Trends in Illness and Injury-Related Absenteeism and

Overtime among Publicly Employed Canadian Registered Nurses, 1987–2002.

Olsen, O., and P. Gøtzsche. 2001. Cochrane Review on Screening for Breast Cancer with

Mammography. Lancet 358:1340–2.

Parashar, U., J. Bresee, and R. Glass. 2003. The Global Burden of Diarrhoeal Disease in Children.

Bulletin of the World Health Organisation 81:236. Available at

http://www.who.int/bulletin/volumes/81/4/en/Editorial2.pdf (accessed May 10, 2004).

Person, S.D., J.J. Allison, C.I. Kiefe, M.T. Weaver, O.D. Williams, R.M. Centor, and N.W. Weissman.

2004. Nurse Staffing and Mortality for Medicare Patients with Acute Myocardial Infarction. Medical

Care 42:4–12.

Pitayarangsarit, S. 2003. The Introduction of the Universal Coverage of Health Care Policy in

Thailand: Policy Responses. Ph.D. thesis, Health Policy Unit, Department of Public Health and

Policy, London School of Hygiene and Tropical Medicine, University of London.

Pittman, P. 2003. Health Equity Research: Beyond the Sound of One Hand Clapping. International

Exchange for Health Services Research and Policy: Learning from Cross-Country Comparisons.

Available at http://www.academyhealth.org/2003/presentations/pittman2.pdf (accessed June 28, 2004).

Shisana, O. 2003. Presentation at Washington conference. Available at

http://www.icsbhs.org/presentations/shisana.pdf (accessed May 10, 2004).

34AcademyHealth

The USA-Russia Joint Commission on Economic and Technological Cooperation. 2001. (Re)Designing

the System of Care for Neonates Suffering from Respiratory Distress Syndrome. Tver Oblast. Available at

http://www.qaproject.org/pubs/PDFs/Russian/neonat501.pdf (accessed May 27, 2004).

Vega, J., L. Jadue, I. Delgado, R. Burgos, F. Brown, F. Marin, and V. Zuñiga. 2002. Disentangling the

Pathways to Health Inequities, Chilean Health Equity Gauge Report. Available at

http://www.paho.org/English/HDP/Equity-Chile.pdf (accessed June 28, 2004).

Vega, J., and H. Sandoval. 2003. Building the Bridge between Knowledge in Health Equity and

Practice: The Chilean National Objectives for the 2000–2010 Decade. Paper commissioned by

AcademyHealth for the meeting “Health Equity Research: Beyond the Sound of One Hand Clapping,”

Rockefeller Conference Center, Bellagio, Italy (April).

Water with Sugar and Salt (editorial). 1978. Lancet 2:300–1.

W E B S I T E S

http://www.aegis.com/news/re/2003/RE031116.html (accessed May 10, 2004).

http://www.cochrane.org (accessed May 10, 2004).

http://www.doh.gov.za/docs/pr/2002/pr1205.html (accessed May 10, 2004).

http://www.gega.org.za/profiles/chile.php (accessed February 4, 2004).

http://www.icsbhs.org (accessed May 10, 2004).

http://www.ihi.org/collaboratives/ (accessed May 10, 2004).

http://www.nice.org.uk/cat.asp?c=57705 (accessed May 10, 2004).


http://www.who.dk/HEN/20030703_1 (accessed January 14, 2004).

http://www1.va.gov/hsrd/publications/ (accessed May 27, 2004).

http://www1.va.gov/resdev/prt/QUERIQuarterly_VO5_NO1.pdf (accessed May 10, 2004).


Ray Moynihan is a Washington, D.C.–based reporter with an international reputation for quality

medical journalism. A long-time radio and television reporter with the Australian Broadcasting

Corporation, he has also published original studies, opinion pieces, and investigative reports in the

New England Journal of Medicine, Lancet, BMJ, and the Medical Journal of Australia. He has also

written previously for the Milbank Memorial Fund and is the author of Too Much Medicine? (ABC

Books, 1998). In 1999 he was a Harkness Fellow in Health Care Policy at Harvard University. His most

recent award is for outstanding journalism, given by the Medical Journalists’ Association in Britain in

2003. He is currently a visiting editor and contributor to BMJ. His television documentary and book,

Selling Sickness, will be available in 2005.

T H E A U T H O R

36AcademyHealth

A complete list of the Fund’s reports may be viewed online at www.milbank.org. Reports published

since 1993 are available electronically. Single or multiple copies of reports that have print editions are

available without charge while supplies last.

Emergency Preparedness, Bioterrorism, and the States: The First Two Years after September 11

by Gerald Markowitz and David Rosner

2004 92 pages

Evidence-Based Mental Health Treatments and Services: Examples to Inform Public Policy

by Anthony F. Lehman, Howard H. Goldman, Lisa B. Dixon, and Rachel Churchill

2004 44 pages

Addressing the HIV/AIDS Pandemic: A U.S. Global AIDS Strategy for the Long Term

co-published with the Council on Foreign Relations

2004 44 pages

Evaluating Health Services: A Reporter Covers the Science of Research Synthesis

by Ray Moynihan

2004 64 pages

Making Sense of the System: How States Can Use Health Workforce Policies to Increase Access and

Improve Quality of Care

by Edward Salsberg

co-published with the Reforming States Group

2003

Available electronically only at

http://www.milbank.org/reports/2003salsberg/2003salsberg.html

Food Safety Updated: Developing Tools for a More Science- and Risk-Based Approach

by Michael R. Taylor, Margaret O’K. Glavin, J. Glenn Morris, Jr., and Catherine E. Woteki

co-published with Resources for the Future

2003 56 pages

Implementing the Resident Assessment Instrument: Case Studies of Policymaking for Long-Term Care

in Eight Countries

2003


http://www.milbank.org/reports/interRAI/030222interRAI.html

S E L E C T E D P U B L I C A T I O N S O F T H E

M I L B A N K M E M O R I A L F U N D


September 11 and the Shifting Priorities of Public and Population Health in New York

by David Rosner and Gerald Markowitz

2003 68 pages

Rebalancing Long-Term Care in New Jersey: From Institutional toward Home and Community Care

by Susan C. Reinhard and Charles J. Fahey

2003


http://www.milbank.org/reports/030314newjersey/030314newjersey.html

2000-2001 State Health Care Expenditure Report

co-published with the National Association of State Budget Officers and the Reforming States Group

2003 92 pages

Proactive Hazard Analysis and Health Care Policy

by John E. McDonough

co-published with ECRI

2002 36 pages

Achieving Better Health Outcomes: The Oregon Benchmark Experience

by Howard M. Leichter and Jeffrey Tryens

2002 60 pages

2001 Robert H. Ebert Memorial Lecture: Health Care Quality and How to Achieve It

by Kenneth I. Shine

2002 24 pages

Informing Judgment: Case Studies of Health Policy and Research in Six Countries

co-published with the Cochrane Collaboration

2001 212 pages

1998-1999 State Health Care Expenditure Report

co-published with the National Association of State Budget Officers and the Reforming States Group

2001 204 pages

38AcademyHealth

C A L I F O R N I A / M I L B A N K B O O K S O N H E A L T H A N D T H E P U B L I C

The following books are co-published with and distributed by the University of California Press.

For information or to order, call 1-800-822-6657 or visit http://www.ucpress.edu.

The Employee Retirement Income Security Act of 1974: A Political History

by James A. Wooten

2005 (forthcoming) 400 pages

$65.00 cloth

Sick to Death and Not Going to Take It Anymore! Reforming Health Care for the Last Years of Life

by Joanne Lynn

2004 217 pages

$24.95 cloth

What Price Better Health? Hazards of the Research Imperative

by Daniel Callahan

2003 341 pages

$29.95 cloth

When Walking Fails: Mobility Problems of Adults with Chronic Conditions

by Lisa I. Iezzoni

2003 380 pages

$60.00 cloth; $19.95 paper

Death Is That Man Taking Names: Intersections of American Medicine, Law, and Culture

by Robert A. Burt

2002 256 pages

$29.95 cloth; $18.95 paper

Deceit and Denial: The Deadly Politics of Industrial Pollution

by Gerald Markowitz and David Rosner

2002 464 pages

$45.00 cloth; $19.95 paper

Big Doctoring in America: Profiles in Primary Care

by Fitzhugh Mullan

2002 302 pages

$29.95 cloth; $19.95 paper


Public Health Law and Ethics: A Reader

edited by Lawrence O. Gostin

2002 536 pages

$60.00 cloth; $35.00 paper

Public Health Law: Power, Duty, Restraint

by Lawrence O. Gostin

2000 518 pages

$29.95 paper

Experiencing Politics: A Legislator’s Stories of Government and Health Care

by John E. McDonough

2000 336 pages

$21.95 paper

The Corporate Practice of Medicine: Competition and Innovation in Health Care

by James C. Robinson

1999 306 pages

$21.95 paper

The Fund also publishes the Milbank Quarterly, a multidisciplinary journal of population health and

health policy. Information about subscribing to the Quarterly is available by calling toll-free

1-800-835-6770, or at www.milbank.org/quarterly/.

Information about other work of the Fund is available from the Fund at

645 Madison Ave., 15th Floor, New York, NY 10022, (212) 355-8400. Fax: (212) 355-8599.

E-mail: [email protected]. On the Web: www.milbank.org.

40AcademyHealth

A complete list of AcademyHealth’s reports can be viewed online at www.academyhealth.org.

State of the States 2004

January 2004

by Isabel Friedenzohn, Elizabeth Greenbaum, Madeleine Konig, Benjamin Wheatley, LeAnne

DeFrancesco, Jeremy Alberga, and Christina Folz

http://www.statecoverage.net/pdf/stateofstates2004.pdf

Health-Based Risk Assessment: Risk-Adjusted Payments and Beyond

January 2004

by Kathryn E. Martin, Sharon B. Arnold, and Deborah L. Rogal

http://www.hcfo.net/pdf/riskadjustment.pdf

Mapping State Health Insurance Markets, 2001: Structure and Change

September 2003

by Deborah Chollet, Fabrice Smieliauskas, and Madeleine Konig

http://www.statecoverage.net/pdf/mapping2001.pdf

Using Rural Health Networks to Address Local Needs: Five Case Studies

July 2003

by Ira Moscovice and Walter Elias

http://www.academyhealth.org/ruralhealth/casestudies.pdf

Financing End-of-Life Care: Challenges for an Aging Population

February 2003

by Bonnie Austin and Lisa Fleisher

http://www.hcfo.net/pdf/eolcare.pdf

State of the States 2003

January 2003

by Christina Folz, Isabel Friedenzohn, LeAnne DeFrancesco, Madeleine Konig, Lesly Hallman, and

Jeremy Alberga

http://www.statecoverage.net/pdf/stateofstates2003.pdf

For further information on AcademyHealth or this report, please write to AcademyHealth, 1801 K

Street, NW, #701-L, Washington, D.C. 20006, call at (202) 292-6700, fax (202) 292-6800, or visit the

organization’s Web site, www.academyhealth.org.

S E L E C T E D P U B L I C A T I O N S O F A C A D E M Y H E A L T H

Design and Typography:

The Boland Design Company

Printing:

Prestone Printing Company, Inc.

Using Health Research in Policy and Practice:

Documents

Transcript of Using Health Research in Policy and Practice: