Estimating the Global Health Impact of Estimating the Global Health Impact of Improved Diagnostic ToolsImproved Diagnostic Tools

Jeffrey WassermanJeffrey Wasserman

Federico GirosiFederico Girosi

Emmett KeelerEmmett Keeler

November 2006, April 2007November 2006, April 2007

2 3/2008

Introduction to Diagnostic Tools Project for BMGFIntroduction to Diagnostic Tools Project for BMGF

Better diagnosis might reduce the burden of disease Better diagnosis might reduce the burden of disease

Modeling the Benefits of New DiagnosticsModeling the Benefits of New Diagnostics

Basic approachBasic approach

Key issues and decisionsKey issues and decisions

FindingsFindings

Benefits of New Diagnostics for TuberculosisBenefits of New Diagnostics for Tuberculosis

Selected Findings from Other DiseasesSelected Findings from Other Diseases

OutlineOutline

3 3/2008

The Burden of Disease Remains Large in The Burden of Disease Remains Large in the Developing Worldthe Developing World

0 1,000,000 2,000,000 3,000,000

Malaria

Child diarrhea

Child respiratory

Tuberculosis

HIV/AIDS

* World Health Organization, 2003 (HIV) and 2002 (all others)* World Health Organization, 2003 (HIV) and 2002 (all others)

Annual Deaths Worldwide*Annual Deaths Worldwide*

4 3/2008

Diseases Are Concentrated in Some Diseases Are Concentrated in Some RegionsRegions

LatinLatinAmericaAmerica

AfricaAfrica

AsiaAsia

5 3/2008

Many Factors Might Contribute to Improved Many Factors Might Contribute to Improved Health Outcomes in the Developing WorldHealth Outcomes in the Developing World

GoodGoodHealthHealth

LifestyleLifestyleNutritionNutrition

Public HealthPublic HealthProgramsPrograms

Clinical CareClinical Care

DiagnosisDiagnosis TreatmentTreatment

6 3/2008

How Better Diagnostics Could HelpHow Better Diagnostics Could Help

More accurate tests could help target therapy to More accurate tests could help target therapy to those who need it and eliminate wasteful treatmentsthose who need it and eliminate wasteful treatments

Earlier diagnosis could allow therapy to start sooner, Earlier diagnosis could allow therapy to start sooner, reducing impact on the individual and reducing reducing impact on the individual and reducing spread of the diseasespread of the disease

Simpler, easy-to-use tests could help increase Simpler, easy-to-use tests could help increase access to diagnostics to those who currently have no access to diagnostics to those who currently have no carecare

7 3/2008

But Barriers Exist to Using Many Current But Barriers Exist to Using Many Current Diagnostic Tools in Resource-Poor CountriesDiagnostic Tools in Resource-Poor Countries

Tests often require Tests often require advanced infrastructureadvanced infrastructure

High-performing tests High-performing tests are expensiveare expensive

Some existing tests are Some existing tests are inadequate and slow inadequate and slow

Cultural and political Cultural and political considerations may considerations may impede acceptance impede acceptance

Needs may vary across Needs may vary across countriescountries

8 3/2008

We were asked two Key QuestionsWe were asked two Key Questions

What are the global health benefits of better clinical diagnostic tools?

What performance specifications and infrastructure requirements does a test need to achieve the estimated benefits?

9 3/2008

IntroductionIntroduction Better diagnosis might reduce the burden of Better diagnosis might reduce the burden of

disease disease

Modeling the Benefits of New DiagnosticsModeling the Benefits of New Diagnostics Basic approachBasic approach Key issues and decisionsKey issues and decisions

FindingsFindings Benefits of New Diagnostics for TuberculosisBenefits of New Diagnostics for Tuberculosis Selected Findings from Other DiseasesSelected Findings from Other Diseases

OutlineOutline

10 3/2008

We Worked with the Gates Foundation to We Worked with the Gates Foundation to Create a Global Health Diagnostics ForumCreate a Global Health Diagnostics Forum

MalariaMalaria TuberculosisTuberculosis

Acute LowerAcute LowerRespiratoryRespiratoryInfectionsInfections

HIV and HIV and Sexually-Sexually-

TransmittedTransmittedDiseasesDiseases

DiarrhealDiarrhealDiseasesDiseases

Forum Working GroupsForum Working Groups The Forum’s RoleThe Forum’s Role

Provide expertise on Provide expertise on relevant diseases, relevant diseases, diagnostic needs, emerging diagnostic needs, emerging technologiestechnologies

Identify key intervention Identify key intervention points for each diseasepoints for each disease

Refine the approach for Refine the approach for assessmentassessment

Serve as conduits to the Serve as conduits to the broader scientific broader scientific communitycommunity

11 3/2008

Status quo

Model for calculating benefits of new TestModel for calculating benefits of new Test

• Population characteristicsPopulation characteristics• Health statusHealth status• Access to diagnostics Access to diagnostics and treatment and treatment

HealthHealthoutcomesoutcomes

With new diagnostic test

HealthHealthoutcomesoutcomes

Difference in outcomes =

Gains from improved diagnostic tests

12 3/2008

Access to a New Diagnostic Depends on Access to a New Diagnostic Depends on the Level of Infrastructure Availablethe Level of Infrastructure Available

Advanced/Advanced/ModerateModerate

• Hospitals andHospitals and urban clinics urban clinics

• Electricity,Electricity, clean water, clean water, well-equipped well-equipped laboratories, laboratories, trained clinicians trained clinicians

MinimalMinimal

• Health clinics (Africa),Health clinics (Africa), rural clinics (Asia, rural clinics (Asia, Latin America) Latin America)

• No reliable electricityNo reliable electricity or clean water, no or clean water, no laboratory, minimal laboratory, minimal expertise expertise

NoneNone

• Village or Village or community community

• No electricity,No electricity, clean water, clean water, physical physical infrastructure, or infrastructure, or trained staff trained staff

13 3/2008

Characteristics of potential new testsCharacteristics of potential new tests

Accuracy = sensitivity, specificityAccuracy = sensitivity, specificity

AccessAccess

Time to diagnosis ( related to loss to follow-up)Time to diagnosis ( related to loss to follow-up)

Cost of equipment and operationCost of equipment and operation

Other (specimen type, multiplex diseases….)Other (specimen type, multiplex diseases….)

14 3/2008

Introduction to Diagnostic Tools Project for BMGFIntroduction to Diagnostic Tools Project for BMGF

Better diagnosis might reduce the burden of disease Better diagnosis might reduce the burden of disease

Global Health Diagnostics ForumGlobal Health Diagnostics Forum

Modeling the Benefits of New DiagnosticsModeling the Benefits of New Diagnostics

Basic approachBasic approach

Key issues and decisionsKey issues and decisions

FindingsFindings

Benefits of New Diagnostics for TuberculosisBenefits of New Diagnostics for Tuberculosis

Selected Findings from Other DiseasesSelected Findings from Other Diseases

OutlineOutline

15 3/2008

Key modeling issues and decisionsKey modeling issues and decisions

Get intervention points from experts or models?Get intervention points from experts or models?

Decision trees are good for studying diagnostic testsDecision trees are good for studying diagnostic tests

Static vs. Epidemic Models, future trendsStatic vs. Epidemic Models, future trends

What about costs, especially of over-treatment?What about costs, especially of over-treatment?

Potential vs probable access, diffusion Potential vs probable access, diffusion

Where can we get data to populate the models?Where can we get data to populate the models?

16 3/2008

Static or dynamic modelsStatic or dynamic models

For TB and HIV, state of the art is dynamic models: people flow For TB and HIV, state of the art is dynamic models: people flow between states according to diff. equations, researchers calculate between states according to diff. equations, researchers calculate long-run policy impacts.long-run policy impacts.

states: no disease, treated early dis., untreat dis.,…states: no disease, treated early dis., untreat dis.,…

decision trees give some parameters for models decision trees give some parameters for models

An alternative is static decision trees: calculate costs and effects in An alternative is static decision trees: calculate costs and effects in one year, with incidence assumed unaffected by new DX tool.one year, with incidence assumed unaffected by new DX tool.

long run effects ~ proportional to one year effectslong run effects ~ proportional to one year effects

our main interest is comparative performanceour main interest is comparative performance

transmission handled by multiplierstransmission handled by multipliers

Static model can be used for all five diseasesStatic model can be used for all five diseases

Simpler method allows us to do project with limited time and money. Simpler method allows us to do project with limited time and money.

Susceptiblepeople

Each case infects kn-1 others

Sn-1 Sn

The difference equations Sn = Function(Sn-1, other “n-1” conditions)

show how the system evolves over the generations.

EpidemicsEpidemics

1

2

0

If k = 2

Let Sn be the number of cases in generation n

18 3/2008

A dynamic model:A dynamic model:Diseases in Equilibrium (TB)Diseases in Equilibrium (TB)

Active TB TB Death

curesHealthy

Latent TB

Each active case “infects” about 20 people on average.- they then have latent TB, but a few progress

Some active cases are cured, some dieA generation is ~ 4 years from activation to resolutionIf Disease is in equilibrium, what % p of latent cases

progress to be active each generation?What happens if a higher % progresses?

New diagnostic tool affects the transitions from active TB.

20

p?p?

19 3/2008

What are the costs of over-treatment?What are the costs of over-treatment?

BMGF said not to consider costs, but…BMGF said not to consider costs, but…

If no cost of treatment, no need for Dx, just treat all If no cost of treatment, no need for Dx, just treat all equivocal patients.equivocal patients.

What are the costs?What are the costs?

Side-effects of treatmentSide-effects of treatment

Possible super-germs?Possible super-germs?

Opportunity costsOpportunity costs

20 3/2008

Methods to calculate C = Methods to calculate C = cost*cost*of over-treatmentof over-treatment

Ask panel of experts: identifying one more person with Ask panel of experts: identifying one more person with TB justifies the costs of tagging N more people who TB justifies the costs of tagging N more people who don’t have TB as having it?don’t have TB as having it?

Use opportunity costs of wasted money, assuming we Use opportunity costs of wasted money, assuming we can save a life by spending $x on another program.can save a life by spending $x on another program.

Girosi: Use treatment guidelines to bound C Girosi: Use treatment guidelines to bound C

If we don’t treat when p(dis) < PminIf we don’t treat when p(dis) < Pmin then C > lower bound Lthen C > lower bound L

If we treat whenever p(dis)>PmaxIf we treat whenever p(dis)>Pmax then C < upper bound Uthen C < upper bound U

* This cost is called the Harm in Hunink.* This cost is called the Harm in Hunink.

21 3/2008

Girosi’s Method to calculate C = Girosi’s Method to calculate C = cost of over-treatment (2)cost of over-treatment (2)

Girosi: Often, recommended treatment involves a test Girosi: Often, recommended treatment involves a test and then only the positives get treated. for ARI it’s a and then only the positives get treated. for ARI it’s a clinical judgment with TPR = .9, TNR =.7, the benefit of clinical judgment with TPR = .9, TNR =.7, the benefit of treatment = .2 -.1 = .1 death averted, and prior p =.015.treatment = .2 -.1 = .1 death averted, and prior p =.015.

costs of treating everyone = H(1-p)~Hcosts of treating everyone = H(1-p)~H

costs of treating no one = Bpcosts of treating no one = Bp

costs of following clin judgment: (1-.9)pB + (1-.7) (1-p)Hcosts of following clin judgment: (1-.9)pB + (1-.7) (1-p)H

If the last is best .1pB +.3H <H and also <pB.If the last is best .1pB +.3H <H and also <pB.

So .7 H> .1pB and .3H<.9pBm so pB/7<H <3pB. So H=pB So .7 H> .1pB and .3H<.9pBm so pB/7<H <3pB. So H=pB is a reasonable guess. H = .015x.1 = .0015is a reasonable guess. H = .015x.1 = .0015

22 3/2008

Potential vs Actual accessPotential vs Actual access

Actual: people currently being tested for disease X in Actual: people currently being tested for disease X in Level L facilitiesLevel L facilities

Potential: people who could get to a facility of level L Potential: people who could get to a facility of level L with a certain amount of time and effortwith a certain amount of time and effort

We used potential:We used potential:

with better tests and treatment, demand would with better tests and treatment, demand would riserise

thinking about the post diffusion futurethinking about the post diffusion future

Fit potential access using geographic data from a few Fit potential access using geographic data from a few countries, and data on actual access to TB clinics in a countries, and data on actual access to TB clinics in a regression and estimate for all countries of interest.regression and estimate for all countries of interest.

23 3/2008

Data ProblemsData Problems

Models need incidence of disease of interest, incidence Models need incidence of disease of interest, incidence of seeking treatment for symptoms, outcomes of treated of seeking treatment for symptoms, outcomes of treated and untreated cases, accuracy of status quo testsand untreated cases, accuracy of status quo tests

Future test characteristics can be whatever we choose.Future test characteristics can be whatever we choose.

24 3/2008

Introduction Introduction

Better diagnosis might reduce the burden of disease Better diagnosis might reduce the burden of disease

Modeling the Benefits of New DiagnosticsModeling the Benefits of New Diagnostics

Basic approachBasic approach

Key issues and decisionsKey issues and decisions

FindingsFindings

Benefits of New Diagnostics for Bacterial Lower RIBenefits of New Diagnostics for Bacterial Lower RI

Selected Findings from Other DiseasesSelected Findings from Other Diseases

OutlineOutline

25 3/2008

Respiratory Infections Are the Leading Cause Respiratory Infections Are the Leading Cause of Childhood Mortalityof Childhood Mortality

Respiratory infections contribute to the deaths of Respiratory infections contribute to the deaths of more than 2 million children each year, mostly in more than 2 million children each year, mostly in Africa and Southeast AsiaAfrica and Southeast Asia

Most of these 2 million children die of bacterial Most of these 2 million children die of bacterial pneumonia, which is treatable with antiobioticspneumonia, which is treatable with antiobiotics

In developing countries, the main form of diagnosis In developing countries, the main form of diagnosis is clinical assessment is clinical assessment

A large number of children are not being diagnosed, A large number of children are not being diagnosed, while others are being treated unnecessarily, leading while others are being treated unnecessarily, leading to wasted treatments and antibiotic resistanceto wasted treatments and antibiotic resistance

26 3/2008

Child withChild withsymptomssymptoms

Access toAccess toclinical diagnosisclinical diagnosis

Self-treatSelf-treat

No careNo care

StatusStatusQuoQuo

New New TestTest

Access to Access to new testnew test

No accessNo accessto new testto new test

We Modeled the Status Quo and Access to a New We Modeled the Status Quo and Access to a New Test for Bacterial Pneumonia in Children Under 5Test for Bacterial Pneumonia in Children Under 5

Disease YesDisease Yes

Disease NoDisease NoTest + TreatTest + Treat

Test – No TreatTest – No Treat

SurvivesSurvivesDiesDiesSurvivesSurvivesDiesDiesSurvivesSurvivesDiesDiesSurvivesSurvivesDiesDies

Disease YesDisease Yes

Disease NoDisease No

27 3/2008

We Traded Off Test Performance and We Traded Off Test Performance and Infrastructure RequirementsInfrastructure Requirements

AdvancedAdvancedinfrastructureinfrastructure

Good Good PerformancePerformance

Lives saved by new test forLives saved by new test forbacterial pneumonia*bacterial pneumonia*

PerfectPerfectPerformancePerformance

MinimalMinimalinfrastructureinfrastructure

*Results *Results assumeassumeaccess toaccess totreatmenttreatment

28 3/2008

Easier Tests Produce Large Health Benefits, Easier Tests Produce Large Health Benefits, Even with Less Than Perfect PerformanceEven with Less Than Perfect Performance

AdvancedAdvancedinfrastructureinfrastructure

Good Good PerformancePerformance

261,000261,000142,00142,0000

596,000596,000405,000405,000

SignificantSignificantgains,gains,potentiallypotentiallyachievableachievable

PerfectPerfectPerformancePerformance

MinimalMinimalinfrastructureinfrastructure

*Results *Results assumeassumeaccess toaccess totreatmenttreatmentLives saved by new test forLives saved by new test for

bacterial pneumonia* bacterial pneumonia*

29 3/2008

Our Recommendations for a New DiagnosticOur Recommendations for a New Diagnosticfor Bacterial Lower Respiratory Infectionsfor Bacterial Lower Respiratory Infections

• Requires minimal infrastructureRequires minimal infrastructure

• Preferred samples types include saliva, urine, or Preferred samples types include saliva, urine, or dried blood spot dried blood spot

• Results should be available within 2 hours or lessResults should be available within 2 hours or less

30 3/2008

Much of the Benefit of the New Diagnostic Much of the Benefit of the New Diagnostic Would Be Due to Reductions in OvertreatmentWould Be Due to Reductions in Overtreatment

0

50

100

150

200

250

300

Developing WorldDeveloping World

LivesLivesSavedSaved(1000s)(1000s)

ReductionReductionin overtreatmentin overtreatment

ReductionReductionin diseasein disease

burdenburden

Benefits of New Test forBenefits of New Test forBacterial PneumoniaBacterial Pneumonia With C = .001 !With C = .001 !

31 3/2008

Most Lives Saved from Reducing Disease Burden Most Lives Saved from Reducing Disease Burden Accrue to Africa, While Other Regions Benefit Accrue to Africa, While Other Regions Benefit

from Reducing Overtreatmentfrom Reducing Overtreatment

0

50

100

150

200

250

0

50

100

150

200

250

300

Developing WorldDeveloping World AfricaAfrica AsiaAsia Latin Latin AmericaAmerica

LivesLivesSavedSaved(1000s)(1000s)

ReductionReductionin overtreatmentin overtreatment

ReductionReductionin diseasein disease

burdenburden

ReductionReductionin overtreatmentin overtreatment

ReductionReductionin diseasein disease

burdenburden

Benefits of New Test forBenefits of New Test forBacterial PneumoniaBacterial Pneumonia Benefits of New Test by RegionBenefits of New Test by Region

32 3/2008

All Disease Models Show a Significant All Disease Models Show a Significant Benefit from New DiagnosticsBenefit from New Diagnostics

0 500,000 1,000,000 1,500,000 2,000,000

TB

Children under age 5 (Africa)Children under age 5 (Africa)

Children under age 5Children under age 5

Pregnant women (Africa)Pregnant women (Africa)

Adults with persistent coughAdults with persistent cough

Lives Saved Annually from New Forum-Recommended TestsLives Saved Annually from New Forum-Recommended Tests

Additional benefits were found for other diseases studiedAdditional benefits were found for other diseases studied

Malaria

Syphilis

Bacterialpneumonia

33 3/2008

Study Findings Highlight Other ThemesStudy Findings Highlight Other Themes

Tests that are more accessible have greater benefitsTests that are more accessible have greater benefits

Access can be more important than test performanceAccess can be more important than test performance

For many diseases, a more accurate test is not neededFor many diseases, a more accurate test is not needed

For example, current rapid tests for malaria could lead For example, current rapid tests for malaria could lead to significant benefits if made more widely availableto significant benefits if made more widely available

Current diagnostics do not pay enough attention to harm of Current diagnostics do not pay enough attention to harm of overtreatmentovertreatment

Tests are typically better at identifying people with Tests are typically better at identifying people with disease than identifying those withoutdisease than identifying those without

Reducing overtreatment provides a public health benefitReducing overtreatment provides a public health benefit