Policy Evaluation

Syndemics

Prevention Network

Edinburgh Evaluation Summer SchoolEdinburgh, Scotland

June 6, 2007

Exploring the Dynamic and Democratic Dimensions of Health Protection Policies

Bobby Milstein Syndemics Prevention NetworkCenters for Disease Control and

[email protected]

http://www.cdc.gov/syndemics

Policy Evaluation

Syndemics

Prevention Network

Appreciating the Unique Character of Evaluative Inquiry

“It is easier to find facts than it is to face them.”

Centers for Disease Control and Prevention. What procedures are available for planning and evaluating initiatives to prevent syndemics? Syndemics Prevention Network, 2001. Available at <http://www.cdc.gov/syndemics/overview-planeval.htm>.

Questions of Fact(descriptions, associations, effects)

ResearchSystematic

MethodsEvaluation

Questions of Values(merit, worth, significance)

Syndemics

Prevention Network

Picture a Neighborhood Where…

• Conditions are not supportive of healthy living

• People are either afflicted by or at risk for numerous mutually reinforcing health problems

• Citizen leaders are making an effort to alleviate afflictions and improve living conditions, but their power is limited

• More could be done through better local organizing and with effective assistance from outside allies (e.g., philanthropy, government)James Nachtwey in Sachs J. How to end poverty.

Time Magazine 2005 March 14.

How does public health policy typically proceed in such circumstances?

Which forms of policy planning and evaluation are most relevant and promising?

Syndemics

Prevention Network

Policy Planning & EvaluationEngages Questions of Social Navigation

Prevalence of Diagnosed Diabetes, US

0

10

20

30

40

1980 1990 2000 2010 2020 2030 2040 2050

Mill

ion

pe

op

le

HistoricalData

Markov Model Constants• Incidence rates (%/yr)• Death rates (%/yr)• Diagnosed fractions(Based on year 2000 data, per demographic segment)

Honeycutt A, Boyle J, Broglio K, Thompson T, Hoerger T, Geiss L, Narayan K. A dynamic markov model for forecasting diabetes prevalence in the United States through 2050. Health Care Management Science 2003;6:155-164.

Jones AP, Homer JB, Murphy DL, Essien JDK, Milstein B, Seville DA. Understanding diabetes population dynamics through simulation modeling and experimentation. American Journal of Public Health 2006;96(3):488-494.

Markov Forecasting Model

Trend is not destiny!

How?

Why?

Where?

Who?

What?

Syndemics

Prevention Network

Modern public health policy—and evaluation—are becoming more…

• Inter-connected (ecological, multi-causal, dynamic, systems-oriented) Concerned more with leverage than control

• Public (broad-based, partner-oriented, citizen-led, inter-sector, democratic) Concerned with many interests and mutual-accountability

• Questioning (evaluative, reflexive, critical, practical)Concerned with creating and protecting values like health, equity,dignity, security, satisfaction, justice, wealth, and freedom in both means and ends

A Field in Transition

Syndemics

Prevention Network

Left Unexamined…

• Singular “program” as the unit of inquiry (N=1 organizational depth)

• Dynamic aspects of program effectiveness (e.g., better-before-worse patterns of change)

• Democratic aspects of public health work (e.g., alignment among multiple actors, including those who are not professionals and who may be pursuing other goals)

• Evaluative aspects of planning

Milstein B, Wetterall S, CDC Evaluation Working Group. Framework for program evaluation in public health. MMWR Recommendations and Reports 1999;48(RR-11):1-40. Available at <http://www.cdc.gov/mmwr/PDF/RR/RR4811.pdf>.

Framework for Program Evaluation“Both a synthesis of existing evaluation practices

and a standard for further improvement.”

Syndemics

Prevention Network

Are We Posing Questions About Attribution or Contribution?

“…if a program’s activities are aligned with those

of other programs operating in the same setting,

certain effects (e.g., the creation of new laws or

policies) cannot be attributed solely to one

program or another. In such situations, the goal

for evaluation is to gather credible evidence that

describes each program’s contribution in the

combined change effort. Establishing

accountability for program results is predicated

on an ability to conduct evaluations that assess

both of these kinds of effects.” p.11-12

Calls into question the conditions in which one focuses on a “program” as the unit of analysis

Syndemics

Prevention Network

• Locating categorical disease or risk prevention programs within a broader system of health protection

• Constructing credible knowledge without comparison/control groups

• Differentiating questions that focus on attribution vs. contribution

• Balancing trade-offs between short- and long-term effects

• Avoiding the pitfalls of professonalism (e.g., over-specialization, arrogance, reinforcement of the status quo)

• Harnessing the power of intersectoral and citizen-led public work

• Defining standards and values for judgment

• Others…

Serious Challenges for Planners and Evaluators

Syndemics

Prevention Network

Topics for Today

• Health Protection Policy in a Dynamic and Democratic World– Concepts, keywords, structures

• Looking Backward, Looking Forward

– Retrospectively evaluating past policy

– Prospectively crafting/evaluating future policy

• Highlighting One Promising Methodology

– System Dynamics simulation modeling

• Questions and Discussion Throughout

Syndemics

Prevention Network

Defining Keywords

Adapted from:

Milio N. Glossary: healthy public policy. Journal of Epidemiology and Community Health 2001;55(9):622-623.

Forrester JW. Policies and decisions. In: Industrial Dynamics. Cambridge, MA: MIT Press; 1961. p. 93-108.

Bennett T, Grossberg L, Morris M. New keywords: a revised vocabulary of culture and society. Malden, MA: Blackwell Pub., 2005.

Scriven M. Evaluation thesaurus. 4th ed Newbury Park, CA: Sage Publications, 1991.

Policy evaluation is…

Policy is…• The plans, programs, principles, or more broadly the

course of action of some actor(s), which may include a degree of deliberate inaction as well

• Explicit or implicit rules for deciding how to respond to circumstances and pressures

• Priorities guiding resource allocation

• The systematic process of determining—and improving—the merit, worth, or significance of decisions about what to do, or not to do, in a given domain

• The articulation and assessment of alternative possible futures, each corresponding to a different policy

Syndemics

Prevention Network

Policy is our general approach toward a particular problem or area of concern…

Continual, Iterative Process of Policy Planning & Evaluating

POLICYDEVELOPMENT

ASSESSMENT

ASSURANCE

Assuring Healthful Conditions for All

Many Methodologies…Pilot and Demonstration

Theories of ChangeHealth impact assessment

Simulation modelingFuturing or Storytelling

Many Methodologies…Pilot and Demonstration

Theories of ChangeHealth impact assessment

Simulation modelingFuturing or Storytelling

Many Methodologies…Communications

AuditingLaw Enforcement

Leadership & OrganizingPower mapping

Non-violent actionSocial Navigation

Many Methodologies…Communications

AuditingLaw Enforcement

Leadership & OrganizingPower mapping

Non-violent actionSocial Navigation

Many Methodologies…Surveys

Needs AssessmentAsset MappingFrame analysis

Concept mappingNetwork analysis

Time-trend analysis

Many Methodologies…Surveys

Needs AssessmentAsset MappingFrame analysis

Concept mappingNetwork analysis

Time-trend analysis

Institute of Medicine. The future of public health. Washington, DC: National Academy Press, 1988.

Institute of Medicine. The future of the public's health in the 21th century. Washington, DC: National Academy Press, 2002.

Syndemics

Prevention Network

Defining Keywords

Walt G. Health policy: an introduction to process and power. Atlantic Highlands, NJ: Zed Books, 1994.

Ignatieff M. The grey empitness inside John Major. The Observer 1992 November 15; 25.

“Policy is the selection of non-contradictory means to achieve non-contradictory ends over the medium to long term. Policy is the thread of conviction that keeps a government from

becoming the prisoner of events.”

-- Michael Ignatieff

Artist: Boyce Watt

Policy vs. Decisions

• Policy usually involves a series of specific decisions, programs, actions

• But the distinction is blurry– Policy makers never start from a blank

sheet of possibilities– Ad hoc decisions may together add up to

forceful implicit policy

Syndemics

Prevention Network

Events

Pattern

Events

WaterTemperature

FloodDamage

Economic Activity& Emissions

WaterLevel

Structure

R

Melting

Syndemics

Prevention Network

Time Series Models

Describe trends

Multivariate Stat Models

Identify historical trend drivers and correlates

Patterns

Structure

Events

Increasing:

• Depth of causal theory

• Robustness for longer-term projection

• Value for developing policy insights

• Degrees of uncertainty

Increasing:

• Depth of causal theory

• Robustness for longer-term projection

• Value for developing policy insights

• Degrees of uncertaintyDynamic Simulation Models

Anticipate new trends, learn about policy consequences,

and set justifiable goals

Tools for Policy Planning & Evaluation

Syndemics

Prevention Network

Consider the Track Record…

Sterman JD. Learning from evidence in a complex world. American Journal of Public Health 2006;96(3):505-514.

Forrester JW. Counterintuitive behavior of social systems. Technology Review 1971;73(3):53-68.

• Low tar and low nicotine cigarettesLead to greater carcinogen intake

• Fad dietsProduce diet failure and weight gain

• Antibiotic & pesticide useStimulate resistant strains

• Road building to ease congestion Attracts development, increases traffic, delays, and pollution

• Air-conditioning useRaises neighborhood heat

• Forest fire suppressionBuilds deadwood fueling larger, hotter, more dangerous fires

• War on drugs Raises price and attracts supply

• Suppressing dissent Inspires radicalization and extremism

Syndemics

Prevention Network

Policy Resistance is…

“The tendency for interventions to be delayed, diluted, or defeated

by the response of the system to the intervention itself.”

Meadows DH, Richardson J, Bruckmann G. Groping in the Dark: The First Decade of Global Modelling. Wiley: New York, 1985.

-- Meadows, Richardson & Bruckmann

Defining Keywords

Syndemics

Prevention Network

Seeking High-Leverage Policies

Wall painting in the Stanzino delle Matematiche in the Galleria degli Uffizi (Florence, Italy). Painted by Giulio Parigi in the years 1599-1600.

“Give me a firm place to stand and I will move the earth.”

-- Archimedes

Meadows DH. Leverage points: places to intervene in a system. Sustainability Institute, 1999. Available at <http://www.sustainabilityinstitute.org/pubs/Leverage_Points.pdf>.

Syndemics

Prevention Network

Public Health Work Literally Involves Redirecting the Course of Change

600

500

400

200

100

501950 1960 1970 1980 1990 1995

Ag

e-a

dju

ste

d D

eath

Rat

e p

er 1

00,

000

Po

pu

lati

on

1955 1965 1975 1985

300

700

Peak Rate

Rate if trend continued

Year

Actual and Expected Death Rates for Coronary Heart Disease, 1950–1998

Marks JS. The burden of chronic disease and the future of public health. CDC Information Sharing Meeting. Atlanta, GA: National Center for Chronic Disease Prevention and Health Promotion; 2003.

Centers for Disease Control and Prevention. Achievements in public health, 1900-1999: decline in deaths from heart disease and stroke -- United States, 1900-1999. MMWR 1999;48(30):649-656. Available at <http://www.cdc.gov/mmwr/preview/mmwrhtml/mm4830a1.htm>

Actual Rate

Overall Decline is Linked to…

• Reduced smoking

• Changes in diet

• Better diagnosis and treatment

• More heath services utilization

Overall Decline is Linked to…

• Reduced smoking

• Changes in diet

• Better diagnosis and treatment

• More heath services utilization

684,000 fewer deaths in 1998 alone

684,000 fewer deaths in 1998 alone

Syndemics

Prevention Network

“Public health is probably the most successful system of science and

technology combined, as well as social policy, that has ever been devised…It is, I think, a paradigmatic model for how you do concerned, humane, directed science.”

-- Richard Rhodes

Rhodes R. Limiting human violence: an emerging scientific challenge. Sarewitz D, editor. Living With the Genie: Governing Science and Technology in the 21st Century; New York, NY: Center for Science, Policy, and Outcomes; 2002.

One Observer's View…

Syndemics

Prevention Network

Immense Challenges Ahead

United Nations Department of Economic and Social Affairs. Population Division. The world at six billion. Washington D C: Population Division Dept. of Economic and Social Affairs United Nations Secretariat, 1999.

CNN. Sarajevo baby to be honored as 6 billionth person on Earth. CNN, 1999. Accessed July 5, 2003 at <http://www.cnn.com/WORLD/europe/9910/11/population.02/>.

World Population Growth

Syndemics

Prevention Network

Resource Depletion & Related Conflict

Syndemics

Prevention Network

A Glimpse Into 2020

Murray CJL, Lopez AD. The global burden of disease: summary. Cambridge, MA: Harvard University Press, 1996.

Syndemics

Prevention Network

A Glimpse Into 2020

Murray CJL, Lopez AD. The global burden of disease: summary. Cambridge, MA: Harvard University Press, 1996.

On the List

War

HIV

Violence

Self-inflicted injury

Cancer of the trachea, bronchus, and lung

Off the List

Measles

Malaria

Falls

Anemia

Malnutrition

Syndemics

Prevention Network

Broad Dynamics of the Health Protection Enterprise

Prevalence of Vulnerability, Risk, or Disease

Time

HealthProtection

Efforts

-

B

Responsesto Growth

Resources &Resistance

-B

Obstacles

Broader Benefits& Supporters

R

ReinforcersPotentialThreats

The concepts and methods of policy evaluation must engage the basic features of this

dynamic and democratic system

The concepts and methods of policy evaluation must engage the basic features of this

dynamic and democratic system

Size of the Safer, Healthier

Population-

Prevalence of Vulnerability,

Risk, or Disease

B

Taking the Toll

0%

100%

R

Drivers ofGrowth

Values for Health & Equity

Syndemics

Prevention Network

A Complementary Science of Relationships

• Efforts to Reduce Population Health ProblemsProblem, problem solver, response

• Efforts to Organize a System that Assures Healthful Conditions for All Dynamic interaction among multiple problems, problem solvers, and responses

Institute of Medicine. The future of public health. Washington, DC: National Academy Press, 1988.

Institute of Medicine. The future of the public's health in the 21th century. Washington, DC: National Academy Press, 2002.

Bammer G. Integration and implementation sciences: building a new specialisation. Cambridge, MA: The Hauser Center for Nonprofit Organizations, Harvard University 2003.

True innovation occurs when things are put together for the first time that had been separate.

– Arthur Koestler

Syndemics

Prevention Network

Summers J. Soho: a history of London's most colourful neighborhood. Bloomsbury, London, 1989. p. 117.

Broad Street, One Year Later

John Snow Heroic Success or Cautionary Tale?

“No improvements at all had been made...open cesspools are still to be seen...we have all the materials for a fresh epidemic...the water-butts were in deep cellars, close to the undrained cesspool...The overcrowding appears to increase."

Syndemics

Prevention Network

“At least six times since the Depression, the United States has tried and failed to enact a national health insurance program.”

Lee P, Paxman D. Reinventing public health. Annual Reviews of Public Health 1997;18:1-35.

Number of Uninsured Americans, 1976-2003

Himmelstein, Woolhandler, Carrasquillo – Tabulation from CPS and NHIS – Lee & Paxman

Another Prototypical ExampleAttempts to Reform the U.S. Health Care Delivery System

Syndemics

Prevention Network

• Piecemeal approaches

• Failure to address root problems

• Inattention to the larger political and economic system

Heirich M. Rethinking health care: innovation and change in America. Boulder, CO: Westview Press, 1999.

Crafting Health Policies that will Succeed in a Large, Dynamic System

Efforts to reform health care policy have been ineffective because of

“Most of the analytic strategies popular among academics, politicians, and policy makers fail to observe the system as a whole…to discuss processes of mutual change that are occurring, or to analyze how innovations fit into larger nonequilibrium dynamics that are developing.”

-- Max Heirich

Syndemics

Prevention Network

Understanding Dynamic Complexity


Meadows DH. Leverage points: places to intervene in a system. Sustainability Institute, 1999. Available at <http://www.sustainabilityinstitute.org/pubs/Leverage_Points.pdf>.

Richardson GP. Feedback thought in social science and systems theory. Philadelphia, PA: University of Pennsylvania Press, 1991.

Sterman JD. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin McGraw-Hill, 2000.

Syndemics

Prevention Network

Changing Views of Population HealthWhat Accounts for Poor Population Health?

• God’s will

• Humors, miasma, ether

• Poor living conditions, immorality (e.g., ?)

• Single disease, single cause (e.g., ?)

• Single disease, multiple causes (e.g., ?)

• Single cause, multiple diseases (e.g., ?)

• Multiple causes, multiple diseases (but no feedback dynamics) (e.g., ?)

• Dynamic feedback among afflictions, living conditions, and public strength (e.g., ?)

1880

1950

1960

1980

2000

1840

Milstein B. Hygeia's constellation: navigating health futures in a dynamic and democratic world [Doctoral Dissertation]. Cincinnati, OH: Union Institute & University; 2006.

Richardson GP. Feedback thought in social science and systems theory. Philadelphia, PA: University of Pennsylvania Press, 1991.

Syndemics

Prevention Network

Placing Health in a Wider Set of Relationships

Health

LivingConditions

Power toAct

This orientation explicitly includes within it our power to craft policies, along with an understanding of the

changing pressures, constraints, and consequences that shape it.

“Health Policy”

“Social Policy”

Syndemics

Prevention Network

Two Orientations

Prospective

Retrospective• What have been the observed consequences of

prior decisions?• For whom? When? Why?

At what cost?• Recommendations to continue or change strategy

• What is the range of plausible consequences of policy options?

• For whom? When? Why? At what cost?

• Which alternative futures are most highly valued, or feared?

• What must be done to move in the desired direction?

Syndemics

Prevention Network

Explicitly recognizes the evaluative aspects of planning:

• Defining problems

• Setting priorities

• Developing options

• Selecting strategies

Risley J. Public policy evaluation. Kalamazoo, MI: The Evaluation Center, Western Michigan University; February 26, 2004. <http://www.wmich.edu/evalctr/evalcafe/risley022604slides.pdf>.

Prospective Policy Evaluation

Syndemics

Prevention Network

When Faced with the Vast Scope of Public Health Threats…

Narrow the Focus and Specialize

• Identify problem

• Formulate policy

• Implement policy

• Evaluate policy

• Repeat steps 1-4, as necessary!

Breeding Ground for Disease (Karen Kasmauski, National Geographic, 2001).

Syndemics

Prevention Network

Diseases of Disarray

Hardening of the categories

Tension headache between treatment and prevention

Hypocommitment to training

Cultural incompetence

Political phobia

Input obsession

Wiesner PJ. Four disease of disarray in public health. Annals of Epidemiology. 1993;3(2):196-8.

Chambers LW. The new public health: do local public health agencies need a booster (or organizational "fix") to combat the diseases of disarray? Canadian Journal of Public Health 1992;83(5):326-8.

Syndemics

Prevention Network

Dangers of Getting Too Specific

Krug EG, World Health Organization. World report on violence and health. Geneva: World Health Organization, 2002.

Conventional problem solving proliferates problems

Opens a self-reinforcing niche for professional problem solvers

Obscures patterns that transcend any specific problem (e.g., nonviolence is entirely neglected)

Syndemics

Prevention Network

Examples of Nonviolent Action

Albert Einstein Institution. Applications of nonvilolent action. Albert Einstein Institution, 2001.

Powers RS, Vogele WB, Kruegler C, McCarthy RM. Protest, power, and change: an encyclopedia of nonviolent action from ACT-UP to women's suffrage. New York: Garland Pub., 1997.

Dismantling dictatorships

Blocking coups d’état

Defending against foreign invasions and occupations

Providing alternatives to violence in extreme ethnic conflicts

Challenging unjust social and economic systems

Developing, preserving and extending democratic practices, human rights, civil liberties, and freedom of religion

Resisting genocide

“A phenomenon that cuts across ethnic, cultural, religious, geographic,

socioeconomic and other demographic lines.”

-- Albert Einstein Institution

Syndemics

Prevention Network

Systems Archetype

“Fixes that Fail”

Kim DH. Systems archetypes at a glance. Cambridge, MA: Pegasus Communications, Inc., 1994.

Fix

+

ProblemSymptom

-

UnintendedConsequence

+

Delay

+

-B

+R

Characteristic Behavior:

Better before Worse

Syndemics

Prevention Network

“Fixes that Fail” in Public Health Vocabulary

The Risk of Targeted Interventions

+

HealthProblem -

-

Exclusions

+

+

TargetedResponseB

Delay+R

What issues tend to be excluded?

Syndemics

Prevention Network

Some Categories of Exclusions

Conceptual

Social

Organizational

Political

Disarray

Disorientation

Disparity & Disconnection

Together, these forces may seriously undermine the effectiveness of health protection policy

Syndemics

Prevention Network

Wickelgren I. How the brain 'sees' borders. Science 1992;256(5063):1520-1521.

How Many Triangles Do You See?

Syndemics

Prevention Network

Boundary Judgments(System of Reference)

Observations(Facts)

Evaluations(Values)

Ulrich W. Boundary critique. In: Daellenbach HG, Flood RL, editors. The Informed Student Guide to Management Science. London: Thomson; 2002. p. 41-42. <http://www.geocities.com/csh_home/downloads/ulrich_2002a.pdf>.

Ulrich W. Reflective practice in the civil society: the contribution of critically systemic thinking. Reflective Practice 2000;1(2):247-268. http://www.geocities.com/csh_home/downloads/ulrich_2000a.pdf

Boundary CritiqueCreating a new theory is not like destroying an old barn and erecting a skyscraper in its

place. It is rather like climbing a mountain, gaining new and wider views, discovering unexpected connections between our starting point and its rich environment.

-- Albert Einstein

Syndemics

Prevention Network

Boundary CritiqueEqualizing Experts and Ordinary Citizens

• “Professional expertise does not protect against the need for making boundary judgements…nor does it provide an objective basis for defining boundary judgements. It’s exactly the other way round: boundary judgements stand for the inevitable selectivity and thus partiality of our propositions.

• It follows that experts cannot justify their boundary judgements (as against those of ordinary citizens) by referring to an advantage of theoretical knowledge and expertise.

• When it comes to the problem of boundary judgements, experts have no natural advantage of competence over lay people.”

Ulrich W. Reflective practice in the civil society: the contribution of critically systemic thinking. Reflective Practice 2000;1(2):247-268.

-- Werner Ulrich

Syndemics

Prevention Network

“You Can Argue with Einstein”

Yankelovich D. Coming to public judgment: making democracy work in a complex world. 1st ed Syracuse, NY: Syracuse University Press, 1991. p. 220.

“For certain purposes, public judgment should

carry more weight than expert opinion – and not simply

because the majority may have more political power than

the individual expert but because the public’s claim to

know is actually stronger than the experts’...the judgment

of the general public can, under some conditions, be

equal or superior in quality to the judgment of experts

and elites who possess far more information, education,

and ability to articulate their views.”

-- Daniel Yankelovich

Syndemics

Prevention Network

Ulrich W. Reflective practice in the civil society: the contribution of critically systemic thinking. Reflective Practice 2000;1(2):247-268. http://www.geocities.com/csh_home/downloads/ulrich_2000a.pdf

Boundary Critique

Syndemics

Prevention Network

Epi·demic

• The term epidemic is an ancient word signifying a kind of relationship wherein something unknown (or unknowable) is put upon the people

• Epidemiology first appeared just over a century ago (in 1873), in the title of J.P. Parkin's book "Epidemiology, or the Remote Cause of Epidemic Diseases“

• Ever since then, the conditions that cause health problems have increasingly become matters of public concern and public work

Elliot G. Twentieth century book of the dead. New York,: C. Scribner, 1972.

Martin PM, Martin-Granel E. 2,500-year evolution of the term epidemic. Emerging Infectious Diseases 2006. Available from http://www.cdc.gov/ncidod/EID/vol12no06/05-1263.htm

National Institutes of Health. A Short History of the National Institutes of Health. Bethesda, MD: 2006. Available from http://history.nih.gov/exhibits/history/

Parkin J. Epidemiology; or the remote cause of epidemic diseases in the animal and the vegetable creation. London: J and A Churchill, 1873.

A representation of the cholera epidemic of the nineteenth century.Source: NIH

“The pioneers of public health did not change nature, or men, but adjusted the active relationship of men to certain aspects of nature so that the relationship became one of watchful and healthy respect.”

-- Gil Elliot

Syndemics

Prevention Network

Syn·demic

• The term syndemic, first used in 1992, strips away the idea that illnesses originate from extraordinary or supernatural forces and places the responsibility for affliction squarely within the public arena

• It acknowledges relationships and signals a commitment to studying population health as a a fragile, dynamic state requiring continual effort to maintain and one that is imperiled when social and physical forces operate in harmful ways

Confounding

Connecting*

Synergism

Syndemic

Events

System

Co-occurring

* Includes several forms of connection or inter-connection such as synergy, intertwining, intersecting, and overlapping

Milstein B. Hygeia's constellation: navigating health futures in a dynamic and democratic world. Doctoral dissertation. Cincinnati, OH: Union Institute and University. November, 2006.

Milstein B. Spotlight on syndemics. Centers for Disease Control and Prevention, 2001. <http://www.cdc.gov/syndemics>

Syndemics

Prevention Network

Milstein B, Homer J. The dynamics of upstream and downstream: why is so hard for the health system to work upstream, and what can be done about it? CDC Futures Health Systems Workgroup; Atlanta, GA; 2003.

TertiaryPrevention

SecondaryPrevention

PrimaryPrevention

TargetedProtection

Society's HealthResponse

Demand forresponse

PublicWork

SaferHealthierPeople Becoming

vulnerable

Becoming saferand healthier

VulnerablePeople Becoming

afflicted

Afflictedwithout

Complications Developingcomplications

Afflicted withComplications

Dying fromcomplications

Health System Dynamics

Adverse LivingConditions

GeneralProtection

Milstein B, Homer J. The dynamics of upstream and downstream: why is so hard for the health system to work upstream, and what can be done about it? CDC Futures Health Systems Work Group; Atlanta, GA; December 3, 2003.

Gerberding JL. CDC's futures initiative. Atlanta, GA: Public Health Training Network; April 12, 2004.

Gerberding JL. FY 2008 CDC Congressional Budget Hearing. Testimony before the Committee on Appropriations, Subcommittee on Labor, Health and Human Services, Education and Related Agencies, United States House of Representatives; Washington, DC; March 9, 2007.

Homer JB, Hirsch GB. System dynamics modeling for public health: background and opportunities. American Journal of Public Health 2006;96(3):452-458.

“One major task that CDC is intending to address is balancing this portfolio of our health system so that there is much greater emphasis placed on health protection, on making sure that we invest the same kind of intense resources into keeping people

healthier or helping them return to a state of health and low vulnerability as we do to disease care and end of life care."

-- Julie Gerberding

Syndemics

Prevention Network

Understanding Health as Public Work

SaferHealthierPeople

VulnerablePeople

Afflictedwithout

Complications

Afflicted withComplicationsBecoming

vulnerable


Becomingafflicted

Developingcomplications




Demand forresponse

GeneralProtection

TargetedProtection

PrimaryPrevention

SecondaryPrevention

TertiaryPrevention

-

Public Work-

Vulnerable andAfflicted People

Fraction of Adversity,Vulnerability and AfflictionBorne by Disadvantaged

Sub-Groups (Inequity)

-

PublicStrength

Citizen Involvementin Public Life

Social Division

Syndemics

Prevention Network

Evaluating Dynamic, Democratic Policies

How can we learn about the consequences of alternative policies in a system of this kind?


VulnerablePeople

Afflictedwithout

Complications

Afflicted withComplicationsBecoming

vulnerable


Becomingafflicted





Demand forresponse

GeneralProtection

TargetedProtection

PrimaryPrevention

SecondaryPrevention

TertiaryPrevention

-

Public Work-

Vulnerable andAfflicted People

Fraction of Adversity,Vulnerability and AfflictionBorne by Disadvantaged

Sub-Groups (Inequity)

PublicStrength

-

Citizen Involvementin Public Life

Social Division

Syndemics

Prevention Network

What Affects the Balance of Upstream and Downstream Work?

Upstream Prevention and Protection-----------------------------------Total 3%

Downstream Care and Management--------------------------------Total 97%

Brown R, Elixhauser A, Corea J, Luce B, Sheingod S. National expenditures for health promotion and disease prevention activities in the United States. Washington, DC: Battelle; Medical Technology Assessment and Policy Research Center; 1991. Report No.: BHARC-013/91-019.

Syndemics

Prevention Network

Balancing Two Major Areas of Emphasis


VulnerablePeople

Afflictedwithout

ComplicationsAfflicted with

ComplicationsBecomingvulnerable


Becomingafflicted





Demand forresponse

GeneralProtection

TargetedProtection

PrimaryPrevention

SecondaryPrevention

TertiaryPrevention

Public Work

World of Providing…

• Education• Screening• Disease management • Pharmaceuticals• Clinical services• Physical and financial access• Etc…

Medical and Public Health Policy

MANAGEMENT OF DISEASES AND RISKS

World of Transforming…

• Deprivation• Dependency• Violence• Disconnection• Environmental decay• Stress• Insecurity• Etc…

By Strengthening…

• Leaders and institutions• Foresight and precaution• The meaning of work• Mutual accountability• Plurality• Democracy• Freedom• Etc…

Healthy Public Policy & Public Work

DEMOCRATIC SELF-GOVERNANCE


Syndemics

Prevention Network

Two Broad Types of Policy

Types of Policy

Upstream Downstream

Type• Macro policy• System-wide scope

• Micro policy• Sector-specific scope

Examples

• Guaranteed living wage• War and the preparation for war• Regulation of “private” corporate

behavior

• Breast cancer screening• Educational testing• Housing vouchers

Procedures • “High politics” • “Low politics”

Crick BR. In defense of politics. 4th ed Chicago, IL: University of Chicago Press, 1993.

Walt G. Health policy: an introduction to process and power. Atlantic Highlands, NJ: Zed Books, 1994.

Syndemics

Prevention Network

Defining Keywords

Crick BR. In defense of politics. 4th ed Chicago, IL: University of Chicago Press, 1993.

Boyte HC. Everyday politics: reconnecting citizens and public life. Philadelphia, PA: University of Pennsylvania Press, 2004.

• PartisanFervent, sometimes militant support for a party, cause, faction, person, or idea, from Middle French, part, “faction”

• PoliticalThe action of diverse people negotiating their differences for common governance, from the Greek, politikos, “of the citizen”

Syndemics

Prevention Network

Healthy Public Policy Medical and Public Health Policy

Concerned chiefly with assuring safer, healthier conditions for all

Concerned chiefly with preventing and alleviating affliction, managing complications, and delaying premature death or disability

Relies heavily on multiple, small-scale, local solutions, with low technology

Relies heavily on specific high-technology solutions, widely applied

Combines analyses into a broad systems view, transcending sector boundaries

Confines analyses to the health sector

Future-oriented (reacting to long-term dynamics) Present-oriented (reacting to immediate events)

Questions the givens, focuses on plausible outcomes

Accepts the givens, focuses on probable outcomes

Evaluated first through simulation, then through implementation

Evaluated through implementation

Main resources are citizen leadership and broad- based public work (including that of professionals)

Main resources are money, professional expertise, and technology (often excluding citizen leadership)

Two Policy Orientations for Health Action

Adapted from: Hancock T. Beyond health care: from public health policy to healthy public policy. Can J Public Health 1985;76 Suppl 1:9-11.

Syndemics

Prevention Network

Healthy Public Policy Medical and Public Health Policy

Concerned chiefly with assuring safer, healthier conditions and reducing vulnerability for all

Concerned chiefly with preventing and alleviating affliction, managing complications, and delaying premature death or disability

Relies heavily on multiple, small-scale, local solutions, with low technology

Relies heavily on specific high-technology solutions, widely applied

Combines analyses into a broad systems view, transcending sector boundaries

Confines analyses to the health sector

Future-oriented (concerned with long-term dynamics) Present-oriented (reacting to immediate events)

Questions the givens, focuses on plausible outcomes

Accepts the givens, focuses on probable outcomes

Evaluated first through simulation, then through implementation

Evaluated through implementation

Main resources are citizen leadership and broad- based public work (including that of professionals)

Main resources are money, professional expertise, and technology (often excluding citizen leadership)

Two Policy Orientations for Health Action

Adapted from: Hancock T. Beyond health care: from public health policy to healthy public policy. Can J Public Health 1985;76 Suppl 1:9-11.

Syndemics

Prevention Network

Looking Backward Retrospective Policy Evaluation

Syndemics

Prevention Network

Adult Per Capita Cigarette Consumption and Major Smoking and Health Events

United States, 1900-1998

U.S. Department of Health and Human Services. Reducing tobacco use: a report of the Surgeon General. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; 2000. Available at <http://www.cdc.gov/tobacco/data_statistics/sgr/sgr_2000/index.htm#full>.

1st SurgeonGeneral’s

Report

1st Smoking-Cancer Concern

NonsmokersRights Movement Begins

BroadcastAd Ban

Federal CigaretteTax Doubles

0

1,000

2,000

3,000

4,000

5,000

1900 1910 1920 1930 1940 1950 1960 1970 1980 1990

Nu

mb

er o

f C

igar

ette

s

Health promotion does not seek to control for secular trends.

It tries to create them!

-- Marshall Kreuter

Health promotion does not seek to control for secular trends.

It tries to create them!

-- Marshall Kreuter

Syndemics

Prevention Network

Tracking Statewide Tobacco Control Efforts

California Department of Health Services. California tobacco control update: the social norm change approach. Sacramento, CA: Tobacco Control Section, California Department of Health Services 2006. <http://www.dhs.ca.gov/tobacco/documents/pubs/CTCUpdate2006.pdf>.

Syndemics

Prevention Network

What was Happening in California?

Comprehensive policy featuring…

• Statewide focus

• Community programs to reduce tobacco use

• Chronic disease programs to reduce the burden of tobacco-related diseases

• School-based efforts

• Enforcement

• Counter-marketing

• Cessation programs

• Surveillance and evaluation

• Administration and management

Syndemics

Prevention Network

Green LW. A federal agency's journey from bootstrap epidemiology to evidence-based practice to practice-based evidence. 4th Annual CDC Evaluation Summer Institute; Atlanta, GA: Centers for Disease Control and Prevention; June 10, 2004. Available at <http://www.chronicdisease.org/SEpresentations/GREEN--CDC%20Evidence-Based%20Prax%20to%20Prax-Based%20Evidence.ppt>.

The Comprehensiveness Imperative

• Interventions by themselves ineffective when taken to scale

• In trying to isolate the essential components of tobacco control programs that made them effective, none could be shown to stand alone

• Any combination of methods was more effective than the individual methods

• The more components, the more effective

• The more components, the better coverage

What was Happening in California?

Syndemics

Prevention Network

Observed blood lead

U.S. Policy Response to Concerns About Elevated Blood Lead Levels

Year

1975 1976 1977 1978 1979 1980 1981

Mean bloodlead levels

(g/dL)

9

10

11

12

13

14

15

16

17

Gasoline lead

Predicted blood lead

Data: National Health and Nutrition Examination Survey II

Lead used ingasoline

(thousandsof tons)

30

40

50

60

70

80

90

100

110

Intervention Effect: Blood lead fell 10 times

more than predicted!

Intervention Effect: Blood lead fell 10 times

more than predicted!

Syndemics

Prevention Network

Continuing Effects and Further ActionsBlood Lead Levels in the U.S. Population, 1976–1999

1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000

Year

18

2

4

6

8

10

12

14

16

Blo

od

Lea

d L

evel

s (m

g/d

L)

0

2.72.0

unleadedgasoline

introduced1979

unleadedgasoline

introduced1979

can solderphase-out

begins1978

can solderphase-out

begins1978

leadpaintban1976

leadpaintban1976

lead &copper

rule1991

lead &copper

rule1991

cansolderends1992

cansolderends1992

leadedgas

ends1996

leadedgas

ends1996

20

Data Source: NHANES II, III, 99+

Syndemics

Prevention Network

Lead-Based Paint in Housing

• 24 million housing units (25% of the nation’s housing) have significant lead-based paint hazards

• 1.2 million homes with significant lead-based paint hazards housed low income families with children under the age of 6

Source: National Lead-Based Paint Survey (1998-2000)Source: National Lead-Based Paint Survey (1998-2000)

Syndemics

Prevention Network

Puska P. The North Karelia Project: 20 year results and experiences. Helsinki: National Public Health Institute, 1995.

National Public Health Institute. North Karelia international visitor's programme. National Public Health Institute, 2003. Available at <http://www.ktl.fi/eteo/cindi/northkarelia.html>.

Navigational VenturesFinland’s North Karelia Project

Syndemics

Prevention Network

Puska P. The North Karelia Project: 20 year results and experiences. Helsinki: National Public Health Institute, 1995

Focusing the Intervention Policy

Policy A: Focus on High Risk Individuals

Policy B: Focus on Risk Conditions for All

Syndemics

Prevention Network

Broad Intervention PolicyNorth Karelia Project

Disease Burden

Disease Burden

Individual Effort

Public Work

Syndemics

Prevention Network

Directing ChangeNorth Karelia Project

Selected Action Strategies

• Medical services, if necessary

• Newspaper coverage: articles, editorials, letters

• TV time: highly rated 30-45 minute shows (no PSAs)

• Housewives’ organization: cooking and dietary choices

• Opinion leaders: role models, support groups, public action

• Tax shifting: tobacco, butter, milk

• Economic Renewal– Decline of dairy – Rise of berry – Rise of vegetable oil and rapeseed oil– Rise of healthier breads, cheeses, sausages, etc

Puska P. The North Karelia Project : 20 year results and experiences. Helsinki: National Public Health Institute, 1995.

Syndemics

Prevention Network

Transforming All Dimensionsof the System

Health

LivingConditions

Power toAct

Efforts to Fight Afflictions

Efforts to Improve Adverse Living Conditions

Efforts to Build Power

Syndemics

Prevention Network

Efforts to Fight Afflictions (design/deliver)

• Screening

• Education

• Risk reduction counseling

• Medical/pharmaceutical treatment

• Disease self-management


Syndemics

Prevention Network

Efforts to Improve Adverse Living Conditions (develop/promote)

• Tobacco legislation

• Food-labeling requirements

• Margarines and oils

• Low-fat milk

• Low-fat, low-salt, high-fiber bread

• Vegetable-containing sausage (with mushrooms)

• Berry farming and consumption

• Community competitions, morale, and social norms

• State welfare system (at the national, regional, sub-regional, and local levels)


Syndemics

Prevention Network

Health Professionals

• Physicians

• Health Educators

• Psychologists

• Epidemiologists

• Sociologists

• Hospital administrators

• Pharmaceutical manufacturers

• Nurses

• Rehabilitation therapists

Other Citizens

• Bakers

• Farmers

• Grocers

• Food scientists, manufacturers

• Restaurant owners

• Housewives

• Entertainers

• Entrepreneurs

• Journalists, media professionals

• Teachers

• School administrators

• Elected representatives

Building PowerNorth Karelia Project

Syndemics

Prevention Network

Charting ProgressNorth Karelia Project

Vartiainen E, Puska P, Pekkanen J, Toumilehto J, Jousilahti P. Changes in risk factors explain changes in mortality from ischaemic heart disease in Finland. British Medical Journal 1994;309(6946):23-27.

Syndemics

Prevention Network

Mortality changes in North Karelia in 1970-1995 (per 100 000, 35-64 years, men, age adjusted)

0

200

400

600

800

1000

1200

1400

1600

All Causes All CVD CHD All Cancers Lung CancerCause of Death

Ra

te p

er

10

0,0

00

1979

1995

-49%

-68%-73%

-44%-71%

Puska P. The North Karelia Project : 20 year results and experiences. Helsinki: National Public Health Institute, 1995.

National Public Health Institute. North Karelia international visitor's programme. National Public Health Institute, 2003. Accessed May 30, 2004 at <http://www.ktl.fi/eteo/cindi/northkarelia.html>.

Charting ProgressNorth Karelia Project

Syndemics

Prevention Network

Looking Forward Prospective Policy Evaluation

Featuring Systems Thinking & Modeling

Syndemics

Prevention Network

Milstein B, Homer J. The dynamics of upstream and downstream: why is so hard for the health system to work upstream, and what can be done about it? CDC Futures Health Systems Workgroup; Atlanta, GA; 2003.

TertiaryPrevention

SecondaryPrevention

PrimaryPrevention

TargetedProtection


Demand forresponse

PublicWork

SaferHealthierPeople Becoming

vulnerable


VulnerablePeople Becoming

afflicted

Afflictedwithout

Complications Developingcomplications

Afflicted withComplications


Health System Dynamics


GeneralProtection

Milstein B, Homer J. The dynamics of upstream and downstream: why is so hard for the health system to work upstream, and what can be done about it? CDC Futures Health Systems Work Group; Atlanta, GA; December 3, 2003.

Gerberding JL. CDC's futures initiative. Atlanta, GA: Public Health Training Network; April 12, 2004.

Gerberding JL. FY 2008 CDC Congressional Budget Hearing. Testimony before the Committee on Appropriations, Subcommittee on Labor, Health and Human Services, Education and Related Agencies, United States House of Representatives; Washington, DC; March 9, 2007.

Homer JB, Hirsch GB. System dynamics modeling for public health: background and opportunities. American Journal of Public Health 2006;96(3):452-458.

“One major task that CDC is intending to address is balancing this portfolio of our health system so that there is much greater emphasis placed on health protection, on making sure that we invest the same kind of intense resources into keeping people

healthier or helping them return to a state of health and low vulnerability as we do to disease care and end of life care."

-- Julie Gerberding

Syndemics

Prevention Network

• PossibleWhat may happen?

• PlausibleWhat could happen?

• ProbableWhat will likely happen?

• PreferableWhat do we want to have happen?

Bezold C, Hancock T. An overview of the health futures field. Geneva: WHO Health Futures Consultation; 1983 July 19-23.

“Most organizations plan around what is most likely. In so doing they reinforce what is, even though they want

something very different.”

-- Clement Bezold

Seeing Beyond the Probable

Syndemics

Prevention Network

Diseasemgmt

Riskmgmt

Riskprevention

Urgent &long-term care

A “Bathtub” View of Chronic Illness Dynamics

Low risk

High risk

Mildly ill

Severely ill

Risk onset

Illness onset

Complications onset

Death

Bathtubs = Accumulations = Stocks;

Drains & Faucets = Flows

Bathtubs = Accumulations = Stocks;

Drains & Faucets = Flows

Booth-Sweeney LB, Sterman JD. Bathtub dynamics: initial results of a systems thinking inventory. System Dynamics Review 2000;16(4):249-286.

Syndemics

Prevention Network

Re-Directing the Course of ChangeQuestions Addressed by System Dynamics Modeling

How?

Where?

0

10

20

30

40

50

1960-62 1971-74 1976-80 1988-94 1999-2002

Prevalence of Obese Adults, United States

Why?

Data Source: NHANES

20202010

Who?

What?

Simulation Experiments

in Action Labs

Syndemics

Prevention Network

Simulations for Learning in Dynamic Systems

Morecroft JDW, Sterman J. Modeling for learning organizations. Portland, OR: Productivity Press, 2000.

Sterman JD. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin McGraw-Hill, 2000.

Multi-stakeholder Dialogue

Dynamic Hypothesis (Causal Structure)

X Y

Plausible Futures (Policy Experiments)

Obese fraction of Adults (Ages 20-74)

0%

10%

20%

30%

40%

50%

1970 1980 1990 2000 2010 2020 2030 2040 2050

Fra

ctio

n o

f p

op

n 2

0-74

Syndemics

Prevention Network

A Model Is…

An inexact representation of the real thing

It helps us understand, explain, anticipate, and make decisions

“All models are wrong, some are useful.”

-- George Box


-- George Box

Syndemics

Prevention Network

CDC Obesity Dynamics Modeling Project Contributors

Core Design Team• Dave Buchner• Andy Dannenberg• Bill Dietz• Deb Galuska• Larry Grummer-Strawn• Anne Hadidx• Robin Hamre• Laura Kettel-Khan• Elizabeth Majestic • Jude McDivitt• Cynthia Ogden• Michael Schooley

System Dynamics Consultants• Jack Homer• Gary Hirsch

Time Series Analysts

• Danika Parchment

• Cynthia Ogden

• Margaret Carroll

• Hatice Zahran

Project Coordinator• Bobby Milstein

Workshop Participants• Atlanta, GA: May 17-18 (N=47)• Lansing, MI: July 26-27 (N=55)

Homer J, Milstein B, Dietz W, Buchner D, Majestic D. Obesity population dynamics: exploring historical growth and plausible futures in the U.S. 24th International Conference of the System Dynamics Society; Nijmegen, The Netherlands; July 26, 2006.

Cover of "The Economist", Dec. 13-19, 2003Cover of "The Economist", Dec. 13-19, 2003.

Syndemics

Prevention Network

Phase 2:

More Detailed Drivers of Change

Obesity Prevalence Over the Decades Two Broad Phases

Consequences Over TimeChanging Prevalence of

Four BMI Categories: 1970-2050

Dynamic Population Weight Framework(BMI Surveillance, Demography, and

Nutritional Science)

Policy Drivers(Trends & Interventions

Affecting Caloric Balance by Age, Sex, BMI Category, etc…)

Phase 1: Calculating Obesity Dynamics

Policy Drivers(Trends & Interventions

Affecting Caloric Balance by Age, Sex, BMI Category, etc…)

Syndemics

Prevention Network

Focus of Obesity Dynamics Simulation Model• Explore effects of new interventions affecting caloric balance

(intake less expenditure) – What are the likely consequences?

• How much impact on adult obesity?• How long will it take to see?• Should we target other subpopulations?

(age, sex, weight category)

• Consider two classes of interventions– Changes in food & activity environments – Weight loss/maintenance services for individuals

• Additional intervention details (composition, coverage, efficacy, cost) left outside model boundary for now– Available data are inadequate to quantify impacts and cost-effectiveness – Could stakeholder Delphi help?

Syndemics

Prevention Network

Obesity Dynamics Over the Decades Dynamic Population Weight Framework

Dynamic Population Weight Framework

Population by Age (0-99) and Sex

Flow-rates betweenBMI categories

Overweight andobesity prevalence

Birth Immigration

Death

CaloricBalance

Yearly aging

NotOverweight

ModeratelyOverweight

ModeratelyObese

SeverelyObese

Syndemics

Prevention Network

Obesity Prevalence Over the DecadesDynamic Population Weight Framework

NotOverweight


ModeratelyObese

SeverelyObese

NotOverweight


ModeratelyObese

SeverelyObese

NotOverweight


ModeratelyObese

SeverelyObese

Births Births Births Births

Age 0

Age 1

Age 99

No Change in BMI Category (maintenance flow)

Increase in BMI Category (up-flow)

Decline in BMI Category (down-flow)

Syndemics

Prevention Network

Information Sources

Topic Area Data Source

Prevalence of Overweight and Obesity

BMI prevalence by sex and age (10 age ranges)National Health and Nutrition Examination Survey (1971-2002)

Translating Caloric Balances into BMI Flow-Rates

BMI category cut-points for children and adolescents

CDC Growth Charts

Median BMI within each BMI category National Health and Nutrition Examination Survey (1971-2002)Median height

Average kilocalories per kilogram of weight change

Forbes 1986

Estimating BMI Category Down-Flow Rates

In adults: Self-reported 1-year weight change by sex and age

NHANES (2001-2002) *indicates 7-12% per year*

In children: BMI category changes by grade and starting BMI

Arkansas pre-K through 12th grade assessment (2004-2005) *indicates 15-28% per year*

Population Composition

Population by sex and ageU.S. Census and Vital Statistics (1970-2000 and projected)

Death rates by sex and age

Birth and immigration rates

Influence of BMI on Mortality

Impact of BMI category on death rates by age Flegal, Graubard, et al. 2005.

Syndemics

Prevention Network

(a) Overweight fraction

0%

20%

40%

60%

80%

1970 1975 1980 1985 1990 1995 2000 2005

Fra

ctio

n o

f w

om

en a

ge

55-6

4

NHANES Simulated

(b) Obese fraction

0%

10%

20%

30%

40%

50%

1970 1975 1980 1985 1990 1995 2000 2005

Fra

ctio

n o

f w

om

en a

ge

55-6

4

NHANES Simulated

(c) Severely obese fraction

0%

5%

10%

15%

20%

25%

1970 1975 1980 1985 1990 1995 2000 2005

Fra

ctio

n o

f w

om

en a

ge

55-6

4

NHANES Simulated

Reproducing Historical Trends One of 20 {sex, age} Subgroups: Females age 55-64

Note: S-shaped curves, with inflection in the 1990s

Syndemics

Prevention Network

Obesity Dynamics Over the DecadesTwo Classes of Interventions

Dynamic Population Weight Framework

Population by Age (0-99) and Sex

Flow-rates betweenBMI categories

Overweight andobesity prevalence

Birth Immigration

Death

CaloricBalance

Yearly aging

NotOverweight


ModeratelyObese

SeverelyObese

Trends and PlannedInterventions

Changes in the Physicaland Social Environment

Weight Loss/MaintenanceServices for Individuals

Syndemics

Prevention Network

Assumptions for Future Scenarios

Base Case• Caloric balances stay at 2000 values through 2050

Altering Food and Activity Environments

• Reduce caloric balances to their 1970 values by 2015

• Focused on

– ‘School Youth’: youth ages 6-19

– ‘All Youth’: all youth ages 0-19

– ‘School+Parents’: school youth plus their parents

– ‘All Adults’: all adults ages 20+

– ‘All Ages’: all youth and adults

Subsidized Weight Loss Programs for Obese Individuals

• Net daily caloric reduction of program is 40 calories/day (translates to 1.8 kg weight loss per year)

• Fully effective by 2010 and terminated by 2020

Syndemics

Prevention Network

Alternative FuturesObesity in Adults (20-74)

Obese fraction of Adults (Ages 20-74)

0%

10%

20%

30%

40%

50%

1970 1980 1990 2000 2010 2020 2030 2040 2050

Fra

cti

on

of

po

pn

20-

74

Base SchoolYouth AllYouth

School+Parents AllAdults AllAges

AllAges+WtLoss

Syndemics

Prevention Network

U.S. policy discourse is primarily focused on:

• Prevention among school-aged youth

• Medical treatment for the severely obese

Syndemics

Prevention Network

Findings & Limitations

• This model improves our understanding of obesity dynamics and supports pragmatic planning and evaluation– Traces plausible impacts of intervention and addresses questions of

whom to target, by how much, and by when

– Inflection point in obesity probably occurred during the 1990s

– Impacts of changing environments on adult obesity take decades to play out fully: “Carryover effect”

– Youth interventions have only small impact on overall adult obesity (assuming adult habits are determined primarily by adult environment)

– Effective weight-loss for the obese could greatly accelerate progress—but is there a realistic alternative to risky bariatric surgery?

• But it has limitations related to its narrow scope – Does not indicate exact nature of trends and interventions affecting

caloric intake, nor cost-effectiveness nor likely socio-political responses (reinforcing or resistant) of interventions

– Concentrating on detailed life stage data came at expense of a broader analysis of trends, interventions, and feedback effects

Syndemics

Prevention Network

Mokdad AH, Bowman BA, Ford ES, Vinicor F, Marks JS, Koplan JP. The continuing epidemics of obesity and diabetes in the United States. Journal of the American Medical Association 2001;286(10):1195-200.

Kaufman FR. Diabesity: the obesity-diabetes epidemic that threatens America--and what we must do to stop it. New York, NY: Bantam Books, 2005.

Syndemics

Prevention Network

CDC Diabetes System Modeling ProjectDiscovering Stock-Flow Dynamics Through Action Labs

Jones AP, Homer JB, Murphy DL, Essien JDK, Milstein B, Seville DA. Understanding diabetes population dynamics through simulation modeling and experimentation. American Journal of Public Health 2006;96(3):488-494.

Syndemics

Prevention Network

Project Background

• Diabetes programs face tough challenges and questions

– Pressure for results on disease burden, not just behavioral change

– Diabetes Prevention Program indicates primary prevention is possible, but may be difficult and costly

– What is achievable on a population level?

– How should funds be allocated?

• Standard epidemiological models rarely address such policy questions

• Starting Fall 2003, CDC initiates System Dynamics modeling project

• Starting Spring 2005, some states join as collaborators in further developing and using the SD model

Syndemics

Prevention Network

Inflow

Volume

Outflow

Developing

Burden ofDiabetes

Total Prevalence(people with diabetes)

Unhealthy Days(per person with

diabetes)

Costs(per person with diabetes)

People withDiagnosedDiabetes

Diagnosis Deaths

abPeople withPrediabetes

Developing

DiabetesOnset

c

d

People withNormal

Blood SugarLevels

PrediabetesOnset

Recovering fromPrediabetes

e

DiabetesManagement

DiabetesDetection

Obesity in theGeneral

Population

PrediabetesDetection &

Management

People withUndiagnosed

Diabetes

Deaths

Overview of Diabetes Stock-and-Flow Model

Syndemics

Prevention Network

Overview of Diabetes Stock-and-Flow Model

Inflow

Volume

Outflow

Developing

Burden ofDiabetes

Total Prevalence(people with diabetes)

Unhealthy Days(per person with

diabetes)

Costs(per person with diabetes)

People withDiagnosedDiabetes

Diagnosis Deaths

abPeople withPrediabetes

Developing

DiabetesOnset

c

d

People withNormal

Blood SugarLevels

PreDiabetesOnset

Recovering fromPreDiabetes

e

DiabetesManagement

DiabetesDetection

Obesity in theGeneral

Population

PrediabetesDetection &

Management


Diabetes

Deaths

Standard boundary

This larger view takes us beyond standard epidemiological models and most intervention programs

Syndemics

Prevention Network

Using Available Data to Ground the Model

Information Sources Data

U.S. Census

• Population growth and death rates• Fractions elderly, black, hispanic• Health insurance coverage

National Health Interview Survey• Diabetes prevalence• Diabetes detection

National Health and Nutrition Examination Survey• Prediabetes prevalence

• Obesity prevalence

Behavioral Risk Factor Surveillance System

• Eye exam and foot exam• Taking diabetes medications• Unhealthy days (HRQOL)

Professional Literature• Effects of risk factors and mgmt on onset, complications, and costs• Direct and indirect costs of diabetes

Syndemics

Prevention Network

One way we establish the model’s value is by looking at its ability to reproduce historical data

(2 variables out of 10 such comparisons)

Diagnosed diabetes per thousand total popn60

45

30

15

0

1980 1984 1988 1992 1996 2000 2004

Model

NHIS

Model

Diagnosed fraction of diabetes popn1

0.8

0.6

0.4

1980 1984 1988 1992 1996 2000 2004

NHANES IIINH

’99 -’00NH II

Homer J. Reference guide for the CDC Diabetes System Model. Atlanta, GA: Division of Diabetes Translation, Centers for Disease Control and Prevention; August, 2006. <<http://sustainer.org/pubs/diabetessystemreference.pdf>.

Syndemics

Prevention Network

Prevalence=92 AND RISING

Although we expect obesity to increase little after 2006, diabetes keeps growing robustly for another 20-25 years

Obese Fraction and Diabetes per Thousand1300.7

850.35

400

1980 1990 2000 2010 2020 2030 2040 2050Time (Year)

Diabetes Prevalenc

e

Obesity Prevalenc

e

Diabetes prevalence keeps growing after

obesity stopsWHY?

With high (even if flat) onset, prevalence tub

keeps filling until deaths (4-5%/yr)=onset

Onset=6.3 per thou

Estimated 2006 values

Death=3.8 per thou

Prevalence=92 / thou

Syndemics

Prevention Network

Unhealthy days impact of prevalence growth, as affected by diabetes management: Past and one possible future

Unhealthy Days per Thou and Frac ManagedObese Fraction and Diabetes per Thousand1300.7

850.35

400

1980 1990 2000 2010 2020 2030 2040 2050Time (Year)

Diabetes Prevalenc

e

Obesity Prevalenc

e

5000.65

25001980 1990 2000 2010 2020 2030 2040 2050

3750.325

Unhealthy Daysfrom Diabetes

Managed

fraction

Diabetes prevalence keeps growing after

obesity stops

If disease management gains end, the burden

grows

Syndemics

Prevention Network

Further Increases in Diabetes Management

People with Diabetes per Thousand Adults150

125

100

75

501980 1990 2000 2010 2020 2030 2040 2050

Monthly Unhealthy Days from Diabetes per Thou500

450

400

350

300

250

1980 1990 2000 2010 2020 2030 2040 2050

Base

Diab mgt Base

More people living with diabetes

Keeping the burden at bay for nine years

longer

Diab mgt

Increase fraction of diagnosed diabetes getting managed from 58% to 80% by 2015. (No change in the mix of conventional and intensive.) What do you think will happen?

Diabetes mgmt does nothing to slow the growth of prevalence—in fact, it

increases it. As soon as diabetes mgmt stops

improving, unhealthy days start to grow as fast as

prevalence.

Syndemics

Prevention Network

Managing Prediabetes AND Reducing Obesity

The more you reduce obesity, the sooner you

stop the growth in diabetes—and the more

you bring it down

… Same with the burden of diabetes


125

100

75

50

1980 1990 2000 2010 2020 2030 2040 2050


450

400

350

300

250

1980 1990 2000 2010 2020 2030 2040 2050

Base

PreD mgmt

PreD & Ob 25%

PreD & Ob 18%

Base

PreD mgmt

PreD & Ob 18%

PreD & Ob 25%

What do you think will happen if, in addition to PreD mgmt, obesity is reduced moderately by 2030? What if it is reduced even more?

Syndemics

Prevention Network

Intervening Effectively Upstream AND Downstream


125

100

75

50

1980 1990 2000 2010 2020 2030 2040 2050


450

400

350

300

250

1980 1990 2000 2010 2020 2030 2040 2050

Base

PreD mgmt PreD mgmt

Base

PreD & Ob 25%

Pred & Ob 25%

All 3 --PreD & Ob 25% & Diab mgmt

All 3

With a combination of effective upstream and downstream interventions we could hold the burden of diabetes nearly flat

through 2050!

With pure upstream intervention, burden still grows for many years before turning around. What do you think will happen if we add the prior diabetes mgmt intervention on top of the PreD+Ob25 one?

Syndemics

Prevention Network

Healthy People 2010 Diabetes Objectives:What Can We Accomplish?

-11%7.88.8 per 1,000

Reduce Diabetes–related Deaths Among Diagnosed

(5-6)

-38%2540 per 1,000

Reduce Prevalence of Diagnosed Diabetes

(5-3)

-29%2.53.5per 1,000

Reduce New Cases of Diabetes (5-2)

Increase Diabetes Diagnosis (5-4)

+18%80%68%

Percent Change

HP 2010 Target

Baseline

U.S. Department of Health and Human Services. Healthy People 2010. Washington DC: Office of Disease Prevention and Health Promotion, U.S. Department of Health and Human Services; 2000. http://www.healthypeople.gov/Document/HTML/Volume1/05Diabetes.htm

Syndemics

Prevention Network

20

30

40

50

60

70

1980 1985 1990 1995 2000 2005 2010

Pe

op

le w

ith

dia

gn

ose

d d

iab

ete

s p

er

1,0

00

Reported Simulated

Status Quo

Meet Detection Objective (5-4)

Meet Onset Objective (5-2)

HP 2010 Objective (5-3)

HP 2000 Objective

History and Futures for Diabetes PrevalenceReported Trends, HP Objectives, and Simulation Results

A

B

C

D

E

F

G

H

I

Milstein B, Jones A, Homer J, Murphy D, Essien J, Seville D. Charting plausible futures for diabetes prevalence: a role for system dynamics simulation modeling. Preventing Chronic Disease 2007 (in press).

Does this imply failure of the national policy?

Or a problem in the goal-setting process itself?

Does this imply failure of the national policy?

Or a problem in the goal-setting process itself?

Syndemics

Prevention Network

Connecting the ObjectivesPopulation Flows and Dynamic Accounting 101

It is impossible for any policy to reduce prevalence

38% by 2010!


Diabetes

People withDiagnosedDiabetes Dying from Diabetes

Complications

DiagnosedOnset

InitialOnset

PeoplewithoutDiabetes

As would stepped-up detection effort

Reduced death wouldadd further to prevalence

With a diagnosed onset flow of

1.1 mill/yr

And a death flow of 0.5 mill/yr

(4%/yr rate)

The targeted 29% reduction in diagnosed onset can only

slow the growth in prevalence

Milstein B, Jones A, Homer J, Murphy D, Essien J, Seville D. Charting plausible futures for diabetes prevalence: a role for system dynamics simulation modeling. Preventing Chronic Disease 2007 (in press).

Syndemics

Prevention Network

• All models, including simulations, are incomplete and imprecise

• But some are better than others and capture more important aspects of the real world’s dynamic complexity

• A valuable model is one that can help us understand and anticipate better than we do with the unaided mind

How Should We Value Simulation Studies?

Artist: Rene Magritte

Sterman JD. All models are wrong: reflections on becoming a systems scientist. System Dynamics Review 2002;18(4):501-531.

Meadows DH, Richardson J, Bruckmann G. Groping in the dark: the first decade of global modelling. New York, NY: Wiley, 1982.



-- George Box

Syndemics

Prevention Network

Sterman J. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin McGraw-Hill, 2000.

Barriers to Learning in Dynamic Systems

Syndemics

Prevention Network

But We Can Create Virtual Worlds for Learning

Sterman J. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin McGraw-Hill, 2000.

“In [dynamically complex] circumstances simulation becomes the only reliable way to test a hypothesis and evaluate the likely effects of policies."

-- John Sterman

Syndemics

Prevention Network

Learning In and About Dynamic Systems

Benefits of Simulation

• Formal means of evaluating options

• Experimental control of conditions

• Compressed time

• Complete, undistorted results

• Actions can be stopped or reversed

• Tests for extreme conditions

• Early warning of unintended effects

• Opportunity to assemble stronger support

• Visceral engagement and learning

Complexity Hinders

• Generation of evidence (by eroding the conditions for experimentation)

• Learning from evidence (by demanding new heuristics for interpretation)

• Acting upon evidence (by including the behaviors of other powerful actors)

Sterman JD. Learning from evidence in a complex world. American Journal of Public Health (in press).

Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. Boston, MA: Irwin McGraw-Hill, 2000.

“In [dynamically complex] circumstances simulation becomes the only reliable way to test a hypothesis and evaluate the likely effects of policies."

-- John Sterman

Syndemics

Prevention Network

“Simulation is a third way of doing science.

Like deduction, it starts with a set of explicit

assumptions. But unlike deduction, it does not

prove theorems. Instead, a simulation generates

data that can be analyzed inductively. Unlike

typical induction, however, the simulated data

comes from a rigorously specified set of rules

rather than direct measurement of the real world.

While induction can be used to find patterns in

data, and deduction can be used to find

consequences of assumptions, simulation

modeling can be used as an aid to intuition.”

-- Robert Axelrod

Axelrod R. Advancing the art of simulation in the social sciences. In: Conte R, Hegselmann R, Terna P, editors. Simulating Social Phenomena. New York, NY: Springer; 1997. p. 21-40. <http://www.pscs.umich.edu/pub/papers/AdvancingArtofSim.pdf>.

Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. Boston, MA: Irwin McGraw-Hill, 2000.

Simulation ExperimentsOpen a Third Branch of Science

“The complexity of our mental models vastly exceeds our ability to understand their implications without simulation."

-- John Sterman

How?

Where?

0

10

20

30

40

50

1960-62 1971-74 1976-80 1988-94 1999-2002

Prevalence of Obese Adults, United States

Why?

Data Source: NHANES 20202010

Who?

What?

Syndemics

Prevention Network

An Alternative Philosophical Tradition

Shook J. The pragmatism cybrary. 2006. Available at <http://www.pragmatism.org/>.

Addams J. Democracy and social ethics. Urbana, IL: University of Illinois Press, 2002.

West C. The American evasion of philosophy: a genealogy of pragmatism. Madison, WI: University of Wisconsin Press, 1989.

"Grant an idea or belief to be true…what concrete difference will its being true make in anyone's actual life?

-- William James

Pragmatism• Begins with a response to a perplexity or injustice

in the world• Learning through action and reflection• Asks, “How does this work make a difference?”

Positivism • Begins with a theory about the world• Learning through observation and falsification• Asks, “Is this theory true?”

We are not talking about theories to explain, but conceptual, methodological, and moral orientations: the frames of reference

that shape how we think, how we act, how we learn, and what we value

Syndemics

Prevention Network

A Navigational View of Public Health Work

Thompson N. Reflections on voyaging and home. Polynesian Voyaging Society, 2001. Accessed July 18 at <http://leahi.kcc.hawaii.edu/org/pvs/malama/voyaginghome.html>.


Where we want to go?

How do we prepare to get there?

Where do you want your children to live?

Where you do want to live?

Syndemics

Prevention Network

A Navigational View of Public Health Work

"How do you know," I asked, "that in twenty years those

things that you consider special are still going to be

here?" At first they all raised their hands but when they

really digested the question every single one of them

put their hands down. In the end, there was not a single

hand up. No one could answer that question.

It was the most uncomfortable moment of silence that I

can remember…That was the defining moment for me. I

recognized that I have to participate in answering that

question otherwise I am not taking responsibility for the

place I love and the people I love.”

-- Nainoa Thompson

Thompson N. Reflections on voyaging and home. Polynesian Voyaging Society, 2001. Accessed July 18 at <http://leahi.kcc.hawaii.edu/org/pvs/malama/voyaginghome.html>.


Policy Evaluation

Documents

Transcript of Policy Evaluation