Dynamic Modeling Frontiers: A Brief Glimpse Modeling Frontiers: A Brief Glimpse Nathaniel Osgood...

Dynamic Modeling Frontiers: A Brief Glimpse

Nathaniel Osgood

Using Modeling to Prepare for Changing Healthcare Needs

Duke-NUS

April 16, 2014

Some Modeling Frontiers • Multi-scale & hybrid

modeling

• Linking models & data (especially “big data”)

• Linking dynamic simulation with computational statistical tools – Predictor-corrector models

– Posterior derivation

• Real-time predictive simulation

• Model software engineering – Interfaces

– Aspects & Observer processes

– Mocking

– Testing

• Empirically grounded, richer models of agent choices

• Model specification & domain specific languages

• Planning experiments using modeling

• Mathematical analysis tools – Loop gain, Eigenvalue Elasticity,

&etc.

• Numerical analysis considerations (t, integ. Methods)

• Alternative mathematical formalisms (hybrid automata, differential equation variants, DAEs, etc.)

• Addressing performance challenges & parallelization

• Eval. study design & stats w/synth. pop experiments

System Science Methodologies: Highly Complementary

• Different modeling methodologies seek to answer different types of questions

• No one system science methodology offers a replacement for the others

• Significant synergies can be secured by using combinations of methodologies to address the same problem – As cross-checks on understanding where two or more

can be applied

– Exploiting competitive advantages

Multi-Framework Modeling • We have found the use of multiple frameworks

highly effective

– Co-evolving multiple models for

• Cross-validation

• Asking different sorts of questions

• Revealing new questions to answer

– Within a single model

• Dealing with questions at different scales

• Improving robustness of models

• Allowing for representation & changing of factors that are otherwise ignored

Lateral Stock & Flow Model and ABM Integration

Agent Population

Agents Driving Stock & Flow Model

Agent Population (Companies)

Stocks and Flows within Agents

Uninfected

Cells

Infected

Cells

Virus Load

Uninfected Cell

Replentishment

New Cell

InfectionsUninfected Cell

death

Infected Cell

Death

Virion Production

From Infected Cells

Virion Clearance

Uninfected Cell

Replentishment Rate

Mean Infected Cell

Lifetime

Mean Uninfected

Cell Lifetime

Mean Virion

Lifetime

Likelihood Density of

Infection by Single Virion

Per Infected CellVirion

Production Rate

Virion Production Rate

Per Contact Virions Rate1 Person

Mean Viral Load<Population Size>

Mean Uninfected

Cells

Mean Infected

Cells<Population Size>

<Population Size>

Mean of Viral Load

of Neighbors

CTLs

immune response to

infected cellsCTL turnover

CTL

responsiveness

Mean CTL

lifespan

infected cell death

by CTLs rate which infected cells

are killed by CTLs

Virion Production Rate if

Non Quantized Infection

Spatial Patterning

Hybrid ABM-Discrete Event Modeling


modeling







– Mocking

– Testing





&etc.





Context: Increasingly Rich Decision-Oriented Simulation Models

• Many uses of computational models involving human health & behavior require copious data

• Such models help us understand the implications of such data for decision making

• Stratified aggregate models: Detailed cross-sectional views of a population’s health

• Individual based models: Detailed longitudinal & cross-sectional views of a population’s health – Strong motivations: Capturing history, network position,

spatial dynamics, rich heterogeneity • Individual trajectories • Interventions design • Calibration

• Modeling as theory building: Broader & more detailed models typically involve more articulated theories

A Key Challenge: Reliable Data on Key Health Behaviors are Often Hard to Secure • Location (access to care, access to resources,

barriers to activity, environmental risks)

• Physical activity (obesity, T2DM & GDM, risk of falls)

• Spatial proximity (transmission of pathogens and norms, imitative behavior, communication)

• Social context (norms, communication, perception of safety, risk perception)

• Communication: Personal & mass media (risk perception, norms, beliefs, social cues)

• Decision-making rules & heuristics

Absent understanding of such behaviors,

the potential to quantitatively

evaluate policy tradeoffs is greatly limited

Ubiquity of Sensors Smartphones are Amazing Devices

• Seamlessly connect/failover to whatever network is

available

• Track path of morning run or in car

• Take pictures

• Record a lecture

• Reorient when orientation changed

• Interact with printers, computers, TVs, etc.

• Slow down when battery is getting low

• Alert you to nearby attractions .

• Detect & deactivate when battery is too hot

Ubiquity of Sensors Smartphones are Amazing Devices

(A Key Enabling Technology: Sensors) • Seamlessly connect/failover to whatever network is

available (WiFi/GPRS/GSM receivers)

• Track path of morning run or in car (GPS)

• Take pictures (Camera)

• Record a lecture (Microphone)

• Reorient when orientation changed (Accelerometers)

• Interact with printers, computers, TVs, etc. (Bluetooth)

• Slow down when battery is getting low (Battery voltage)

• Alert you to nearby attractions (GPS & Internet access)

• Detect & deactivate when battery is too hot (battery temperature)

Such sensors can be repurposed at little or no cost to their original function

An Enveloping Wireless Cacophony

• WiFi, Bluetooth, GPS, GPRS/GSM, Infrared, RFID, etc. are routinely used for communication

• Such signals can also be repurposed – at virtually no cost – to provide other information

– Indoor/outdoor location

– Inter-individual proximity

– Broad contextual disambiguation

Our Generation 2 Sensing Platform: iEpi • Google Android Smartphone

– Android 4 app or Customized version of Android 2.1

– Commodity hardware (HTC) Lower price

• Richly functional smartphone Internal incentives to carry & charge device

• Episodic data collection bursts prolong battery life

• Diverse sensor modalities & context-triggered surveys

• Opportunistic data backhaul

iEpi System Architecture

Secure opportunistic backhaul using WiFi or Cellular connection (Alternative: Manual upload)

On Smartphone

Some Key Health Areas Addressed

• Location (access to care, access to resources, barriers to activity, environmental risks)

• Physical activity (obesity, T2DM & GDM, risk of falls)

• Spatial proximity (transmission of pathogens and norms, imitative behavior, communication)

• Social context (norms, communication, perception of safety, risk perception)

• Communication: Personal & mass media (risk perception, norms, beliefs, social cues)

Ubiquitous Sensors and Dynamic Models: A Natural Synergy

Big Data:

Grounding the Models

• Rich grounding in observations

• Providing databases for model parameterization & calibration

• Stimulating dynamic hypotheses

Dynamic Models:

“Making Sense” of the data

• Linking to decision outcomes

• “Filling the gaps” between sensor data

• Capturing regularities that underlie sensor data

• “Filtering” of noisy sensor data – Arriving at “consensus” estimates

combining measured data & model predictions

• Understanding proximal & distal implications of observed behavior

• Determining adaptive sampling rates

Uninfected People

Low-Risk Latently Infected People

Active UnDx Non-Infectious TB

Latent

Infection

Reactivation of

LR Latents

Latently Infected

Immigrants

Non-TB Death of

Uninfected

Non-TB Death of

Low-Risk Latents

Non-TB Death of Active

Undiagnosed People

Death Due to TB ofNon-infectious Actives

Latently Infected People Protected Via TLTBI

Waning of

Protection

Mean Time Until Waning of

TLTBI Immunity

LTBI Treatment Count

For Current Time

Historical Dynamic Non-TB

Death Rate For Current Time

<Historical DynamicNon-TB Death Rate For

Current Time>


Current Time>

Historical Dynamic TBDeath Rate For Current

Time

Non-TB Death of

TLTBI Protected

People


Current Time>

Uninfected

Immigrants

<Rate of New

Infection>

Natural

Recovery RateProportion Infected

Prior to Entry

Rate of Progression from

Infection to Active TB

Rate of

Reactivation

Active TB Cases Under Treatment

Treatment Default

with Active TB

Mean TimeUnder Treatment

Annual Likelihood

Density of Default

BCG Vaccinated People by Ethnicity

Vaccination

Mean Time Until

Waning of ImmunityNon-TB Death of

Vaccinated People


Current Time>


Cases Under Treatment


Current Time>

Mean Time UntilDiscovery of UnDx

Infectious TB

New Births

Default Rate Anytime

During Treatment

Annual Likelihood Density

of Treatment Completion

Latently Infected People With Previous TreatmentActive Undiagnosed Non-Infectious TB Cases With

Previous Treatment

Relapses

Natural Recovery

of PT

Diagnosis ofNon-InfectiousRelapsed Cases

Treatment

Completion

Death Due To TB of

Previously Treated

Undiagnosed Active

Cases

Non-TB Death of

PT Latents

Mean Time Until Discovery ofNon-Infectious UnDxPreviously Treated TB

<Natural Recovery

Rate>


Current Time>

Natural

Recovery

Historical NetInmigration For SK By

Ethnicity

Historical NetInmigration for Current

Time

<Time>

Historical Birth Count

For Current Time

<Time>

Historical SK Birth

Count For Year

Non-TB Death of

PT Active


Current Time>

Historical Dynamic

TB Death Rate

<Time>

Historical Total Annual

TLTBI Administered

Total Historical BCG

Vaccination Count For Year

Total Historical BCG

Count For Current Time

<Time>

Historical Dynamic

Non-TB Death Rate

<Time>

Active UnDx Infectious TB

Diagnosis of

Infectious CasesDevelopment of

InfectiousnessMean Time Until

Becoming Infectious

Death Due To TB of

UnDx Infectious Cases

High-Risk Latently Infected PeopleLTBI Treatment

Decrease in Risk

for Disease

Development of Primary TB

Disease in High-Risk Latent

Individuals

Mean Time until

Lowering of Risk

Non-TB Death of

High-Risk Latents

Diagnosis of Non-Infectious Cases

Mean Time Until Discovery

of Non-infectious Cases

<Historical NetInmigration for Current

Time>

<Proportion Infected

Prior to Entry>

Historical Dynamic

Default Rate

Historical DynamicDefault Rate For Current

Time

<Time>

Historical Dynamic

Fraction of BCG

Vaccinations that

Went to RI

Historical Fraction of BCGDelivery going to Ethnicity for

Current Time

Development of Primary TB

in Protected Individuals with

LTBI

Protected People with

LTBI at Risk for Active

Disease

<Rate of Progression from

Infection to Active TB>

<Rate of New

Infection>

Proportion of Protected LTBI

at Risk for Primary Disease

High-Risk LatentsSusceptible to Exogenous

Reinfection

<Historical Dynamic TBDeath Rate For Current

Time>

Proportion of PeopleCompleting TLTBI for

Current Time

Estimated Proportion

Completing TLTBI in 1975

Proportion of PeopleCompleting TLTBI for

Year

Starting Year of Historicdata for Completion of

TLTBI

Time Elapsed

from 1975

Slope of Interpolation Line

between 1975 and 1985

<Proportion of Contacts

in Each Ethnicity>

Historical Total Annual TLTBIAdministered To Each Ethnicity

for Current Time

Historical Total AnnualTLTBI Administered for

Current Time


Current Time>

TLTBI

Intervention

BCG Intervention

Immigration

Intervention

<Time> <Immigration

Intervention>

Use Calibrated

Relapse Rate?

Calibrated

Relapse Rate

Previously Treated

Death Rate Coefficient

Death Due To TB of

Dx Infectious Cases

<HistoricalDynamic TB

Death Rate ForCurrent Time>

Treatment Default

with Latent TB

Fraction of TreatmentDefault Occuring with

Active TB

Treatment Default

Rate by Ethnicity

Exogenous Reinfection

of LR Latents

<Low-Risk Latents at

Risk of Reinfection>

Reinfection of

Previously Treated

<Proportion of Latently InfectedIndividuals Susceptible toExogenous Reinfection>

<Rate of New

Infection>

Fraction of New InfectionsLeading to Primary

Progression

<Previously Treated

Death Rate Coefficient>

Mean time till discovery

of infec TB intervention

Mean time till discovery of

non-infec TB intervention

<Time>

Mean time till

waning of BCG

Mean time till waning of

TLTBI intervention

Infection of BCG

Protected People

RR for Infection

Conferred by BCG

<Rate of New

Infection>

Waning of

Immunity

<Time>

Active Undiagnosed Infectious TB Cases

With Previous Treatment

Development ofInfectiousness Among

Previously Treated Actives

Diagnosis of Infectious

Relapsed Cases

Mean Time Until Discovery ofUnDx Infectious TB for

Current Time

<Time>

Death Due to TB of Previously Treated Undiagnosed Infectious TB Cases

Non-TB Death ofInfectious Previously

Treated Actives


UnDx Infectious TB Cases

Diagnosis of both Infectiousand Non-Infectious Relapsed

Cases

Coefficient for Mean Time UntilDiscovery of Non-Infectious UnDx

TB for Previously Treated

Mean Time Until Discovery ofUnDx Infectious TB for Previously

Treated for Current Time

Coefficient for Mean Time UntilDiscovery of Infectious UnDx TB

for Previously Treated

Coefficient for Mean Time UntilDiscovery of All UnDx TB forPreviously Treated by Ethnicity

Coefficient for Mean Time UntilDiscovery of All UnDx TB for

Previously Treated

Mean Time Until Discovery ofNon-infectious Cases for

Current Time

<Time>

<Mean Time Until Discoveryof Non-infectious Cases for

Current Time>

<Mean Time Until

Becoming Infectious>


Current Time>

Interfacing Models and Dynamic Data: Strategies We’ve Explored

• Parameterizing models to data (e.g. driving contact, mobility patterns)

• Calibrating models to data (generative models w/similar emergent mobility or contact patterns)

• Empirical data based inferencing (e.g. underlying infection transmission pathways)

• Training models on data (e.g.classification schemes)

• Use of data as “ground truth” to evaluate inference schemes & evaluate effectiveness of study designs

Some of Our Modeling Initiatives • H1N1 & ILI transmission (ABM & aggregate)

• Multi-scale (hybrid) models

– Diabetes: (Stock & Flow) Blood glucose dynamics (Sensed: Blood glucose spot checks, insulin, physical activity, sedentary behavior, food)

– Body weight & composition dynamics: (Sensed: Food intake, physical activity, sedentary behavior, weight)

• Mobility models

• Anticipated: Opinion dynamics (cf Sandia’s tobacco use), random utility theory

• Key theme: “Self-Correcting” models using predictor-corrector methods (e.g. MCMC, SMC,KF)


modeling







– Mocking

– Testing





&etc.





Predictor Corrector Models Benefits of Synergizing Models & Ongoing Measurement via “Closed Loop Models”

Benefits to Data

• Interpreting for implications to other areas of the system not directly measured

• Understanding implications for decision making

• Separating signal from noise: Avoiding overconfidence in measurements

• Generalization/abstraction to broader dynamic patterns of behavior

Benefits to Models

• Preventing model state divergence from actual situation

• Maintaining model “freshness” by repeated re-grounding in measured data

• Better understanding of current situation

• More reliable prospective simulation with the model

• Avoiding overconfidence in model output

Predictor-Corrector Methods: Computational Statistics & Systems Science

Noisy Data

Fallible Model

Predictions

Consensus Estimates

• Uassisted, all models eventually diverge from empirical situation • Rough & ready model quickly available to support decision-making, automatically

regrounded & sharpened may be both more valuable and accurate than a far more detailed model that takes longer to create

Predictor Corrector Methods

• “Brute force”

– Parameter point estimates: Recalibrate based on updated data

– Posterior distribution over parameters: Run full (Batch) Monte Carlo (e.g. MCMC) for each new point

• Recursive (Incremental computation)

– Point estimates: Kalman Filter

– Posterior distribution over parameters: Sequential Monte Carlo/Particle Filtering

Rudolf Emil Kalman (1930-)

A More Basic Example The Kalman Filter (R. E. Kalman 1960)

The ongoing discrete Kalman filter cycle. The time update projects the current state estimate ahead in time. The measurement update adjusts the projected estimate by an actual measurement at that time. (Welch, G. and Bishop, G. 2006)

Slide courtesy of Weicheng Qian

Dissecting the Kalman Filter

Measurements Classic Model

Estimation


Higher level view of Kalman Filter

Fixed

Parameter Values


Traditional Parameter Estimation via Calibration in (Dynamic) Simulation Models • Point estimation of parameter values

• Optional: Estimation of confidence intervals surrounding estimate

• Limitations – Single dynamic model assumed

– Single point estimates (doesn’t make apparent how this yields estimate in certain output measures – e.g. comparison between interventions)

– Even with estimate of intervals, assuming that unimodal • No understanding of global shape of likelihood

– Error metric for estimation (e.g. square of estimate) imposes distributional assumptions on errors

Prior & Posterior Parameter Distributions • Before calibration, we often have some sense

as to where the parameters fall – We can encode this with a “prior” distribution

• This is called a “prior” because we can formulate it prior to observing the data or settling on a model

• Calibration can give us an updated distribution, which we call the “posterior” distribution – This takes into account not only our best guesses

for the parameter values, but also the likelihood that the model(s) used could explain the observed data given certain parameter values

Posterior Distributions for Parameters (): Moving Beyond Point Estimates

• A posterior distribution over helps us understand the relative probabil. of parameter values in light of

– The observed data (y)

– One or more models under consideration

– Any pre-existing expectations of the relative likelihood of different parameter values (a prior distribution over )

• Given a posterior over , we can e.g. derive

– Point estimates (MLE, mean, Minimum var, &c) of parameters & model output

– Likelihood intervention A is “better” than intervention B

– “Credibility intervals” for diverse model output

High-Level Operation of Using MCMC for Parameter Estimation of Simulation Model • Elements to specify before algorithm to use in

algorithm – Assign prior distributions (over parameters, models, etc.)

may or may not be parametric (using ‘hyperparameters’) – Formulate likelihood formulae that indicate relative

likelihood of seeing particular values of empirical data (y) in light of values of parameters () & simulation model output • These could be either parametric or non-parametric

– Conceptually (only), apply Bayes’ rule to get rule for likelihood of parameter in terms of data (y) and priors

• A Markov Chain is used to generate samples of parameters (), with (asymptotic stationary) distribution proportional to posterior distribution – For each generated parameter sample, we sample output

measures of interest & accumulate statistics, etc.

2D Projection of Density from Higher Dimensional Sampling

0.032 0.034 0.036 0.038 0.04 0.042

0.2

0.4

0.6

0.8

mcmcBounded1MRuns[, 1]

mcm

cB

ou

nd

ed

1M

Ru

ns[,

2]

1

151

300

450

599

749

898

1048

1198

1347

1497

1646

1796

1945

2095

2244

2394

Counts


modeling







– Mocking

– Testing





&etc.





Improved Agent Based Languages • System Dynamics software is widely

distinguished by its support for a declarative model specification

– We specify the “what” of the model – the framework figures out the “how”

– This allows for a sustained attention on the problem domain

• ABMs still make use of general-purpose programming language

– Implementation (e.g. Software engineering) concerns distract from and obscure the model specification

• Much promise exists for declarative ABM langs.

Towards Declarative Specification of ABMS

Vendrov, Dutchyn, Osgood 2014


modeling







– Mocking

– Testing





&etc.





Evaluating Using a Synthetic Population

• Analytic approaches (and study designs) are often challenging and costly to test in the real world – Expensive to establish study

– Time consuming

– Ethical barriers

– Lack of definitive knowledge of how conclusions compare to some “ground truth”

• We can often evaluate such approaches using “synthetic populations” drawn from simulation models – Here, the simulation model helps to identify potential

weaknesses of study designs & analysis approaches

Synthetic Population Studies

– Establish a “synthetic population” for a “virtual study”

– Perform simulation, simulating study design of interest

• Actual underlying situation is blinded from researcher

• Collect data from the synthetic population similar to what would collect in the external world

• Optionally, may actually simulate roll out and dynamic decision protocols

– Analysis procedures being evaluated are applied to the data from the synthetic population

– We compare the findings from those analysis procedures to the underlying “ground truth” in the simulation model

Example: Evaluating Particle Filtering or Kalman Filtering

Agent-Based Model Using Sensor Data

Simulation Measured Data (Estimates of count of Susceptibles, Infectives Recovereds)

Kalman Filtering or Particle Filtering

Aggregate System Dynamics SIR Model

Updated System Dynamics Model Assumptions

“Synthetic ground truth” for evaluation of filtered output


modeling







– Mocking

– Testing





&etc.





Representing Adaptive Agent Decision Making and Learning

• ABMs are highly attractive in representing situated agents – Such agents can in principle make decisions based

on local contacts

• Most of our ABMs have featured agent “automatons” with little adaptation in behavior based on environment or leaning

• Tools such as Random Utility Theory can help us represent agent decision making

RUT & ABM: Complementary

Discrete Choice Theory/RUT

• Grounded understanding of varied choice behavior – Diverse heuristics

– Boundedly rational

– Rational

• Statistical tools for estimating preferences from – Stated preferences

– Revealed preferences

• Sophisticated stated preference elicitation mechanisms – Smaller # questions, respondents

ABM

• Changing choice sets

• Experiences that change – Agent confidence about

evaluations

– Preferences

• Capturing influence of other people on agent preferences

• Capturing real-world responses to choices – e.g. in context of limited resources

Mair, 2014

Performance

• Model scaling

• Concurrency

– Problem decomposition

– Embarrasing parallelism

• Monte Carlo ensembles for stochastic

• Sensitivity analysis

• Language impact on parallelizability

Other ABM Modeling Frameworks

• ABM

– Repast Simphony

– Netlogo

• Others

– SD: Vensim/Powersim/Ithink/Berkeley Madonna

– Insightmaker

– http://www.systemswiki.org/index.php?title=Health_Care_System_Dynamics_Insights

http://www.systemswiki.org/index.php?title=Health_Care_System_Dynamics_Insights

http://www.systemswiki.org/index.php?title=Health_Care_System_Dynamics_Insights

Books • ABM – Railsback, S.F. and Grimm, V. Agent-Based and Individual-

Based Modeling: A Practical Introduction. Princeton: Princeton University Press. ISBN 978-0-691-13674-5. 2012

– Epstein, J.. Computational Social Science.

• Networks – Newman, M. Structure & Dynamics of Networks

– Valente. Social Networks in Health.

– Popular: • Strogatz,S. Sync

• Barabasi. Linked.

• Watts. Small Worlds.

• System Dynamics – J. Morecroft.

– Sterman, Business Dynamics.

– Ford. Modeling the Environment.

Generativist Perspective

• Epstein, Joshua M. "Remarks on the foundations of agent-based generative social science." Handbook of computational economics 2 (2006): 1585-1604.

• Epstein, Joshua M. Generative social science: Studies in agent-based computational modeling. Princeton University Press, 2006.

• Grimm, Volker, et al. "Pattern-oriented modeling of agent-based complex systems: lessons from ecology." Science 310.5750 (2005): 987-991.

• Axelrod, Robert M. The complexity of cooperation: Agent-based models of competition and collaboration. Princeton University Press, 1997.

Conferences

• Winter Simulation Conference – With 2012 hosting the first meeting of the AnyLogic

Users’ Group

• Microsimulation International

• International Conference of the System Dynamics Society

• Social Computing, Behavioral-Cultural Modeling and Prediction (SBP)

• International Health Informatics Symposium (IHI)

Community Contributed Content

• AnyLogic Users Group on LinkedIn

• Health regions of SystemsWiki

• ABM Bootcamp Google Group

Dynamic Modeling Frontiers: A Brief Glimpse Modeling Frontiers: A Brief Glimpse Nathaniel Osgood...

Documents

Transcript of Dynamic Modeling Frontiers: A Brief Glimpse Modeling Frontiers: A Brief Glimpse Nathaniel Osgood...