1.3 Modeling process and outcomes Walter Sermeus, RN, PhD Catholic University Leuven Belgium Witten,...
-
Upload
thomasine-baker -
Category
Documents
-
view
222 -
download
0
Transcript of 1.3 Modeling process and outcomes Walter Sermeus, RN, PhD Catholic University Leuven Belgium Witten,...
1.3 Modeling process and outcomes
Walter Sermeus, RN, PhD
Catholic University Leuven
Belgium
Witten, Fri 02.07.10Session 1: 11:00-12:30Session 2: 13:30-15:00
The European Academy of Nursing Science 2
Background: MRC CI-framework
Modeling
The European Academy of Nursing Science 3
What does the MRC-CI framework say on Modelling process and outcomes?
• Modelling a complex intervention prior to a full scale evaluation can provide important information about the design of both the intervention and the evaluation.
• One useful approach to modelling is to undertake a pre-trial economic evaluation. This may identify weaknesses and lead to refinements, or it may show that a full-scale evaluation is unwarranted, for example because the effects are so small that a trial would have to infeasibly large.
• Formal frameworks for developing and testing complex interventions, such as MOST or RE_AIM may be a good source of ideas, and the National Institute for Health and Clinical Excellence has produced detailed guidance on the development and evaluation of behaviour change
The European Academy of Nursing Science 4
What will we do in this session ?
• What & why
• Models – Tools
• Economic evaluation
• Examples
• Excercises
The European Academy of Nursing Science 5
What kind of models ?
• No scale models
• Rather causal models:– A causal model is an abstract model that
uses cause and effect logic to describe the behaviour of a system.
– Focus on understanding, relationships, which effects might be expected
– Using theories, from various disciplines
Why models ?
• Refining conceptual models– influences, components, relations,
consequences
• Generating (tentative) estimates of effect size
• Identifying barriers
• Optimising combinations of components of the intervention
The European Academy of Nursing Science 6
(Source: Campbell et.al., BMJ, 2007)
Relation between context, problem definition, intervention and evaluation
The European Academy of Nursing Science 7
(Source: Campbell et.al., BMJ, 2007)
MODELS: PRIME
• PRocess Modelling in ImpleMEntation research– Identifying active ingredients in professional behaviour
change– Process modelling: understanding of factors underlying
clinical practice, in order to identify what sorts of processes should be targeted in implementation interventions
– 4 levels: individuals, teams, organisations, systems– Different kind of theories
The European Academy of Nursing Science 8
(Source: Walker AE et.al., BMC HSR, 2003)
Example of PRIME Theories - individual level
The European Academy of Nursing Science 9
Normalization process model
• Theoretical framework for understanding complex interventions
• Normalization vs adoption / rejection• 4 dimensions:
– Interactional workability: effect on interactions between people and practices
– Relational integration: relation to existing knowledge and relationships
– Skill-set workability: effect on current division of labour– Contextual integration: relation to the organisation
The European Academy of Nursing Science 10
May C, BMC Health Services Research, 2006
Example of NPM to telehealth services
The European Academy of Nursing Science 11May C, BMC Health Services Research, 2006
Tools: RE-AIM
Dimension Level Description
Reach Individual Proportion of target population that participated in the intervention
Efficacy / effectiveness
Individual Success rate; positive / negative outcomes
Adoption Organization Proportion of settings that will adopt the intervention
Implementation Organization Extent to which intervention is implemented as intented in real world
Maintenance I&O Extent to which a program is sustained over time
The European Academy of Nursing Science 12
Source: Glasgow R. et.al., 1999 (www.re-aim.org)
The goal of RE-AIM is to encourage program planners, evaluators,funders, and policy-makers to pay more attention to essential program elements including external validity that can improve the sustainable adoption and implementation of effective, generalizable, evidence-based interventions.
TOOLS : MOST
• Multiphase Optimization Strategy– Screening phase:
• Components that are candidates for inclusion in an intervention (active components)
• Based on significant effect, effect size, cost,…
– Refining phase:• Fine tuning e.g. optimal level
– Confirming phase:• Evaluating the intervention as a package
The European Academy of Nursing Science 13
(Source: Collins et.al. Am J Prev Med, 2007)
Economic evaluation
Are Costs and Outcome Examined? outcomes costs Costs &
outcomes Comparison
of two alternatives?
No Outcome description
Cost description
Cost-outcome description
Yes Effectiveness evaluation
Cost-analysis
Full economic evaluation
Source: R. Taylor, Masterclass Complex Interventions, Exeter 2010
Incremental Cost Effectiveness Ratio (ICER)
cost [complex intervention – usual care]
outcome [complex intervention – usual care]
ICER =
Quality adjusted life years (QALYs)
Cost utility analysis (CUA)
Utility
Life expectancy
10
0.7
1
A
B
Intervention A = 5 QALYs
perfect health
death 0
Intervention B = 7 QALYs
QALY difference = +2 QALYs
0.5
EQ-5D(EuroQoL)
(www.euroqol.org)
EQ-5D scoring guide
Example of Modelling exercise
• ProActive causal model (Hardeman et.al., Health Education Research, 2005)
• Aim: increase physical activity among individuals at risk of Type 2 diabetes
• The model provides:– Rational guide for appropriate measures– Intervention points– Intervention techniques
The European Academy of Nursing Science 19
Generic causal model
• Epidemiology– Defining health outcomes and precise objective measures,
target group, likely impact of achievable behaviour change on physiological and biochemical variables
• Psychology– Theory-based determinants, intervention points, techniques
to support behaviour, measures of change in behavioural determinants
The European Academy of Nursing Science 20
Methods used
The European Academy of Nursing Science 21
Theory of Planned Behaviour (Ajzen 1991)
belie
fs
Results : ProActive causal model
The European Academy of Nursing Science 23
… continued
The European Academy of Nursing Science 24
Hardeman causal modelling case study: extending the MRC framework
• Concise one-page representation of causal pathways• Guides the choice of intervention points and
measures• Assists in choice of behaviour change techniques• Informs the assessment of ‘fidelity’ to theories• Enables statistical modelling of the relationships
between behaviours and health outcomes
Exercises
• Exercise 1: Joint arthoplasty surgery
• Exercise 2: diabetes
The European Academy of Nursing Science 26