Researching Complex Interventions. Welcome, and thank you for coming We hope you enjoy it.
-
Upload
lawrence-freeman -
Category
Documents
-
view
218 -
download
0
Transcript of Researching Complex Interventions. Welcome, and thank you for coming We hope you enjoy it.
ResearchingComplex Interventions
ResearchingComplex Interventions
Welcome, and thank you for coming
We hope you enjoy it
HELP
Chaos and complexity theories!
Not that bad, but it is about black boxes
Outline of Presentation
1. What makes an intervention complex?
2. Why are complex interventions important?
3. The MRC guidelines
4. An example of a complex intervention trial
5. This workshop
1) What is a ‘CI’?
Complex interventions are traditionally described as: “Interventions that contain
several interacting components”
Some Dimensions of Complexity
Some Dimensions of Complexity
1. Number of interacting components within the experimental and control interventions
Some Dimensions of Complexity
1. Number of interacting components within the experimental and control interventions
2. Number/difficulty of skills and behaviours required by those delivering or receiving the intervention
Some Dimensions of Complexity
1. Number of interacting components within the experimental and control interventions
2. Number/difficulty of skills and behaviours required by those delivering or receiving the intervention
3. Number of groups or organisational levels targeted
Some Dimensions of Complexity
1. Number of interacting components within the experimental and control interventions
2. Number/difficulty of skills and behaviours required by those delivering or receiving the intervention
3. Number of groups or organisational levels targeted
4. Degree of flexibility or tailoring of interventions permitted (individualisation)(and others - Lewin et al)
But some are more complex than others:
“Interventions built up of a number of components which may act both
independently and interdependently”
But some are more complex than others:
“Interventions built up of a number of components which may act both
independently and interdependently”
UNEXPECTED OUTCOMES
What is NOT a CI?
A drug trial?
What is NOT a CI?
A drug trial?
Is the whole RCT/EBM business really all about the evil multi-
national pharmaceutical industry?
2) Why are CIs important?
2) Why are CIs important?
Partly because they encompass all types of intervention other than
drug trials
Why are CIs important?
1. Chronic disease and health behaviours
Why are CIs important?
1. Chronic disease and health behaviours
2. The context in which an intervention is used
Why are CIs important?
1. Chronic disease and health behaviours
2. The context in which an intervention is used
3. In the real world we rarely use single interventions in isolation
Why are CIs important?
1. Chronic disease and health behaviours
2. The context in which an intervention is used
3. In the real world we rarely use single interventions in isolation
4. Unexpected consequences of interventions
1) Chronic Disease and Health Behaviours
Interventions that attempt to change behaviour are complex
Chronic Disease and Health Behaviours
We are not going to make inroads into chronic disease management unless we can learn how to change behaviours of health care professionals, patients and the public
Example: Diabetes
2) The Context
The Context
• The results of ‘simple’ trials may not be generalisable because of the influence of context on outcomes
• Context include many factors that can CHANGE over time
‘Macro’and ‘micro’-context effects
• ‘Macro’-context effects: Socio-economic and cultural circumstances of a trial and related cultural assumptions; the health care system; characteristics of population; how the problem is caused and sustained
• ‘Micro’-context effects: the environment in which the intervention is carried out, the interactions between HCP and patient
A ‘macro’ context effect
A ‘macro’ context effect
Healing rates of gastric ulcers, in trial ‘control’ groups
A ‘macro’ context effect
Healing rates of gastric ulcers, in trial ‘control’ groups
Germany: 62.4%Denmark: 16.7%
(Moerman D 2000)
A ‘micro’ context effect:the ‘placebo’ response
RCT effects
Intervention ‘Control’
(placebo)
Change
No Treatment
RCT effects (Kirsch, depression)
Intervention
Change
Adjust for no treatment
RCT effects (Kirsch, depression)
Intervention ‘Control’
(placebo)
Change
Adjust for no treatment
RCT effects (Kirsch, depression)
Intervention ‘Control’
(placebo)
Change
Adjust for no treatment
‘Medicine’ only worries about this bit
Assumption of the classic RCT
• That specific effects and context effects (characteristic effects and incidental effects) are divisible and additive
Specific effect and context effect
? SE
CE
Assumption of the classic RCT
• That specific effects and context effects (characteristic effects and incidental effects) are divisible and additive
• In many ‘complex interventions’ this is not the case (e.g. acupuncture and physiotherapy)
Paterson and Dieppe BMJ 2005
Interaction of specific effects and context effects
• Individualisation of therapy by adjusting what you do according to the response of the individual and their health beliefs
Interaction of specific effects and context effects
• Individualisation of therapy by adjusting what you do according to the response of the individual and their health beliefs
• What is generally done in clinical practice, especially by AHP and CAM practitioners. Regarded as a breach of intervention fidelity within the world of the traditional RCT – but can be OK in complex interventions
Trial ‘Fidelity’
• A principle of the classical RCT is that everyone gets exactly the same intervention
• This does not mirror ‘real life’
• During a complex intervention trial there may be variations in intervention delivery (by different people for example) and/or changes over time
This is OK as long as carefully recorded
3) Packages of Care
Packages of Care
• In chronic disease we rarely use interventions singly, we combine different modalities
Packages of Care
• In chronic disease we rarely use interventions singly, we combine different modalities
• NICE guideline committee on management of OA: Lots of trials of single interventions (mostly drugs that don’t help much), very few trials of packages of care (CIs)
Packages of Care
• In chronic disease we rarely use interventions singly, we combine different modalities
• NICE guideline committee on management of OA: Lots of trials of single interventions (mostly drugs that don’t help much), very few trials of packages of care (CIs)
• ERGO: we do not know what to recommend!
4) Unexpected Consequences
Unexpected Consequences
• Many trials of CIs have produced negative results when people thought the intervention was bound to work
• Some trials of CIs have produced unexpected positive and negative effects which were not understood
Complex Interventions can be seen as a ‘black box’
Complex Interventions can be seen as a ‘black box’
??
Complex Interventions can be seen as a ‘black box’
??
Emergent properties
Unexpected consequences
We need to unravel the ‘causal chain of understanding’
??
Emergent properties
Unexpected consequences
We need to unravel the ‘causal chain of understanding’
• This means assessing process as well as outcome
• This requires theory (what you think might happen and why) and may need both quantitative and qualitative approaches
3) MRC Frameworks for Complex Interventions
(Campbell et al BMJ 2000, Craig et al BMJ 2008)
The first framework (2000)
Preclinical Phase 1 Phase 2 Phase 3 Phase 4
Theory Modelling Exploratory RCT Implementation
Problems
• Model from drug development
• Linearity implied by the diagram
• Theory and modelling phases not well explained
• Implication that traditional RCTs are the preferred (only?) option for assessment
• Lack of attention to contextand others
The revised MRC framework for complex
interventions(Craig et al 2008)
Developing The Revised Framework
1. Workshop funded by MRC PHSRN, led by Paul Dieppe, Janet Darbyshire and Sally McIntyre, held in May 2006, c 30 invited attendees, with some of those who developed the 1st framework
2. Writing committee of PD, JD, SM + Peter Craig, Susan Michie, Irwin Nazareth, Mark Petticrew
3. Review by MRC HSPHRB
The revised framework1. Summary of what makes an intervention
complex, and the process of development and testing a CI
2. Questions researchers need to ask themselves
3. Examples of complex intervention studies using a wide variety of designs including non-randomisation/quasi-experimental studies
The ‘new’ model
The ‘new’ model
The ‘new’ model
Development:
1. Evidence base2. Theory3. Modelling process/outcome
Development:
1. Evidence base2. Theory3. Modelling process/outcome
Are you clear about what you are trying to do and why?
Feasibility/Piloting:
1. Testing procedures2. Recruitment/retention3. Determining sample size
Feasibility/Piloting:
1. Testing procedures2. Recruitment/retention3. Determining sample size
Can this work?Is it practical and sensible?What outcomes (interactions)?
Evaluation:
1. Assessing effectiveness2. Understanding process3. Cost-effectiveness
Evaluation:
1. Assessing effectiveness2. Understanding process3. Cost-effectiveness
What design are you going to use and why?
Implementation:
1. Dissemination2. Surveillance/monitoring3. Long-term follow-up
Implementation:
1. Dissemination2. Surveillance/monitoring3. Long-term follow-up
Publication is not enough!
4) An example of a CI Trial
1. The question – does physiotherapy work for people with isolated patello-femoral OA?
1. The question – does physiotherapy work for people with isolated patello-femoral OA?
2. The intervention – home based exercises and patella strapping with ‘education’ to aid behaviour change
1. The question – does physiotherapy work for people with isolated patello-femoral OA?
2. The intervention – home based exercises and patella strapping with ‘education’ and leaflets to aid behaviour change
3. Recruitment – from a community cohort
1. The question – does physiotherapy work for people with isolated patello-femoral OA?
2. The intervention – home based exercises and patella strapping with ‘education’ and leaflets to aid behaviour change
3. Recruitment – from a community cohort
4. The design – Modified Zelen design
Pre-screened: 178
Suitable for inclusion: 157
Consent obtained: 87
Pre-screened: 178
Suitable for inclusion: 157
Consent obtained: 87
Intervention: 43 No intervention:44
Pre-screened: 178
Suitable for inclusion: 157
Consent obtained: 87
Intervention: 43(38) No intervention:44 (44)
Final assessment: 82
Pre-screened: 178
Suitable for inclusion: 157
Consent obtained: 87
Intervention: 43(38) No intervention:44 (44)
Final assessment: 82
First consent
Second consent
Pre-screened: 178
Suitable for inclusion: 157
Consent obtained: 87
Intervention: 43(38) No intervention:44 (44)
Final assessment: 82
First consent
Second consent No trial consent
1. The question – does physiotherapy work for people with isolated patello-femoral OA?
2. The intervention – home based exercises and patella strapping with an educational package to aid behaviour change
3. The design – modified Zelen design
4. Recruitment – from a community cohort
5. Outcomes – ‘WOMAC’ pain and function
1. The question – does physiotherapy work for people with isolated patello-femoral OA?
2. The intervention – home based exercises and patella strapping with an educational package to aid behaviour change
3. The design – modified Zelen design
4. Recruitment – from a community cohort
5. Outcomes – ‘WOMAC’ pain and function
6. Process measures – quads strength, nested qualitative interviews
The ‘Result’
• Small difference in favour of intervention group at 5 months, disappeared by 12 months
Quilty et al – HTA Report 1997
Intervention
Control
0 5 m 12 m
Pain
The ‘Result’
• Small difference in favour of intervention group at 5 months, disappeared by 12 months
BUT:
The ‘Result’
• Small difference in favour of intervention group at 3 months, disappeared by 6 months
BUT:
• Huge variations in response - some a lot better some a lot worse; interviews showed that most people did not do physio (for good reasons); and that many thought they were a lot better, even if the WOMAC etc did not show that!
So how did we do with this trial?
Development
RATHER badly:
We did look at the literature to help design the intervention – BUT
1. No patient involvement in design2. No theoretical framework for behaviour
change aspect of the intervention3. No piloting of feasibility/acceptability4. No economic thinking or modelling
Feasibility/Pilot Stage
Quite badly:
1. We did not know if this would work before we started the trial – BUT:
2. We did work out the sample size
3. In practice the recruitment and retention went well (we were lucky)
Evaluation Stage
We did that quite well!
1. An innovative trial design that ‘worked’
2. Process measures to help unpack the black box of the intervention
3. Nested qualitative research to aid understanding
4. Good documentation of the intervention and how much individualisation was allowed
But we did not do any economics!
Implementation
Well, we told people about it
Well, we told people about it
And we published it
Well, we told people about it
And we published it
We published it more than once actually, and it helped several people’s careers ……
Well, we told people about it
And we published it
We published it more than once actually, and it helped several people’s careers ……
PATHETIC!
5) This Workshop
5) This Workshop
Is about methodology and how to do applied health research as much as it is about
complex interventions
5) This Workshop
Is about methodology and how to do applied health research as much as it is about
complex interventions
The MRC framework provides us with a window on all the key methodologies used in applied health research (Richards 2010)
5) This Workshop
1. Workshops
2. Research Cafes
3. Plenary Sessions
The ‘new’ model
The ‘new’ model
The ‘new’ model
HAVE FUN
HAVE FUN
BE NOISY
STAGE 4
STAGE 4:
Dissemination and Implementation
STAGE 4:
Dissemination and Implementation
Surveillance, Monitoring and Long Term follow up
STAGE 4:
Dissemination and Implementation
Surveillance, Monitoring and Long Term follow up
Evidence-Based Practice
Evidence-Based Practice
Trials
Evidence-Based Practice
Trials
Systematic Reviews
Meta-analysis
Evidence-Based Practice
Trials
Systematic ReviewsMeta-analysis
Guidelines/Dissemination
Evidence-Based Practice
Trials
Systematic ReviewsMeta-analysis
Guidelines/Dissemination
‘All will be well’
Evidence-Based Practice
Trials
Systematic ReviewsMeta-analysis
Guidelines/Dissemination
‘All will be well’
The Problem:Guideline Dissemination
does not work
The Problem:Guideline Dissemination
does not work
WHY?
The Problem:Guideline Dissemination
does not work
WHY?WHAT CAN WE DO TO
IMPLEMENT BEST PRACTICE?
Some VERY brief theoretical considerations
• The goal is ‘adoption’ or ‘normalisation’ of what is agreed best practice
• Changing things for the better means changing what individuals do, or changing how organisations work
Different Approaches
• Psychology – its about behaviour change: apply theories such as TPB (there are 33 of them!)
• Sociology – its about organisational structures, culture, interactions and practices: explore these
• Economics – its about money, preferences and choices: use incentives and financial levers
EXERCISE 1
Make a list of barriers and facilitators to the implementation
of the scenario you have
(Single words or short phrases)
Some ways of Disseminating and Implementing Research Findings
Some ways of Disseminating and Implementing Research Findings
1. Dissemination?
Dissemination
1. Academic publications and presentations
2. Stakeholder meetings and documents
3. Targeted one-page summaries for different groups
4. Working with patient groups
5. Simple recommendations for action
6. Politicians and the Press
Some ways of Disseminating and Implementing Research Findings
1. Dissemination?
2. Implementation?
Three Approaches
1. The ‘local champion’ model
2. The ‘Quality Circle’ Model
3. Appropriateness Criteria
1. A ‘Local Champion’
1. A ‘Local Champion’
Someone in the organisation, who knows how things work, and who does what, and is
passionate about the issue and about quality of health care delivery
1. A ‘Local Champion’
Someone in the organisation, who knows how things work, and who does what, and is
passionate about the issue and about quality of health care delivery
THIS IS A SYSTEM THAT WORKS WELL!
1. A ‘Local Champion’
Someone in the organisation, who knows how things work, and who does what, and is
passionate about the issue and about quality of health care delivery
THIS IS A SYSTEM THAT WORKS WELL!
BUT IT DEPENDS ON BEING LUCKY ENOUGH TO HAVE SUCH A PERSON
2. The ‘Quality Circle’
An approach Championed by Don Berwick and his Boston Based
‘Institute for Health Improvement’ (IHI Website)
2. The ‘Quality Circle’
2. The ‘Quality Circle’
Sharing best practice:
2. The ‘Quality Circle’
Sharing best practice:
1. Get groups together from different providers, each of whom has to provide ‘x’
2. The ‘Quality Circle’
Sharing best practice:
1. Get groups together from different providers, each of whom has to provide ‘x’
2. Discuss problem areas, successes and failures
2. The ‘Quality Circle’
Sharing best practice:
1. Get groups together from different providers, each of whom has to provide ‘x’
2. Discuss problem areas, successes and failures
3. Within provider groups decide which area(s) you need to improve on, use others’ work, decide how you are going to know if you have improved things
2. The ‘Quality Circle’
Sharing best practice:
1. Get groups together from different providers, each of whom has to provide ‘x’
2. Discuss problem areas, successes and failures
3. Within provider groups decide which area(s) you need to improve on, use others’ work, decide how you are going to know if you have improved things
4. Put new plan into practice
2. The ‘Quality Circle’
Sharing best practice:
1. Get groups together from different providers, each of whom has to provide ‘x’
2. Discuss problem areas, successes and failures
3. Within provider groups decide which area(s) you need to improve on, use others’ work, decide how you are going to know if you have improved things
4. Put new plan into practice
5. Share outcomes in meetings with other providers
P P P
A D A D A D
S S S
Plan, Act, Study, Do
LS1 LS2 LS3
AP1 AP2 AP3
2. Appropriateness Criteria
Developed by the Rand Organisation in California to deal with wide variations in provision of common procedures such as
coronary artery bypass
3. Appropriateness Criteria
3. Appropriateness Criteria
1. Search the literature to see what factors might affect decision to do ‘x’
3. Appropriateness Criteria
1. Search the literature to see what factors might affect decision to do ‘x’
2. Develop scenarios around doing ‘x’ in which you alter key variables (e.g. age)
3. Appropriateness Criteria
1. Search the literature to see what factors might affect decision to do ‘x’
2. Develop scenarios around doing ‘x’ in which you alter key variables (e.g. age)
3. Get a panel to rate scenarios on a 0-9 scale
3. Appropriateness Criteria
1. Search the literature to see what factors might affect decision to do ‘x’
2. Develop scenarios around doing ‘x’ in which you alter key variables (e.g. age)
3. Get a panel to rate scenarios on a 0-9 scale
4. Meet, discuss discrepancies, re-score
3. Appropriateness Criteria
1. Search the literature to see what factors might affect decision to do ‘x’
2. Develop scenarios around doing ‘x’ in which you alter key variables (e.g. age)
3. Get a panel to rate scenarios on a 0-9 scale
4. Meet, discuss discrepancies, re-score
5. Audit activity in ‘x’ against criteria
EXERCISE 2
Work out how you would implement the intervention outlined in your scenario
(assume cost-effectiveness)