Professor Jane Noyes
30
Qualitative Evidence Synthesis Where are we? What’s New and What’s next and outstanding challenges? Professor Jane Noyes Noreen Edwards Chair in Health Services Research and Child Health, Bangor University Cochrane Fellow and Lead Convenor Cochrane Qualitative Research Methods Group Editor Journal of Advanced Nursing
description
Qualitative Evidence Synthesis Where are we? What’s New and What’s next and outstanding challenges?. Professor Jane Noyes Noreen Edwards Chair in Health Services Research and Child Health, Bangor University Cochrane Fellow and Lead Convenor Cochrane Qualitative Research Methods Group - PowerPoint PPT Presentation
Transcript of Professor Jane Noyes
Qualitative Evidence Synthesis
Where are we?What’s New and What’s next and
outstanding challenges?
Professor Jane NoyesNoreen Edwards Chair in Health Services Research and Child
Health, Bangor UniversityCochrane Fellow and Lead Convenor Cochrane Qualitative
Research Methods GroupEditor Journal of Advanced Nursing
Most methods of primary qualitative data analysis can be applied to qualitative evidence synthesis
‘New’ evidence synthesis approaches have emerged –eg meta-ethnography – but use recognisable qualitative analysis principles
There are many methods with similar sounding names – rush to publish ‘my method’ and call it something slightly different – meta-narrative, meta-ethnography, meta-summary, meta-synthesis, meta-aggregation – etc etc.
Some reviewers are mixing or combining methods – eg thematic and realist into a single review
What have we learned so far?
Many authors appear to stick with familiar methods rather than select the most appropriate method to address the question and type of evidence
Software evolving and improving: EPPI reviewer / Atlas Ti
Searching methods and approaches have evolved for specific methods of qualitative synthesis (purposive versus exhaustive, tipping point)
There are increasing numbers of some very good and inevitably some very bad reviews published!
Cochrane is finally getting to grips with qualitative evidence synthesis!
What have we learned so far cont?
• Huge increases in qualitative syntheses published
• There is a big gap between what people claim to use as a synthesis approach and what is actually done in practice
• Do not need more methods – but better application and further evaluation of existing ones
Aggregated themes versus theory development versus– could we do better?
Karin is particularly critical of the way
meta-ethnography is reported in published
reviews – line of argument synthesis
Like others my experience of theory development is that it
takes months and months of thinking,
meeting, discussing, and ‘arguing’!
Some health and policy funders eg NICE require a
product within 90 days
RAPID QUALITATIVE EVIDENCE SYNTHESIS
METHODOLOGY
New and updated guidance and new acceptance of the value of qualitative evidence
Guidance on new applications
Greater specification and further adaptation of methods for qualitative evidence synthesis
Thematic synthesis –various types with different starting points - inductive and deductive
3 stage thematic synthesis - Line by line inductive coding- eg Thomas and Harden
A priori - eg 5 stage Ritchie and Spencer Framework Synthesis
Conceptual frameworks and models to guide analysis and synthesis
Additional adaptations to conceptual frameworks to ‘best-fit’ specific reviews – eg conceptual development by the back door
Richie and Spencer Framework Synthesis ApproachWith Normalisation Process Theory Elements
(Watson et al 2011)
‘Best Fit Framework Synthesis’
Carroll, C., Booth, A., Leaviss, J., & Rick, J. (2013). “Best fit” framework synthesis:
refining the method. BMC medical research methodology, 13(1), 1-16.
A new typology put forward by Gough, Thomas and Oliver 2012
Development of typologies of methods and greater classification of terminology
Methods for the synthesis of qualitative research: a critical review.
Barnett-Page E, Thomas J.
Therapist dependent interventions (where the intervention is a combination of the therapist effect and the therapy or procedure and the effectiveness is potentially dependent on both);
Complex healthcare interventions (where the intervention is a combination of several actions, e.g. multidisciplinary health care in stroke units); multilevel public health interventions (e.g., a healthy living initiative that aims to impact behaviour at the community, school, and individual levels);
Professional or patient education interventions (e.g., introduction of clinical guidelines).
Complex interventions may contain a mix of effective, ineffective, and even harmful actions which may interact synergistically or dysynergistically or be interdependent.
Much greater emphasis on qualitative evidence synthesis in the context of complex interventions
Therapist dependent interventions (where the intervention is a combination of the therapist effect and the therapy or procedure and the effectiveness is potentially dependent on both);
Complex healthcare interventions (where the intervention is a combination of several actions, e.g. multidisciplinary health care in stroke units); multilevel public health interventions (e.g., a healthy living initiative that aims to impact behaviour at the community, school, and individual levels);
Professional or patient education interventions (e.g., introduction of clinical guidelines).
Complex interventions may contain a mix of effective, ineffective, and even harmful actions which may interact synergistically or dysynergistically or be interdependent.
Much greater emphasis on qualitative evidence synthesis in the context of complex interventions
Methodological Investigation of Cochrane Reviews of Complex Interventions
(MICCI)
Acknowledge fellow methodologists who met at Montebello
January 2012
Applying qualitative methods to explore complexity
Development of little used methods Qualitative Comparative Analysis (QCA); (actually a mixed method
approach) is a promising method for providing evidence in situations where interventions interact with contexts, enabling causal pathways to be discerned from how sets of conditions combine with particular outcomes.
David Byrne (leading proponent of QCA) and Bridget Candy & Jackie Chandler (PhD students)
Byrn D. Evaluating complex social interventions in a complex world 2013
Blackman et al (2013) Using Qualitative Comparative Analysis to understand complex policy problems
Candy, B., King, M., Jones, L., & Oliver, S. (2013). Using qualitative evidence on patients' views to help understand variation in effectiveness of complex interventions: a qualitative comparative analysis.
Development of little used methods
Use of content analysis to conduct knowledge-building and theory-generating qualitative systematic reviews Finfgeld-Connett 2013
Systematically reviewing and synthesizing evidence from conversation analytic and related discursive research to inform healthcare communication practice and policy: an illustrated guide Parry and Land 2013
Systematic text condensation: A strategy for qualitative analysis Malterud 2012
The procedure consists of the following steps: 1) total impression – from chaos to themes; 2) identifying and sorting meaning units – from themes to codes; 3) condensation – from code to meaning; 4) synthesizing – from condensation to descriptions and concepts.
Greater use of Conceptual Frameworks, Theoretical and Logic models
• Programme logic is the way in which a ‘programme’ fits
together, usually in a simple sequence of inputs, activities,
outputs, and outcomes.
• Programme theory goes a step further and attempts to
build an explanatory account of how the
intervention/programme/service works, with whom, and
under what circumstances.
Mechanisms
MECHANISMS• Programme theory can be expressed as an elaborated programme logic
model, where the emphasis is on causal explanation using the idea of ‘‘mechanisms’’ that are at work.
• Mechanisms ‘occur’ between the delivery of the intervention/programme/ service and the occurrence of outcomes of interest.
• Mechanisms are participants’ responses to the intervention/programme/ service.
• The mechanism of change is not the intervention/programme/service per se but the response that the activities generate (ie the human behaviour – that can be largely captured by qualitative research )
Astbury 2010
“I think you should be more explicit here in Step Two.”
Anderson et al. 2011 Using logic models to capture complexity in systematic review
Use of logic models
Turley et al 2013
Slum upgrading
review:
methodological
challenges that
arise
in systematic
reviews of
complex
interventions
Health belief model: public information to prevent skin cancer
Multiple methods within a single HTA review
New publication standards
Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ BMC
Medical Research Methodology, Vol. 12, No. 1. (27 November 2012), 181, doi:10.1186/1471-2288-
12-181 by Allison Tong, Kate Flemming, Elizabeth McInnes, Sandy Oliver, Jonathan Craig
Most downloaded article JCE
Continuing challenges and issues
• Many methods but a relatively small number consistently used
• Need more high quality published examples and methodological testing
• Evidence synthesis is cheaper than primary research – funders increasingly fund systematic reviews – need more teams with quan/qual skills
• Need greater specification of methods for implementation reviews
