Outline of Cerqual Approach - Trusted evidence. Informed ...
Transcript of Outline of Cerqual Approach - Trusted evidence. Informed ...
![Page 1: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/1.jpg)
TheGRADE-CERQual approach:Assessingconfidenceinfindingsfromsynthesesofqualitativeevidence
![Page 2: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/2.jpg)
ConfidenceintheEvidencefromReviewsofQualitativeresearch
![Page 3: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/3.jpg)
WhydidwedevelopGRADED-CERQual?
• Systematicreviewsofqualitativeresearch(alsocalledqualitativeevidencesyntheses)becomingincreasinglycommon
• Alsoincreasinglybeingusedinguidelineorpolicydevelopmentprocesses
• Usersneedmethodstoassesshowmuchconfidencetoplaceinfindingsfromthesereviews
• Userslikelytomakethesejudgementsanyway– itishelpfultoprovideasystematicandtransparent wayfordoingthis
![Page 4: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/4.jpg)
HowwasGRADE-CERQual developed?• Researcherswithbackgroundsinqualitativeresearchandsystematicreviews• Broadconsultationwithwidegroupofstakeholders
Neededanapproachthat:• Couldbeappliedtotypicaltypesofqualitativestudyapproachesanddata• Waseasytouse• Allowedjudgementstobereportedtransparently• Allowedthejudgementstobeunderstood
![Page 5: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/5.jpg)
RelationshiptoGRADE
• CERQualispartoftheGRADEWorkingGroup
• CERQualsharesthesameaimastheGRADEtoolusedtoassessthecertaintyofevidenceofeffectiveness
• However,CERQualisgroundedintheprinciplesofqualitativeresearch
![Page 6: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/6.jpg)
CERQualisnotatoolfor:
•Assessinghowwellanindividualqualitativestudywasconducted
•Assessinghowwellasystematicreviewofqualitativestudieswasconducted
•Assessingquantitativestudiesofqualityofcare
•Assessingconfidencein‘narrative’or‘qualitative’summariesoftheeffectivenessofanintervention,wheremeta-analysisisnotpossible
![Page 7: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/7.jpg)
Whatdoes theCERQualapproachdo?•CERQualaimstotransparentlyassessanddescribehowmuchconfidencetoplaceinfindingsfromqualitativeevidencesyntheses
![Page 8: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/8.jpg)
CERQualisappliedtoindividualsynthesisfindings
• Inthecontextofaqualitativeevidencesynthesis,afindingis…:
• Findingsfromqualitativeevidencesynthesescanbepresentedas:• themes,categoriesortheories• Asbothdescriptiveormoreinterpretivefindings
…ananalyticoutputthatdescribesaphenomenonoranaspectofaphenomenon
![Page 9: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/9.jpg)
Whatdowemeanby’confidenceintheevidence’?
Anassessmentoftheextenttowhichareviewfindingisareasonablerepresentationofthephenomenonofinterest
• i.e.thephenomenonofinterestisunlikelytobesubstantiallydifferentfromthereviewfinding
![Page 10: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/10.jpg)
TheCERQualapproach
![Page 11: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/11.jpg)
What skillsdoyou need toapplyCERQual?
• Anunderstanding of systematic review methodology• Anunderstanding of the principles of qualitative research
![Page 12: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/12.jpg)
CERQualmadeeasy
![Page 13: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/13.jpg)
Scenario:
Decisionmakersareconsideringanewintervention- Buthowarefemalepatientslikelytoexperienceit?
Reviewofqualitativeresearchiscarriedout–findingsdescribewomen’sexperiencesoftheintervention
![Page 14: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/14.jpg)
![Page 15: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/15.jpg)
![Page 16: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/16.jpg)
![Page 17: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/17.jpg)
![Page 18: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/18.jpg)
![Page 19: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/19.jpg)
![Page 20: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/20.jpg)
![Page 21: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/21.jpg)
![Page 22: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/22.jpg)
![Page 23: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/23.jpg)
Afterassessingallfourcomponentsanoverallassessmentismade,expressedaseither:
- Highconfidence- Moderateconfidence- Lowconfidence- Verylowconfidence
![Page 24: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/24.jpg)
Component1:Methodologicallimitations
Theextenttowhichthereareproblemsinthedesignorconductoftheprimarystudiessupportingareviewfinding
![Page 25: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/25.jpg)
Concernsaboutmethodologicallimitations
theprimarystudiesunderlyingareviewfindingareshowntohaveproblemsinthewaytheyweredesignedorconducted
� Wearelessconfidentthatthefindingreflectsthephenomenonofinterestwhen:
� Acritical appraisaltoolforqualitativestudiesshouldbeused tomakethisassessment� Typicallyincludesappraisalsofhowtheparticipants
andsettingswereselected,howdatawascollectedandanalysed,researcherreflexivityetc
� Currentlynowidespreadagreementaboutthebesttool– researchagendainplace
![Page 26: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/26.jpg)
Component2:Relevance
Theextenttowhichthebodyofevidencefromtheprimarystudiessupportingareviewfindingisapplicabletothecontextspecifiedinthereviewquestion
![Page 27: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/27.jpg)
Concernsaboutrelevance
thecontextsoftheprimarystudiesunderlyingareviewfindingaresubstantivelydifferentfromthecontextofthereviewquestion
Wearelessconfidentthatthefindingreflectsthephenomenonofinterestwhen:
![Page 28: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/28.jpg)
Assessing relevance - examples
• Exampleofindirectrelevance:Inasynthesisfocusingonchildren,oneincludedstudyfocusedonchildrenfrom3-5yearswhilethesynthesiswasinterestedintheagegroup10-18years
• Exampleofpartialrelevance:Inthesamesynthesis,severaloftheincludedstudiesfocusedongirlsonly,whilethesynthesiswasinterestedinallchildren
• Exampleofuncertainrelevance:Inthesamesynthesis,theagesofthechildreninsomeofthestudieswasunclear
![Page 29: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/29.jpg)
Component3:Coherence
Anassessmentofhowclearandcogentthefitisbetweenthedatafromtheprimarystudiesandthereviewfinding*
*HasbeenupdatedsincePLOSarticle
![Page 30: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/30.jpg)
Concernsaboutcoherence
Wearelessconfidentthatthefindingreflectsthephenomenonofinterestwhen:- Someofthedatacontradictthefinding- Someofthedataareambiguous
![Page 31: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/31.jpg)
Assessingcoherence–transformationofthedata
Degree of transformation
![Page 32: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/32.jpg)
Component4:Adequacyofdata
Thedegreeofrichness andquantityofdatasupportingareviewfinding
![Page 33: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/33.jpg)
Concernsaboutadequacyofdata
thedataunderlyingareviewfindingarenotsufficientlyrichoronlycomefromasmallnumberofstudiesorparticipants
• Reviewauthorsneedtomakeajudgementonwhatconstitutesdatathatarenotsufficientlyrichortoosmallanumberinthecontextofaspecificreviewfinding
Wearelessconfidentthatthefindingreflectsthephenomenonofinterestwhen:
![Page 34: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/34.jpg)
Making anoverallassessment
![Page 35: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/35.jpg)
METHODO-LOGICAL
LIMITATIONS
COHERENCE
RELEVANCE
ADEQUACY OF DATA
Confidence
Afterassessingeachoftheseparatecomponents,wemakeanoveralljudgementoftheconfidenceineachreviewfinding
![Page 36: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/36.jpg)
Confidencecanbeassessedashigh,moderate,loworverylow
• Highconfidence:Itishighlylikelythatthereviewfindingisareasonablerepresentationofthephenomenonofinterest
•Moderateconfidence:Itislikelythatthereviewfindingisareasonablerepresentationofthephenomenonofinterest
• Lowconfidence:Itispossiblethatthereviewfindingisareasonablerepresentationofthephenomenonofinterest
• Verylowconfidence:Itisnotclearwhetherthereviewfindingisareasonablerepresentationofthephenomenonofinterest
![Page 37: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/37.jpg)
CERQual qualitativeevidenceprofile
� Transparencyoftheprocessofassessingeachcomponentisenhancedthroughtheuseofaqualitativeevidenceprofilelikethisone
� Inthisprofile,reviewauthorsshouldindicate:¡ Whethertheyhaveno,minor,moderateorsubstantialconcernsregardingeachCERQual componentandwhatthese
concernsare¡ TheiroverallCERQual assessmentandtheirexplanationofthis
� TheQualitativeEvidenceProfilecanbeincludedasanappendixtoaqualitativeevidencesynthesis
![Page 38: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/38.jpg)
Summary of Qualitative Findings tables
February17,2017 38
ASoQF tableisthefinaloutputoftheprocessofmakingaCERQualassessment,andincludesfour elements:1. Asummaryofeachreviewfinding2. AnoverallCERQual assessmentforeachreviewfinding3. Anexplanationoftheoverallassessment(couldbeincludedasfootnotes)4. Referencetothestudiescontributingdatatothereviewfinding,including
clarificationofthecontextsinwhichthosestudieswereconducted
![Page 39: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/39.jpg)
Operationalising theCERQual approach• OurrecentlypublishedPLOSMedicinepaperprovidessomeguidanceonoperationalizingeachCERQual component– pleasehavealookatthispaperbeforeyoutrytoapplyCERQual!
•WearenotrecommendinganyspecifictoolstoassesseachCERQual component.Rather,wehaveidentifiedarangeofissuesthatshouldbeconsideredforeachcomponent
•Moredetailedguidanceandpromptsarebeingdevelopedthisyearandwillbepublishedasaseriesofpapers
•MembersoftheCERQual ProjectGroupareavailabletomentornewusers
![Page 40: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/40.jpg)
Insummary:CERQualhasthepotentialto:• Facilitatereflectiononfindingsfromqualitativeevidencesyntheses•Makemoreexplicit:• wheretherearegapsorinsufficientevidenceinrelationtoareviewquestion• whereonlypoorqualitystudiesareavailable
• Promptthegenerationofnewexplanationsorconceptstoexplainpatternsinfindings
ApplyingCERQual involvesjudgements.CERQualattemptstomakethesejudgementstransparentandincreasethecontributionofqualitativeresearchtodecisionmaking
![Page 41: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/41.jpg)
GettinginvolvedinGRADE-CERQual• YouareinvitedtojoinourmailinglistandtheGRADE-CERQualProjectGroupviaourwebsite:www.cerqual.org
• You can makeanimportant contribution by:• Piloting CERQual inyour qualitative evidence synthesis• Recording what worked well andwhat was moredifficult inthe process• Sharing this feedbackwith us,sothat we can use this toimprove ourguidance forundertakingCERQual assessments
• Practical resources (e.g.templates)are available andwe can alsolinknew CERQual users with amentor.Please contact us torequest access tothese
• TheGRADE-CERQual ProjectGroupwill holdmeetings andtrainingsessionsduringthe year – information on these will besentoutviaour mailing list
![Page 42: Outline of Cerqual Approach - Trusted evidence. Informed ...](https://reader031.fdocuments.net/reader031/viewer/2022020621/61ea51bb7d345828c55c4435/html5/thumbnails/42.jpg)
TolearnmoreaboutCERQual
• Jointhemailinglist• Jointheprojectgroup• Joinawebinarorfurtherface-to-facetraining• Forthcomingseriesofpapers,probablyinImplementationScience