Simulation model use to inform colorectal cancer screening ... · 15% Compare screening modalities...
1
Simulation model use to inform colorectal cancer screening delivery: a systematic review Smith, Heather 1,2 ; Varshoei, Peyman 1 ; Boushey, Robin 2 ; Kuziemsky, Craig 1 1 Telfer School of Management, University of Ottawa, 55 Laurier Ave E, Ottawa, Ontario 2 Division of General Surgery, The Ottawa Hospital, Ottawa, Ontario Difficult to estimate resources & effectiveness long-term Colorectal cancer (CRC) & screening • 2nd highest cancer mortality. • Screening detects cancer earlier & improves outcomes. • Optimal screening population, modality, frequency, & age is difficult to assess: ・Population specific ・No feasible RCT ▶ Background ▶ Results ▶ Acknowledgements & References 0 ▶ Conclusion Simulation modeling • Representation of a health system. • Used to assess optimal cancer screening & complex healthcare delivery. • Analyze scenarios & estimate outcomes to inform decision-making. The validity and impact of simulation models used to inform CRC screening is unknown. Systematic review of nine academic databases Jan 2008- Mar 2019. Inclusion Criteria: • Simulation model incorporating clinical data, • Target average risk colorectal cancer screening delivery using FIT, FS, FOBT or colonoscopy ✝ . ✝ FIT(fecal immunohistochemical testing), FS(flexible sigmoidoscopy), FOBT(fecal occult blood test). Assess if simulation modeling informs CRC screening delivery. ▶ Aim ▶ Methods Fecal Test Model Validity • As per international guidelines by ISPOR- SMDM ✶ , 2012 3 . ✶ International Society for Pharamaceconomics &Outcomes Reporting- Soceity of Medical Decision-making 11.6 14.0 25.6 16.3 18.6 7.0 39.5 44.2 14.0 23.3 0 20 40 60 80 Face Validation Internal Validation Cross Validation External Validation Predictive Validation Proportion of articles Validation method SIMULATION MODEL VALIDATION Validate Mention Informing CRC delivery • 88% aimed to address a specific health policy and systems decision in CRC screening. • 11% report an impact on that decision. • 25% involved decision-makers in model development. 32.6 27.9 39.5 44.2 67.4 69.8 76.7 83.7 88.4 0 20 40 60 80 Health equity Undesireable outcome Patient perspective Stakeholder acceptance Feasibility Cost-effectiveness Resource requirements Desireable outcome Priority of problem PROPORTION OF ARTICLES WITH CONTRIBUTING EVIDENCE FACTORS FOR INFORMED DECISIONS CONTRIBUTION TO INFORMING HEALTH POLICY & SYSTEM DECISIONS 1.Telford JJ. Canadian guidelines for colorectal cancer screening. Can J Gastroenterol. 2011;25(9):479-481. 2.Katsaliaki K, Mustafee N. Applications of simulation within the healthcare context. J Oper Res Soc. 2011 Aug 1;62(8):1431–51 3.Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB. Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value in Health. 2012;15(6):843-850. doi:10.1016/j.jval.2012.04.012 4.Alonso-Coello P, Schünemann HJ, Moberg J, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016. doi:10.1136/bmj.i2016 GRADE Evidence to Decision-making Framework Criteria 4 Priority of problem Weight of potential benefits and harms Patient perspective on value Stakeholder acceptance Feasibility Cost-effectiveness Resource requirements Simulation modeling is frequently used to inform specific health system & policy decision in CRC screening. It has been used to generate evidence for a broad range of factors critical to informed decision-making. There is need for improved validation and reporting of outcomes to optimize the application & implementation of simulation modeling in healthcare. NEXT step to develop guidelines for standardized simulation modeling reporting in healthcare. Model Description • Developed using published data (86%), registry data (63%), and unpublished data (13%). • Majority included a figure of the model (67%) & described model limitations (86%). Cost- effectiveness 28% Intervention to increase screening uptake 15% Compare screening modalities 15% Resource utilization 13% Specific population 7% Interval 4% Age of initiation 2% Other 11% Cancer Prevention 5% SIMULATION MODEL AIM Validation • 54% did not report any model validation. • Most frequent validation was cross-validation (44%).* • Only 7% conducted face validation. ✥ *comparing to outputs to similar models ✥confirming accuracy of model with experts (ie clinicians, patients, policymakers). Thank you to Alexandria Davies from the Ottawa Hospital Library for her assistance in conducting the systematic database search. This project was supported by the National Research and Education Council, and by the University of Telfer School of Management Student Research Grant. Improves survival & outcomes. Reduces cancer. Cost-effective. Model informing CRC delivery • Reported contribution to health system/policy decision. • Factors for informed decisions 4 . Colonoscopy
Transcript of Simulation model use to inform colorectal cancer screening ... · 15% Compare screening modalities...
Simulationmodelusetoinformcolorectalcancerscreeningdelivery:asystematicreviewSmith,Heather1,2;Varshoei,Peyman1;Boushey,Robin2;Kuziemsky,Craig11 TelferSchoolofManagement,UniversityofOttawa,55LaurierAveE,Ottawa,Ontario2DivisionofGeneralSurgery,TheOttawaHospital,Ottawa,Ontario
Difficulttoestimateresources&effectivenesslong-term
Colorectalcancer(CRC)&screening• 2ndhighestcancermortality.• Screeningdetectscancerearlier&improvesoutcomes.
• Optimal screeningpopulation,modality,frequency,&ageisdifficulttoassess:・Populationspecific・NofeasibleRCT
▶Background ▶Results
▶Acknowledgements&References
0
▶Conclusion
Simulationmodeling• Representationofahealthsystem.• Usedtoassessoptimalcancerscreening&complexhealthcaredelivery.
• Analyzescenarios&estimateoutcomestoinformdecision-making.
ThevalidityandimpactofsimulationmodelsusedtoinformCRCscreeningis
unknown.
SystematicreviewofnineacademicdatabasesJan2008-Mar2019.
InclusionCriteria:• Simulationmodelincorporatingclinicaldata,
• TargetaverageriskcolorectalcancerscreeningdeliveryusingFIT,FS,FOBTorcolonoscopy✝.
✝ FIT(fecalimmunohistochemicaltesting),FS(flexiblesigmoidoscopy),FOBT(fecaloccultbloodtest).
AssessifsimulationmodelinginformsCRCscreeningdelivery.
▶Aim
▶Methods
FecalTest
ModelValidity• AsperinternationalguidelinesbyISPOR-SMDM✶,20123.
✶InternationalSocietyforPharamaceconomics &OutcomesReporting-Soceity ofMedicalDecision-making
11.6
14.0
25.6
16.3
18.6
7.0
39.5
44.2
14.0
23.3
0 20 40 60 80
FaceValidation
InternalValidation
CrossValidation
ExternalValidation
PredictiveValidation
Proportionofarticles
Validationmetho
d
SIMULATIONMODELVALIDATION
Validate Mention
InformingCRCdelivery• 88%aimedtoaddressaspecifichealthpolicyandsystemsdecisioninCRCscreening.
• 11%reportanimpactonthatdecision.
• 25%involveddecision-makersinmodeldevelopment.
32.627.9
39.544.2
67.469.876.783.788.4
0 20 40 60 80
HealthequityUndesireableoutcome
PatientperspectiveStakeholderacceptance
FeasibilityCost-effectiveness
ResourcerequirementsDesireableoutcomePriorityofproblem
PROPORTIONOFARTICLESWITHCONTRIBUTINGEVIDENCE
FACTORSFORINFO
RMED
DECISIONS
CONTRIBUTIONTOINFORMINGHEALTHPOLICY&SYSTEMDECISIONS
1.TelfordJJ.Canadianguidelinesforcolorectalcancerscreening.CanJGastroenterol.2011;25(9):479-481.2.Katsaliaki K,Mustafee N.Applicationsofsimulationwithinthehealthcarecontext.JOper ResSoc.2011Aug1;62(8):1431–51
3.EddyDM,Hollingworth W,CaroJJ,Tsevat J,McDonaldKM,WongJB.ModelTransparencyandValidation:AReportoftheISPOR-SMDMModelingGoodResearchPracticesTaskForce-7.ValueinHealth.2012;15(6):843-850.doi:10.1016/j.jval.2012.04.012
4.Alonso-Coello P,Schünemann HJ,Moberg J,etal.GRADEEvidencetoDecision(EtD)frameworks:asystematicandtransparentapproachtomakingwellinformedhealthcarechoices.1:Introduction.BMJ.2016;353:i2016.doi:10.1136/bmj.i2016
GRADEEvidencetoDecision-makingFramework Criteria4Priorityofproblem
Weight ofpotentialbenefitsandharmsPatientperspectiveon valueStakeholderacceptance
FeasibilityCost-effectiveness
Resourcerequirements
Simulationmodelingisfrequentlyusedtoinformspecifichealthsystem&policydecisioninCRCscreening.
Ithasbeenusedtogenerateevidenceforabroadrangeoffactorscriticaltoinformeddecision-making.
Thereisneedforimprovedvalidationandreportingofoutcomestooptimizetheapplication&implementationofsimulationmodelinginhealthcare.
NEXTsteptodevelopguidelinesforstandardizedsimulationmodelingreportinginhealthcare.
ModelDescription• Developedusingpublisheddata(86%),registrydata(63%),andunpublisheddata(13%).
• Majorityincludedafigureofthemodel(67%)&describedmodellimitations(86%).
Cost-effectiveness
28%
Interventiontoincreasescreeninguptake15%
Comparescreeningmodalities
15%
Resourceutilization
13%
Specificpopulation
7%
Interval4%
Ageofinitiation
2%
Other11%
CancerPrevention
5%
SIMULATIONMODELAIM
Validation• 54%didnotreportanymodelvalidation.• Mostfrequentvalidationwascross-validation(44%).*
• Only7%conductedfacevalidation.✥
*comparingtooutputstosimilarmodels✥confirmingaccuracyofmodelwithexperts(ie clinicians,patients,policymakers).
ThankyoutoAlexandriaDaviesfromtheOttawaHospitalLibraryforherassistanceinconductingthesystematicdatabasesearch.ThisprojectwassupportedbytheNationalResearchandEducationCouncil,andbytheUniversityofTelfer SchoolofManagementStudentResearchGrant.
Improvessurvival&outcomes.Reducescancer.Cost-effective.
ModelinformingCRCdelivery• Reportedcontributiontohealthsystem/policydecision.
• Factorsforinformeddecisions4.
Colonoscopy
