STATISTICAL ANALYSIS PLANS: PRINCIPLES AND PRACTICE Page 1 ____________________________________________________________________________________

STATISTICAL ANALYSIS PLANS PRINCIPLES AND PRACTICE Workshop Number 8Presented at the Society for Clinical Trials 2007 Annual Meeting Montreal, Canada Sunday May 20, 2007

James R. Johnson, Ph.D. David Fitts, Ph.D., MPH

____________________________________________________________________________________ Johnson and Fitts (20May2007) Workshop #8 (SCT Annual Meeting)

STATISTICAL ANALYSIS PLANS: PRINCIPLES AND PRACTICE Page 2 ____________________________________________________________________________________

STATISTICALANALYSISPLANS PRINCIPLESANDPRACTICE Presentedatthe SocietyforClinicalTrials 2007AnnualMeeting Montreal,Canada SundayMay20,2007PresentedBy:JamesR.Johnson,PhD DavidFitts,PhD,MPH

LearningObjectives Attheendofthissessionyoushouldhavea comprehensiveunderstandingofthe: WhywePlanforStatisticalAnalysis? DefinitionoftheStatisticalAnalysisPlan(SAP). HistoricalandRegulatory(FDA,EMEA,ICH)frameworkfortheSAP. RelationshipoftheSAPtoInternalandExternalDocuments/Publications. SAPaudience(s). TheSAPandStatisticalEthics. StructureandContentoftheSAP. SAPTemplate(s). Drug/DeviceDevelopmentProgramPlanning PhaseIPK/PDSafetyStudies PhaseIIScreeningandDoseFindingStudies PhaseIIb/IIIRegistrationStudies CommunicationandWritingStyle.01May2007 STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE Page2

____________________________________________________________________________________ Johnson and Fitts (20May2007) Workshop #8 (SCT Annual Meeting)

STATISTICAL ANALYSIS PLANS: PRINCIPLES AND PRACTICE Page 3 ____________________________________________________________________________________

WhydowePlanStatistical Analyses?

01May2007

STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE

Page3

WhyPlan?1. 2. StudyDesignSetsthegroundworkforestablishingGood StatisticalPractices

1STUDY DESIGN

StudyMethodsImplementingthePlanoutlinedintheStudy Design.ConcernedprimarilywithData CollectionMethods,andvalidationofthe datacollectionmethodology.

2STUDY METHODS

3.

StudyAnalysisRationalanalysisoftheplannedstudydesign, implementedwithsolidmethodology generallyresultsinobservationsthataretrue totheexperiment.

4STUDY ANALYSIS

3

4.

FeedbackLoopTheStudyAnalysisservestoprovideacheck ontheOriginalStudyDesign.Isanindicator ofhowwellthedesign,datamanagement, andstatisticalpracticeswereestablishedand implemented.

01May2007

STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE

Page4

____________________________________________________________________________________ Johnson and Fitts (20May2007) Workshop #8 (SCT Annual Meeting)

STATISTICAL ANALYSIS PLANS: PRINCIPLES AND PRACTICE Page 4 ____________________________________________________________________________________

StatisticalPlanninginaStudy StatisticalPlanningphasesstartatthebeginningofa studywiththeprimaryresearchquestion. Keyplanningissuesthatsignificantlyinfluence statisticalanalysisinclude: AssessmentofStudyObjectivesandEndpoints. MethodsforDataCollection KeyVariableIdentifications,DataDictionaryDefinitions MethodsforKeyVariableValidationandRobustness MethodsforSubjectRandomization(ifapplicable) RelationshipsforDatabaseDesign

01May2007

STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE

Page5

StudyDesignPhase AtStudyDesigntheconceptofrationalstatisticalpracticesmustbe started. Criticalquestionsthatmustbeaddressedattheconceptionofa ResearchProtocol. WhatistheClear Objectiveofthestudy? WhatistheKeyVariable(s)tobeassessedinthestudy? HowaretheKeyVariable(s)measured? CanKeyVariablemeasurementsbereproducedandvalidated. AretheKeyVariables: ObjectiveAssessmentsbyanIndependentAssessororInstrument? SubjectAssessmentssuppliedbytheSubjectorObservation?

STOPIfyoucannotanswertheseQuestions?01May2007 STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE Page6

____________________________________________________________________________________ Johnson and Fitts (20May2007) Workshop #8 (SCT Annual Meeting)

STATISTICAL ANALYSIS PLANS: PRINCIPLES AND PRACTICE Page 5 ____________________________________________________________________________________

WhyPlan? DataErrors?? ADataErrormaybedefinedasany deviationfromthetrueorexpectedvalue wherethecauseofthedeviationmaybe knoworunknown,duetorandomchance orsystematicbiases. Clearly,weplan toavoidand/oreliminate systematicbiases.01May2007 STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE Page7

WhyPlan? DataErrors? Wheredodataerrors comefromina clinicaltrial? Duringthedesignstageofthetrial(Protocol,CaseRecordForms, PoorStatisticalPlanning). DuringtheTrainingofInvestigatorsandStudyCoordinatorson Howtoassesseachdataiteminthestudy. DuringEditspecificationsandMedicalReview. DuringCodingofAdverseEventsand/orMedications. DuringtheAnalysisandTabulationofStudyResults. DuringthewritingandpublicationofStudyResults.

01May2007

STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE

Page8

____________________________________________________________________________________ Johnson and Fitts (20May2007) Workshop #8 (SCT Annual Meeting)

STATISTICAL ANALYSIS PLANS: PRINCIPLES AND PRACTICE Page 6 ____________________________________________________________________________________

SAPDefinition

01May2007

STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE

Page9

SAPDefinition TheDefinitionofwhatisanSAPcomes primarilyfromprofessionalorganizations andregulatoryauthorities: ICHE9(StatisticalPrinciplesforClinicalTrials) FDA(FederalRegisterVol.63,No.179,Wednesday,September 16,1998) NationalInstitutesofHealth(NIH) PSI(StatisticiansinthePharmaceuticalIndustry) GMDS(DeutscheGesellschaftfrMedizinischeInformatik, BiometrieundEpidemiologiee.V.)01May2007 STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE Page10

____________________________________________________________________________________ Johnson and Fitts (20May2007) Workshop #8 (SCT Annual Meeting)

STATISTICAL ANALYSIS PLANS: PRINCIPLES AND PRACTICE Page 7 ____________________________________________________________________________________

SAPDefinition FromtheICHE9andFDACFR: A statistical analysis plan is a document that contains a more technical and detailed elaboration of the principal features of the analysis described in the protocol, and included detailed procedures for executing the statistical analysis of the primary and secondary variables and other data. (ICH Guidelines: E9) (Federal Register Vol. 63, No. 179, Wednesday, September 16, 1998)

01May2007

STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE

Page11

SAPDefinition FromtheNIH: A comprehensive and detailed description of the methods for, and presentation of, data analyses for a study protocol. The plan ensures that analyses are conducted in a scientifically valid manner and that decisions are documented. (National Institutes of Health: Glossary of Terms)

01May2007

STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE

Page12

____________________________________________________________________________________ Johnson and Fitts (20May2007) Workshop #8 (SCT Annual Meeting)

STATISTICAL ANALYSIS PLANS: PRINCIPLES AND PRACTICE Page 8 ____________________________________________________________________________________

SAPDefinition FromPSI(StatisticiansinthePharmaceutical Industry): The statistical analysis plan is intended to be a comprehensive and detailed description of the methods and presentation of data analyses proposed for a clinical trial, in order to avoid post hoc decisions which may affect the interpretation of the statistical analysis (PSI: Guidelines for Standard Operating Procedures for Good Statistical Practice in Clinical Research and also in North PM. DIA Journal 1998:(32):665-682)

01May2007

STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE

Page13

SAPDefinition From DeutscheGesellschaftfr

MedizinischeInformatik,Biometrieund Epidemiologiee.V.(GMDS): The statistical analysis plan is intended to be a comprehensive and detailed description of the methods and presentation of data analyses proposed for a project, in order to avoid post hoc decisions which may affect the interpretation of the statistical analysis . (GMDS e.V) Furthermore the GMDS states.The statistical analysis plan prevents medical professionals from drowning in technical terms and protects the biometrician from useless data torturing. (Mills JL, NEJM 1993: (329):1196-1199)01May2007 STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE Page14

____________________________________________________________________________________ Johnson and Fitts (20May2007) Workshop #8 (SCT Annual Meeting)

STATISTICAL ANALYSIS PLANS: PRINCIPLES AND PRACTICE Page 9 ____________________________________________________________________________________

SAPDefinition Eachdefinitionisarticulatingthefollowing principles: Elaborationofstudyprotocolhypotheses. Comprehensiveanddetaileddescriptionoftheprospective statisticalmethods toevaluateplannedstudyhypotheses. Proceduresforexecutingtheplannedstatisticalanalyses. Apriori methodstoavoidposthocdecisionswhichmayaffectthe interpretationofthestatisticalanalysis(andRegulatory Acceptance!) Detailsonhowresultswillbepresentedandreported. ObligationsforGoodStatisticalPracticeandStatisticalEthics.01May2007 STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE Page15

HistoricalBasisfortheSAP

01May2007

STATISTICALANALYSISPLANS:PRINCIPLESANDPRACTICE

Page16

____________________________________________________________________