Steps in Conducting a Scholarly Mixed Methods Study
-
Upload
simpatica-pianista -
Category
Documents
-
view
65 -
download
1
description
Transcript of Steps in Conducting a Scholarly Mixed Methods Study
-
University of Nebraska - LincolnDigitalCommons@University of Nebraska - Lincoln
DBER Speaker Series Discipline-Based Education Research Group
11-14-2013
Steps in Conducting a Scholarly Mixed MethodsStudyJohn W. CreswellUniversity of Nebraska - Lincoln, [email protected]
Follow this and additional works at: http://digitalcommons.unl.edu/dberspeakers
This Presentation is brought to you for free and open access by the Discipline-Based Education Research Group at DigitalCommons@University ofNebraska - Lincoln. It has been accepted for inclusion in DBER Speaker Series by an authorized administrator of DigitalCommons@University ofNebraska - Lincoln.
Creswell, John W., "Steps in Conducting a Scholarly Mixed Methods Study" (2013). DBER Speaker Series. Paper 48.http://digitalcommons.unl.edu/dberspeakers/48
http://digitalcommons.unl.edu?utm_source=digitalcommons.unl.edu%2Fdberspeakers%2F48&utm_medium=PDF&utm_campaign=PDFCoverPageshttp://digitalcommons.unl.edu/dberspeakers?utm_source=digitalcommons.unl.edu%2Fdberspeakers%2F48&utm_medium=PDF&utm_campaign=PDFCoverPageshttp://digitalcommons.unl.edu/dber?utm_source=digitalcommons.unl.edu%2Fdberspeakers%2F48&utm_medium=PDF&utm_campaign=PDFCoverPageshttp://digitalcommons.unl.edu/dberspeakers?utm_source=digitalcommons.unl.edu%2Fdberspeakers%2F48&utm_medium=PDF&utm_campaign=PDFCoverPageshttp://digitalcommons.unl.edu/dberspeakers/48?utm_source=digitalcommons.unl.edu%2Fdberspeakers%2F48&utm_medium=PDF&utm_campaign=PDFCoverPages
-
AbstractforDBERGroupDiscussionon20131114Presenter(s),Department(s):JohnCreswellProfessorDepartmentofEducationalPsychologyUniversityofNebraskaLincolnTitle:StepsinConductingaScholarlyMixedMethodsStudyAbstract:
MixedmethodsresearchisarapidlyexpandingmethodologyinthesocialandhumansciencesintheUSandaroundtheworld.InthispresentationIwillfirstdefinemixedmethodsresearch(combiningbothquantitativeandqualitativemethodsofresearch)anddiscusswhatitisandwhatitisnot.ThenIwillreviewabriefhistoryofitsdevelopment,andwhyitisimportanttoday.Iwilldiscussseveralofthescientificdevelopmentsinmixedmethodsthathaveoccurredoverthelasttenyears,suchasthespecificationoftypesofdesigns,theformationofmixedmethodsquestions,andtheuseofinnovativeapproachestojointlydisplayquantitativeandqualitativeresults.Finally,Iwilltalkaboutthefutureofthismethodologywhereitisheadedandsomeimportantworldwidedevelopmentsthathaveencouragedmixedmethodsresearch.
-
StepsinConductingaScholarlyMixedMethodsStudy
JohnW.CreswellUniversityofNebraskaLincoln
-
Numbers PersonalExperiences
HowWeMakeSense
AMixedMethodsApproach
-
ASimpleIdea
Mixedmethodsisaresearchapproach,popularinthesocial,behavioral,andhealthsciences,inwhichresearcherscollect,analyze,andintegratebothquantitativeandqualitativedatainasinglestudyorinasustainedlongtermprogramofinquirytoaddresstheirresearchquestions.
Letsusequantitativeandqualitativedata(orquantitativeresearchandquantitativeresearch)together togainamorecompleteunderstandingofourresearchquestions.
-
5
New NIH funding awards that self-identify as mixed methods (N = 226) (1997-2008)
Source:CRISPdatabase;PlanoClark (2010)
0
10
20
30
40
50
60
70
1996 1998 2000 2002 2004 2006 2008 2010
Year
Nu
mb
er o
f N
ew A
war
ds
POPULARITY FEDERALFUNDING
-
Haines,C.,2011unpublisheddissertation,UniversityofNebraskaLincoln
DISSERTATIONS
-
MixedMethodsSocialScienceArticles
-
MixedMethodsHealthScienceArticles
-
SelectMixedMethodsBooks
-
Jennifer Greene USA- University of Illinois evaluator
John Hunter and Allen Brewer USA -Northwestern and Boston College sociologists
Alan Bryman - England management
John Creswell USA education
Nigel and Jane Fielding sociologist in UK
Jan Morse nursing in Canada
When did mixed methods develop in the social sciences and health/behavioralsciences?(1985-90)
-
Ifyouwalkedintomyoffice,withtheideaofconductingamixedmethodsstudy,Iwouldtakeyouthroughaseriesofquestionstohelpyoudesignyourproject.
Iwilltakethestepsoutoforderofwhatyoutypicallyseeinaresearchproject,butwewillputtheminorderattheend.
-
Question1.Doyouhaveamixedmethodsstudy?(orsimplyaquantitativestudyoraqualitativestudy)
-
WhatIamlookingfor(corecharacteristicsofamixedmethodsstudy):
1. Doyouhaveaquantitativedatabase?(closedended)
2.Doyouhaveaqualitativedatabase?(openended)
3. Doyouplanonbringingthetwodatabasestogether?(integration)
4.Whatprocedures(design)willyouuse?
-
WhatIamNOTlookingfor:
Keepingthedatabasesseparate ThecollectionofBOTHquantitative
andqualitativedata(notcontentanalysis) Multipleformsofquantitativedataormultiple
formsofqualitativedata(multimethod) Onlythetransformationofqualitative
dataintoquantitative(inanalysis) Onlyformative/summativeevaluation Onlyaddinginqualitativedataintoan
experiment Doingmixedmethodsbecauseitistrendy
-
Question2.Doesyourresearchproblemorquestionmeritamixedmethodsstudy?
Theysay,chooseyourmethodbasedonyourproblem.
-
Whenismixedmethodssuitablefora problemorquestion?
Whenqualitativeresearchorquantitativeresearchisinsufficienttofullyunderstandtheproblem
-
Weneedtoexplorebeforeweadministerinstruments
Weneedtoexplainourstatisticalresultsbytalkingtopeople
Weneedtoseeifourquantitativeresultsandourqualitativeresultsmatch
Weneedtoenhanceourexperimentsbytalkingwithpeople
Weneedtodevelopnewinstrumentsbygatheringqualitativedata
Othertypesofproblems,when:
-
Question3:Whatistheoverallintentofyourstudy?
-
Mygoal:
Tolearnwhetheryouarequantitatively orqualitativelyorientedbythewayyoustateyourintent
-
Keywords
in
yo
Quantitative Qualitative
factors explore
determinants generate
relationship develop
causes create
influences meaning
effects personalexperiences
-
1. Willhelpmethinkaboutwhatmixedmethodsdesignyouwilllikelyuse
2. Willhelpmethinkaboutyourdisciplinefieldbackgroundthatyoubringtomixedmethods
3. Willhelpmethinkaboutwhatskillsyouhavetobeginconductingamixedmethodsstudy
YourKeywords:
-
ExamplesofGeneralIntentStatements
ThisarticlereportsonthestudyconductedtounderstandstudentspersistenceintheDistributedDoctoralPrograminEducationalLeadershipinHigherEducation(ELHE)offeredbytheUniversityofNebraskaLincoln(UNL).(Ivankova &Stick,2007)
Weconductedaninterviewbasedstudyofterminallyillindividualswhowerereceivingpalliativecare.(Kutner,Steiner,Corbett,etal.,1999)
tounderstandpatientsexperiencesoftransitionsfromhospitaltoahomelessshelteranddetermineaspectsoftheseexperiencesassociatedwithperceivedqualityofthesetransitions.(Greysen,Allen,Lucas,etal.,2012)
-
Question4:Whatquantitativeandqualitativedatawillyoucollecttoaddressthisintent?
-
Iwillaskyoutolistyourquantitativeandqualitativedatasources:
QuantitativeData QualitativeData
Siteandsample?
Howmany?
Typesofdatatocollect
Specificinstruments/protocols
Typesofscales/questions
-
Iwillaskyoutolistyourquantitativeandqualitativedataanalysissteps:
QuantitativeAnalysis
QualitativeAnalysis
Getting datareadyforanalysis
Preliminaryanalysis
Stepsinanalysis(including statistics,themes)
-
ListMultipleTypesofQuantitativeandQualitativeDatatobeCollectedandAnalyzed
QUANTITATIVERESEARCH Quantitativedata
collection(closedended) Instruments Behavioralchecklists Records
Quantitativedataanalysis Usenumericdata,
Fordescription Forcomparinggroups Forrelatingvariables
QUALITATIVERESEARCH Qualitativedatacollection
(openended) Interviews Observations Documents Audiovisualmaterials
Qualitativedataanalysis Usetextandimagedata,
Forcoding Forthemedevelopment Forrelatingthemes
27
-
AreTheseProceduresRealisticandFeasible?
Time Economically Skills Usefulforstakeholders
-
Question5.Whyareyoucollectingbothquantitativeandqualitativedata?
-
Basicreasonsforusingmixedmethods(calledtherationaleinmixedmethods):
Needdifferent,multipleperspectives,ormorecompleteunderstandings
Needtoconfirmourquantitativemeasureswithqualitativeexperiences
Needtoexplainquantitativeresults Needbettercontextualizedinstruments,measures,or
interventionstoreachcertainpopulations Needtoenhanceourexperiments Needtogathertrenddataandindividualperspectivesfrom
communitymembers Needtoevaluatethesuccessofaprogrambyusinganeeds
assessmentANDatestofthesuccessoftheprogram
-
Questions6.Doyouplanonusingatheory?Doyouplanonadvancingyourphilosophicalstance?
-
32
Frame the study within theory/philosophy:
Paradigm/Worldview (epistemology, ontology, axiology, methodology)
Theoretical lens(e.g., feminist, racial, social science theories)
Methodological approach (e.g., ethnography, experiment, mixed methods)
Methods of data collection (e.g., interviews, checklists, instruments)
Adapted from Crotty M. (1998)
-
Whatisaphilosophy?
Whichphilosophiesaretypicallyusedasthefoundationinmixedmethodsresearch?
Furtherquestionsaboutphilosophy:
-
Pragmatism Transformativeworldview Criticalrealism Dialecticpluralism
Multipleworldviews/paradigms Matchworldviewtodesign Worldviewcomesfromyourscholarly
community
Philosophiestoconsiderinmixedmethods:
-
Whatisatheory?
Howdoyoufindatheory?
Howwillyouuseitinyourmixedmethodsstudy?
Furtherquestionsabouttheory:
-
Question7.Whatmixedmethodsdesignwillyouuse?
-
Parsimonioussetofdesigns:BasicandAdvanced
Basic DesignsConvergent DesignExplanatory Sequential DesignExploratory Sequential Design
Advanced DesignsIntervention DesignTransformative DesignMultiphase Design
BasicDesigns
AdvancedDesigns
-
Basicmixedmethodsdesigns
ConvergentParallelDesign
QuantitativeDataCollectionandAnalysis
DetermineQuantitativeResultstoExplain
QualitativeResults
QualitativeDataCollectionandAnalysis
MergeResultsForComparison
InterpretorExplainConvergence/Divergence
ExplanatorySequentialDesign
QuantitativeDataCollectionandAnalysis
QuantitativeResults
QuantitativeResults
QualitativeDataCollectionandAnalysis
QualitativeResults
InterpretHowQualitativeDataExplainsQuantitativeResults
ExploratorySequentialDesign
QualitativeDataCollectionandAnalysis
QualitativeResults
UseResultstoFormVariables,Instruments,Interventions
QuantitativeDataCollectionandAnalysisBasedonVariables,Instruments,Interventions
QuantitativeResults
InterpretHowQuantitativeResultsProvideNewResults,New,BetterInstruments,andBetterInterventions
-
AdvancedmixedmethodsdesignsInterventionMixedMethodsDesign
SocialJusticeDesign(usinganExplanatorySequentialDesignexample)
MultistageEvaluationDesign(usinganExploratorySequentialDesignexample)
ExperimentalStudy
ExperimentGroup
ControlGroup
Pretest
Pretest
PostTest
PostTest
InterventionQualitativeDataCollection,Analysis,andResults(Exploratory)
QualitativeDataCollection,Analysis,andResults(Convergent)
QualitativeDataCollection,Analysis,andResults(Explanatory)
Theory QuantitativeDataCollectione.g.,survey
QuantitativeResults
QualitativeDataCollectionandAnalysis
QualitativeResults
InterpretHowQualitativeDataExplainsQuantitativeResultsAndCallsforAction
ResearchQuestions
PromoteSocialJustice
FormativeNeedsAssessment(qualitativedatacollection)
Theory/ConceptualFramework(basedonqualitativeresults)
InstrumentDevelopment(basedonquantitativetests)
FormativeProgramAssessment(qualitativedatacollection)
SummativeProgramEvaluation(prepostquantitativetests)
ProgramRevision
SingleProgramObjective
-
Firstselectyourbasicdesign,thenaddifyouhaveanadvanceddesign
Consideryourscholarlycommunity:quantitativeorqualitative?(howtobeginyourstudy)
Consideryourskills:quantitativelystrongorqualitativelystrong(whatwillbeemphasized)
Consideryourresources(collectatonetimeorspaceout)
Considerthecomplexityofthedesign(whatistheeasiest?thehardest?)
Howdoyouchooseyourdesign?
-
AdvancedDesign:Betancourtetal.R24projectexaminingtheuseofafamilystrengthinterventionwithcommunityinputwithrefugeesinBoston
ExploratorySequentialDesign
InterventionDesign
CommunityBasedParticipatoryResearchApproach
-
Question8.Canyoudrawa diagramofyourdesign?
-
Convergent Design Wittink et al. (2006)
Source: based on Wittink et al.(2006)
-
Procedures:Three groups: control group, complianceIntervention group, alliance intervention group group comparisons
Outcome measures: 1) attitudes toward medication 2) adherence to treatment 3) avoidance of relapseDAI measure completed 3 times (pre, post, & follow up)
Products: Numerical item scores Change scores Test statistics
Experimental Methodology
QUAN QUANPre- Intervention Post- &
Follow-up Follow-upMeasure Measures
Procedures: Discuss treatment effectiveness
Discuss themes in context of
interventions and outcomes
Products:Discussion
Procedures: One-on-one semi-
structured interviews exiting the trial, participants from two experimental conditions;
Thematic analysis
Products: Transcripts Themes and quotes
Procedures: One-on-
one semi-structuredinterviews
Thematic analysis
Products: Transcripts Developed intervention
treatment
qual before
intervention
qual after
intervention
Overall results
and interpretation
Flow of the experiment
InterventionDesign(Rogers,Day,Randall,,&Bentall ,2003,studytoimprovethemanagementofantipsychoticmedication)
-
Question9.Canyouwriteagoodpurposestatement(orstudyaim)thatfitsyourdesign?
-
Howtowriteamixedmethodsstudyaim:
Thismixedmethodsstudywilladdress[overall contentaim].Aconvergentparallelmixedmethodsdesignwillbeused,anditisatypeofdesigninwhichqualitativeandquantitativedataarecollectedinparallel,analyzedseparately,andthenmerged.Inthisstudy,[quantitativedata]willbeusedtotestthetheoryof[thetheory]thatpredictsthat[independent variables]will[positively,negatively]influencethe[dependentvariables]for[participants]at[thesite].The[typeofqualitativedata]willexplore[thecentralphenomenon]for[participants]at[thesite].Thereasonforcollectingbothquantitativeandqualitativedataisto[themixingreason].
Intent
Design
DataCollection
Rationale
46
-
Question10.Canyouwriteresearchquestionsforyourmixedmethodsstudy?
-
Quantitativequestionsorhypotheses
Qualitativequestions
Amixedmethodsquestion
Write3typesofquestions:
-
Basicdesignmixedmethodsquestions:
ConvergentDesign
ExplanatoryDesign
ExploratoryDesign
Towhatextentdothequantitativeandqualitativeresultsconverge?
Inwhatwaysdothequalitativedatahelptoexplainthequantitativeresults?
Inwhatwaysdothequantitativeresultsgeneralizethequalitativefindings?
-
Advanceddesignmixedmethodsquestions:
InterventionDesign
TransformativeSocialJusticeDesign
MultistageDesign
Howdothequalitativefindingsprovideanenhancedunderstandingofthequantitativeresults?
Howdothequalitativefindingsprovideanenhancedunderstandingofthequantitativeresultsinordertoexploreinequalities?
Howtothedifferentphasesintheprojecttoaddresstheoverallresearchgoal?
-
Problem Theoryand/orphilosophy Purpose(studyaim) Rationaleforgatheringboth
quantitativeandqualitativedata Researchquestions(quantitative,
qualitative,mixed) Typesofdata/analysis Definitionofmixedmethods Typeofmixedmethodsdesign Diagramofthedesign
Nowplacethesestepsinorderforagoodmixedmethodsproject:
-
StepsinConductingaScholarlyMixedMethodsStudy
JohnW.CreswellUniversityofNebraskaLincoln
University of Nebraska - LincolnDigitalCommons@University of Nebraska - Lincoln11-14-2013
Steps in Conducting a Scholarly Mixed Methods StudyJohn W. Creswell
2013-11-14-Steps_in_Conducting_Mixed_Methods_Research-Creswell2013-11-14-Steps_in_Conducting_Mixed_Methods_ResearchPPT