COMMENTARYQuantitative and Qualitative Research: Beyondthe DebateOmar Gelo & Diana Braakmann & Gerhard BenetkaPublished online: 16 September 2008#Springer Science + Business Media, LLC 2008Abstract Psychologyhasbeenahighlyquantitativefieldsinceitsconceptionasascience. However, a qualitative approach to psychological research has gainedincreasing importance in the last decades, and an enduring debate betweenquantitativeandqualitativeapproacheshasarisen. TherecentlydevelopedMixedMethods Research (MMR) addresses this debate by aiming to integrate quantitativeand qualitative approaches. This article outlines and discusses quantitative,qualitativeandmixedmethodsresearchapproacheswithspecificreferencetotheir(1) philosophical foundations (i.e. basic sets of beliefs that groundinquiry), (2)methodological assumptions (i.e. principles and formal conditions which guidescientific investigation), and (3) research methods (i.e. concrete procedures for datacollection, analysis andinterpretation). We conclude that MMRmayreasonablyovercomethelimitationofpurelyquantitativeandpurelyqualitativeapproachesateach of these levels, providing a fruitful context for a more comprehensivepsychological research.Keywords Research methods.Quantitative.Qualitative.Mixed methods researchPsychological research has relied heavily on experimental and correlationaltechniquestotest theoryusingquantitativedata. Thisisbecausepsychology, likeother behavioural disciplines, has been dominated by a positivist/post-positivistparadigm. However, criticism toward this way of conducting research during the pastfewdecadeshasemerged. Whilequalitativeresearchapproaches(e.g., Silverman2004) have been developed starting fromcompletely different philosophicalassumptions, such as phenomenology and hermeneutics, some quantitativeIntegr Psych Behav (2008) 42:266290DOI 10.1007/s12124-008-9078-3O. Gelo (*): D. BraakmannDepartment of Psychotherapeutic Sciences, Sigmund Freud UniversityVienna, Schnirchgasse 9a,1030 Vienna, Austriae-mail: omar.gelo@sfu.ac.atG. BenetkaDepartment of Psychology, Sigmund Freud UniversityVienna, Schnirchgasse 9a,1030 Vienna, Austriaresearchers (e.g. Michell 1999, 2000; Toomela 2008) have become self-critical abouttheir own research approach. For example, Michell (1999) provided a criticalhistorical overviewoftheconcept ofmeasurement inpsychology, identifyingtwomain issues: (1) most quantitative research is based upon the fact that psychologicalattributes can be measured in a quantitative way rather than upon empiricalinvestigation of the issue; (2) most quantitative researchers adopt a defectivedefinitionofmeasurement, thinkingthat measurement issimplytheassignmentofnumbers to objects and events according to specific rules. In a similar way, Toomela(2008) recently showed how(1) quantitative variables may encode informationambiguously, andhow(2)statistical analysismaynot alwaysallowameaningfultheoretical interpretation, because of ambiguity of information encoded in variables,and because of intrinsic limitation of statistical procedures.Accordingtotheseauthors, thereisafundamental issuewhichhasbeenoftenignored within quantitative research: the issue of the ontology and epistemology ofvariables (Michell 1999; Toomela 2008). Hence the basic concern is whatinformation is encoded in quantitative variables supposed to represent mentalphenomena (ontology of a variable), and how this kind of information may enlightenus about the relationship between these mental phenomena (epistemology of avariable). Toomela(2008) concludes that without aclear understandingof whatinformationis encodedinavariable, it is not possibletomeaningfullyinterpreteventsand their relationship on the basis of any statisticalanalyses.Is this ano-way-outsituation?Dowehavetoabandonquantitativeresearchapproaches? We do not think so. Do we have then to improve and refine the existingquantitative methodologies? We thinkthis wouldbe a more favourable solution.However, we believe that a change of perspective is needed, which should primarilyinvolvethewayresearchisconceived. Qualitativeresearchapproachescouldbeaninterestingsolutioninthisregard.Nonetheless,weclaimthatthereisanevenmoreappropriate alternative, which consists of integrating quantitative and qualitativeresearch approaches.In the present paper we will in the first place introduce the current debate betweenquantitative and qualitative research approaches. Then we will make a step back andreview respectively quantitative and qualitative research approaches in terms of theirspecific paradigmatic postulates, methodological assumptions and research methods.Thereupon we will describe the Mixed Method Research, a relatively recentapproach which combines and integrates qualitative and qualitative research atdifferent levels. Our aimis toshowhowsuchanapproachmayovercome thelimitations of purely quantitative or qualitative approaches, providing a fruitfulcontext for a more comprehensive psychologicalresearch.The Debate Between Quantitative and Qualitative ResearchTo study human beings, psychologists have commonly followed either a quantitative orqualitative approach. Fromanetymological point of view, the former implicatesdetermining how much of an entity there is, while the latter is involved in describing theconstituent propertiesofanentity. Indeed, muchpsychological researchreflectstheessenceof thisdistinction. Agreat deal of quantitativeresearchisconcernedwithIntegr Psych Behav (2008) 42:266290 267counting occurrences, volumes, or the size of the associations between entities, whilequalitative research aims to provide rich or thick (Geertzt 1973) descriptive accountsof the phenomenon under investigation.Quantitativeandqualitativeresearchapproachesclearlydiffer intermsof howdata are collectedandanalyzed. Quantitative researchrequires the reductionofphenomena to numerical values in order to carry out statistical analysis. By contrast,qualitativeresearchinvolvescollectionofdatainanon-numerical form, i.e. texts,pictures, videos, etc. However, quantitative and qualitative approaches also differparticularlyin regard to the aims of scientific investigation as well as theunderlyingparadigmsandmeta-theoretical assumptions. Accordingtoquantitativeapproaches, psychological and social phenomena have anobjective reality. Therelationshipsbetweenthesephenomenaareinvestigatedintermsof generalizablecausal effects, whichinturnallowprediction. Bycontrast, qualitativeapproachesconsider realityassociallyandpsychologicallyconstructed. Theaimof scientificinvestigationistounderstandthebehaviour andthecultureof humansandtheirgroupsfromthepointofviewofthosebeingstudied(Bryman1988,p.46).Anattempt is usually made to understand a small number of participants own frames ofreference or worldviews, rather than trying to test hypotheses on a large sample.Quantitative approaches have always dominated mainstreampsychologicalresearch. Since the conception of psychology as a science in the nineteenthcentury, quantitativeapproacheshaveprevailed. AsstatedbyDanziger (1985) inanalogytoKants categorical imperative, theyhavebecomethemethodologicalimperative. However, since the 1960s various psychologists, especially thosedealingwithsocial phenomena, havebeguntocriticizesuchanapproachtotheinvestigationof thehumannature. Theyhaveproposedanaturalistic, contextual-based and holistic understanding of the human being, which has come to be knownas the qualitative approach. Since this approach has gained ground withinpsychology(seee.g.Smith2003),itsparkedadebateabouttheappropriatenessofeither quantitative or qualitative approaches in psychological research (Patton 1988).Thosetwodiverseapproaches couldjustbeviableoptions; instead, theyhavebecome rather entrenched ideological positions (Todd et al. 2004).The QuantitativeQualitative Debate (QQD) has been sustained by several factorswhichcanbemainlyascribedtotheunderlyingphilosophical andmethodologicalassumptionsandtherelatedresearchmethods(Bryman1984;Krantz1995).Someauthors emphasizetheincompatibilityof quantitativeandqualitativeapproaches.Their basic argument is that the meta-theoretical paradigms underlyingthe twoapproaches are so different that any reconciliationbetween themwould destroy thephilosophical foundations of each (Lincoln and Guba 1985; Noblitt and Hare 1988;Rosenberg1988).Asnotedby Bryman(1984),theQQDisbasedtoa large extenton epistemological issues, and questions relating to research techniques aresystematicallyrelatedtotheseissues. Someotherauthors, though, assumeamorepragmatic position. According to them, it is both possible to subscribe to thephilosophyof oneapproachandemploythemethodsof theother (Reichardt andCook1979;Steckleretal.1992).Recently, thesocalledMixed MethodsResearch(i.e. Tashakkori and Teddlie 2003b) has been developed, which aims to combine andto some extent integrate different methodological and research method perspectivesof both quantitative and qualitative approaches. Following these emergent trends, the268 Integr Psych Behav (2008) 42:266290current QQD can be re-defined with reference to both a methodologically integratedandanempiricallygrounded, practice-orientedset of investigations. Inthis way,controversial philosophical issues maybe seeminglybypassed(Krantz 1995) orcombined, and discussions take place at the point of which research strategy is morelikely to investigate specificphenomena.Quantitative and Qualitative ResearchScientificinvestigationcanbecharacterizedbyaset of philosophical andmeta-theoretical assumptions concerning the nature of reality (ontology), knowledge(epistemology), andtheprinciplesinspiringandgoverningscientificinvestigation(methodology), as well as by technical issues regarding the practical implementationof astudy(researchmethods). Thelatter canbeconsideredasderivingfromtheformer, i.e. the choice of a particular philosophical position and methodology leadsto a preference for a particular research method on the grounds of its appropriatenesswithin that specific philosophical and methodological orientation. While philosoph-ical and meta-theoretical assumptions underlie the worldviews constraining the kindsofquestionswetrytoanswer,andtheprinciplesgoverningourresearchapproach,research methods specify the practical implementation of our scientific investigationin terms of data collection, analysis and interpretation.Themainfeaturescharacterizingquantitativeandqualitativeapproachesmaybedescribed withrespectivereferenceto theirphilosophical foundations, methodolog-icalassumptions, andtotheresearchmethodstheyemploy. Differencesateachofthese levels have contributed to sustain the QQD.Worldviews and Philosophical FoundationsAll research needs a foundation for its inquiry, which is provided by worldviews andscientific paradigms. Worldviews imply how we view and, thus, think about researchandgoabout conductingit. Similarly, scientificparadigmscontainabasicset ofbeliefs or assumptions that guide our inquiries (Guba andLincoln2005). Withreference to quantitative and qualitative research approaches, three main worldviewsmaybeidentified:objectivism(accordingtowhichrealityexistsindependentfromconsciousness), subjectivism(accordingtowhichsubjectiveexperienceis funda-mental to any knowledge process), and constructivism(according to whichknowledgeisaconstructionresultingfromtheinteractionbetweenindividualsandtheir social world)1.Thedifferent worldviewsandparadigmsunderlyingquantitativeandqualitativeapproaches are reflected in different conceptions about the nature of reality(ontology) andknowledge (epistemology). Quantitative paradigms see realityas1Objectivism is often associated with quantitative research approaches and has been articulated at a meta-theoretical and philosophical level in logical positivismand critical rationalism. On the contrary,subjectivismandconstructivismaretypicallyassociatedwithqualitativeinvestigation, andhavebeenexpressedatameta-theoreticalandphilosophicallevel,amongothers, inphenomenology, hermeneuticsand symbolic interactionism.Integr Psych Behav (2008) 42:266290 269singleandtangible, wheretheknowerandtheknownareconsideredasrelativelyseparate and independent. Qualitative paradigms, however, viewreality as amultiple, sociallyandpsychologicallyconstructedphenomenon, wheretheknowerand the known are inextricably connected to each other.Modern social and psychological sciences developed at the end of the 19thcentury. At that time, the natural sciences were established and well known,accompanied by an enthusiastic faith in scientific progress. Social and psychologicalsciencesthus imported thecurrentscientificidealofan axiomatic knowledgetobeexpressed, atbest, inamathematicalform,withgreatemphasisonmeasures, testsand experiments. This approach, which presupposes quantification, has itsfoundationsfromthe perspective of philosophy of sciencein the logicalpositivismof theso-calledfirst Viennacircle, andinthecritical rationalismofKarl Popper in the 1930s (Westermann1987; see also Miller 1994).Qualitative approaches to the study of the human being have developed since the19thcenturyasanalternativetothedominant social andpsychological research.Thesegobacktothephilosophicaltraditionofphenomenology,hermeneutics,andsymbolic interactionism, and reflect the emergent willingness to defend the integrityof human sciences as distinct from the natural sciences. Phenomenology (see Moran2000) deals withthestudyof mental phenomenaas experiencedfromthefirst-personpoint of view(Smith2003). Hermeneutics canbe definedas a specificsystemor methodfor interpretation(Dilthey1989), andinvolves cultivatingtheabilitytounderstandthingsfromsomebodyelsespointofview. Finally, symbolicinteractionism (see Blumer 1969) claims that human beings act toward things on thebasis of attributed meanings, which are constructed within social interaction.Methodological AssumptionsGeneral IssuesThe worldviews and philosophical assumptions described above arereflected in different methodologies. Methodology is the study and logic of researchmethods, and refers to principles governing the research activity; it can be defined asaset ofrules, principlesandformal conditionswhichgroundandguidescientificinquiryinorder toorganizeandincreaseourknowledgeabout phenomena. Morespecifically,methodologyestablisheswhichkindofrelationshipexistsbetweentheresearchers observation, theory, hypothesis and research methods (see nextparagraph).Quantitative and qualitative approaches present different methodologies which, asinthecaseoftheirparadigmaticfoundations, havedeeplycontributedtomaintaintheQQD(seeTable1). Theformer areusuallydescribedtoadopt anomotheticmethodology, whilethelatter adopt anidiographicmethodology. This distinctionwas introduced by Windelband(see Lamiell 1998) inorder todifferentiate thescience of general laws that governgenerality(nomothetic) fromthe science ofspecific events, which describe the particular, the unique, and the individual(idiographic). Nomothetic science (fromthe Greek nomos = law, and thesis =proposition)consistsoftheestablishment, collectionandassimilationoffactswiththe exclusive aim of recognizing and formulating laws that are always and in everycircumstanceimmutableanduniversallyapplicable(tendencytogeneralize). Thischaracterizesthenaturalsciences. Incontrast, idiographicscience(fromtheGreek270 Integr Psych Behav (2008) 42:266290idios=own, private,andgraphein=towrite, todescribe)consistsoftherepresentation of an individual event of singular, temporally limited reality ascompletelyaspossiblewiththe objective ofrecording,and comprehending it initsfactuality(tendencytoindividualize). Thisischaracteristicofhistoricaland humansciences, which thus reveal their nature as sciences of specific events. It is importanttoobservethat thesameobjectsofscientificinvestigationcanbemadesubject ofeither nomothetic or idiographic investigation. There is a very close interplaybetween both approaches: each idiographic science with regard to its generalconcepts must refer back to nomothetic disciplines. On the other hand, every generallawisbasedontheobservationofmanydifferentindividualcases. Thereforeitissuggested to consider both methodologies as the extremes of a continuum.The difference between these two approaches can be best outlined with referencetothedichotomybetweenexplanationandcomprehension. Explanationrepresentsthe establishment of connections between facts through regularities that we observe.Comprehension, by contrast, is the reconstruction of howsomeone else hasestablishedconnectionsbetweenfactsthroughregularitiestheyobserved(Kckeis-Stangl 1980). Quantitative approaches tendtoexplain, i.e. toverifyif observedTable 1 Attributes of quantitative and qualitative methodologiesQuantitative approaches Qualitative approachesNomothetic IdiographicExtensive IntensiveGeneralizing IndividualizingExplanation ComprehensionPrediction InterpretationGeneralization ContextualizationDeduction InductionTheory-driven Data-drivenHypotheses-testing Hypotheses-generatingVerification-oriented (confirmatory) Discovery-oriented (exploratory)Experimental NaturalisticTrue-experiments Case-study (narrative)Quasi-experiments Discourse analysisConversation analysisNon-experimental Focus groupCorrelational Grounded theoryCorrelationalcomparative EthnographicCorrelationalcausalcomparativeEx-post-factoInternal validity Internal validityStatistical conclusion validity Descriptive validityInterpretative validityConstruct validity Explanatory validityCausal validityGeneralizability GeneralizabilityExternal validity TransferabilityIntegr Psych Behav (2008) 42:266290 271phenomena and their systematic relationship confirmthe prediction made by atheory. Qualitative approaches, in turn, tend to comprehend, i.e. aspire to reconstructthepersonalperspectives, experiencesandunderstandingsoftheindividual actors.Thus, whilequantitativeapproaches areusuallydeductiveandtheory-driven(i.e.theyobserve specific phenomena onthe base of specific theories of reference),qualitative ones are inductive and data-driven (i.e. they start from the observation ofphenomenainorder tobuilduptheoriesabout thosephenomena). Inquantitativeapproaches, hypotheses are deductively derived from the theory and have then to befalsified through empirical investigation (confirmatory study). In qualitativeapproaches, however, the development of hypothesis is part of the research processitself, whose aim is to develop an adequate theory according to the observations thathave been made (exploratory study).Research DesignsEach methodology (quantitative vs. qualitative) makes use ofspecificresearchdesigns(seeTable1). Aresearchdesignistheplanof actionsorstructurewhichlinksthephilosophicalfoundationsandthemethodologicalassump-tions of a research approach to its research methods (see next paragraph), in order toprovide credible, accountable and legitimate answers to the research questions.Rigorous research designs are important as they guide the methods decisions thatresearchersmust makeduringtheirstudiesandset thelogicbywhichtheymakeinterpretationsat theendoftheirstudies. Researchdesignswithinthequantitativeapproach include experimental and non-experimental designs. Experimental designsmakecausal inferencesabout therelationshipbetweenanindependent andoneormore dependent variables. They arecharacterizedby the direct manipulationof theindependent variableandbyarigorouscontrol of extraneousvariables2. Inthosesituations where the independent variable cannot be manipulated, a non-experimen-taldesignhastobeimplemented.Theprimaryaimofsuchadesignistodescribethe relationship between two or more variablesof interest3.Contrarytoquantitativeresearchapproacheswhichemployexperimental andnon-experimentaldesignsqualitativeapproachesmakeuseofnaturalistic designs(Lincoln and Guba 1985), whose aim is to study behaviour in natural settings. Thatmeansthatphenomenaofinterestareinvestigatedastheyoccurnaturally,offeringlittle structured context of observations. One fundamental assumption of suchdesigns is that behaviour is best understoodas it occurs inits natural contexts,without external constraints or control. The natural context of observation, instead ofbeing regarded as a source of variability to be controlled, is considered essential for adeeper understanding of the phenomena under investigation.Naturalistic designs include, amongothers (for more details, see DenzinandLincoln 2005;Silverman2004):(a)casestudydesigns,whichinvolveanin-depth,longitudinal examination of a single instance or eventcalled case (e.g. an2Accordingtothedegreeof exertedexperimental control, it is possibletodistinguishbetweentrue-experiments and quasi-experiments (for a detailed description see Polgar and Thomas 2000).3Non-experimental designs includecorrelational designs, correlationalcomparativedesigns, correla-tionalcausalcomparative designs, and ex-post-facto designs.272 Integr Psych Behav (2008) 42:266290organization, an individual, a specific event)in order to gain a sharpenedunderstandingofit4;(b)discourseandconversationanalysisdesigns, whichsharethefocusonlanguageasmediumfor interaction; (c) focusgroupdesigns, whichanalyzetheemergent issuesandthemesfromagroupdiscussionfocusedonaspecific topic5; (d) grounded theory designs, where the field-data is used to generatea grounded theory, that is a set of propositions that pertain to a specific experience,situation, or setting; and (e) ethnographic designs, which enable an in-depthdescription and interpretation of shared patterns of beliefs, expectations, andbehaviours within a cultural or social group.ValidityA final important issue of research methodology is that of validity. Validitycanbegenerallyreferredtoas thelevel of accountabilityandlegitimacythat isstrived through data collection, analysis and interpretation (see Research Methodsinthenext paragraph) (OnwuegbuzieandTeddlie2003). At ageneral level it ispossibletodistinguishbetweeninternalvalidityandgeneralizability(MaxwellandLoomis 2003). These two distinct aspects of validity have been differentlyconceptualized within quantitative and qualitative research approaches (see Table 1).With regard to quantitative research, Cook and Campbell (1979) identified statisticalconclusion validity (i.e. the validity of inferences from the sample to the population),construct validity (i.e. the validity of the theoretical constructs employed), andcausal validity(i.e. thevalidityofthecauseeffect relationshipbetweenobservedvariables) as specific kinds of internal validity. External validity, on the other hand,canbedefinedastheextent towhichtheresultsof astudycanbegeneralizedacrosspopulations, settings, and times (Johnson and Christensen2000; p. 200)6.In relation to qualitative approaches, Maxwell (1992) identified four maincategories of validity: descriptivevalidity(i.e. thevalidityof thedescriptions ofsettingsand events),interpretativevalidity (i.e.thevalidity ofstatementsaboutthemeanings or perspectives held by participants), explanatory validity (i.e. the validityofclaimsabout causal processesandrelationships, includingconstruct validityaswell as causal validity), and generalizability (i.e. the extent to which a researcher cangeneralizetheaccount of aparticular situationor populationtoother individuals,times, setting,or contexts)7.Itis importanttoobservethat causality and causalinferencearecontroversial inqualitative research. Some researchers (e.g. Guba and Lincoln 1989) deny thatcausalityisanappropriateconceptinqualitativeresearch. Someothers(e.g. Sayer2000) argue that causal explanation is relevant also in qualitative research but that itis based on process rather than variance concept of causality (Maxwell andLoomis 2003; p. 255).5The term focus group may also refer to a specific form of data collection (see next paragraph).6For a detailed description of different typologies of validity as well as of validity threats and strategiesfor addressing these threats see Campbell and Stanley 1963 and Cook and Campbell 1979.7Generalizability has also been referred to as transferability (Guba and Lincoln 1989).4Case study designs may also be appropriate to the quantitative approach. In this case it is usual to talkabout single-case research designs (see Hilliard 1993 and Kazdin 1982).Integr Psych Behav (2008) 42:266290 273ResearchMethodsDifferent research designs may be implemented by different research methods.Research methods regard those procedures and techniques involved in datacollection, analysis and interpretation. Collecting and analyzing data are the concretesteps, which allowvalid answers to the research questions. Quantitative andqualitative approaches differ in the research methods they apply (see Table 2). Thesewill be described with reference to data sampling, collection, analysis, andinterpretation(for adetailedaccount seeCreswell 2005). Thesedifferences havealso contributed to the QQD, although to a lesser extent, compared with theparadigmatic foundations.SamplingInquantitativeresearch, theintent of samplingistochooseindividualsthat are representative of a population, so that results can be generalized to it(external validity). Toaccomplishthistask, quantitativeresearchersmayresort toboth probabilistic (i.e. each member of the population has the same probability to beincludedinthesample) andpurposive(i.e. useof somecriterionstoreplacetheprinciple of cancelled random errors) sampling (for a detailed overview see Kemper,Stringfield, and Teddlie 2003). Some of the most adopted strategies of probabilisticTable 2 Attributes of quantitative and qualitative research methodsQuantitative approaches Qualitative approachesSampling SamplingProbabilisticSimple random samplingSystematic random samplingStratified random samplingCluster samplingPurposive PurposiveConvenience sampling Convenience samplingHomogeneous cases samplingExtreme/deviant and Typical case samplingData collection Data collectionPrimary data Primary dataTests or standardized questionnaires Open-ended interviewsStructured interviews Focus groupClosed-ended observational protocols Naturalistic observation protocolsSecondary data Secondary dataOfficial documents Official documentsPersonal documentsData analysis Data analysisDescriptive statistics DescriptionInferential statistics Identification of categories/themesLooking for interconnectedness between categories/themesData interpretation Data interpretationGeneralization ContextualizationPrediction based (theory-driven) Interpretation based (data-driven)Interpretation of theory Personal interpretation274 Integr Psych Behav (2008) 42:266290sampling are the simple randomsampling (i.e. each member of the identifiedpopulationhasanequal chanceof beingincludedinthesample), thesystematicrandomsampling (which involves the selection of each nth unit of the targetpopulationfromarandomlyorderedlist of thepopulation), thestratifiedrandomsampling(whichis obtainedseparatingthe populationintogroups sothat eachelementbelongstoasinglegroup, fromwhichthenarandomsampleisselected),andtheclustersampling(wherearandomsampleofgroupswhicharenaturallyoccurring in the populationis selected). Convenience sampling (whereby elementsare drawn from a subpopulation according to its accessibility and research interests)is a form of purposive sampling usually used within quantitativeresearch designs.Qualitativeapproaches, bycontrast, makeuseof almost exclusivelypurposivesamplingstrategies. Theseallowselectinginformation-richcasestobestudiedindepth (Patton 1990; p. 169). Purposive sampling strategiesinclude,among others:convenience sampling (see above), homogeneous cases sampling (i.e. pickingelements froma subgroup to study in-depth), snowball sampling (i.e. usinginformantstoidentifycasesthatwouldbeusefultoincludeinthestudy),extreme/deviant and typical case sampling (which involve seeking out respectively the mostoutstanding casesin order to learnas much aspossible about the outliersor themost averagecases froma subpopulation)(see Table 2).Data CollectionOnce the sampling is concluded, data has to be collected (see Table 2)(seealsoCreswell 2005, foradetaileddescription). Datamaybecollecteddirectlyfrom the subjects constituting the sample (primary data) or indirectly, e.g. by makinguse of personal and official documents as well as research archives (secondary data).In quantitative research, data has to be collected which are relevant to test theformulatedhypotheses. Data collectionis attainedbyusingtests or standardizedquestionnaires (which assess performances, attitudes, personality, self-perception,etc.), structured interviews (where the interviewer just reads the pre-defined questionsandrecordstheanswersrelatedtooneormoreissuesorphenomenarelevanttotheresearch questions), andclosed-ended observationalprotocols (which allowclassify-ing the behaviour of interest using pre-defined categories8,9). Secondary data may alsobe collected as, for example, referring to official documents (e.g. financial records andcensusdata). Theresultingdataisfinallycodedbyassigningnumericvalues, andsuccessivelyintroducedintoa data matrix, whichwill be usedfor the statisticalanalysis(seenextsection).Itissuggestedtodevelopacodebookthatlistsvariablesnames, their definition, and coding values.Inqualitative research, data has tobe collectedinorder toallowanin-depthunderstanding of the participants perspective. For that reason, qualitative datacollectionproceduresdisplayamuchlower degreeof standardizationcomparedtoquantitative data collection. Qualitative data collection is usually accomplished by using8Whereby the visual data is usually video-recorded in order to allow the subsequent analysis according tothe specific observational protocol used.9See e.g. the Analysis and Treatment of Finger Sucking (Ellingson et al. 2000), which allowsinvestigatingthereinforcementsusefulinmaintainingfingersucking, ortheStrangeSituationProtocol(Ainsworth et al. 1978) for the assessment of attachment in infants (1220 months).Integr Psych Behav (2008) 42:266290 275open-ended interviews (which allows investigating the subjects perspective regardinga pre-defined set of topics10), focus groups (i.e. an in-depth group discussionfocused onone or more specific issues or topics of interest), and naturalisticobservation protocols (which allow the observation of specific events and/orbehaviours of one or more subjects in real-world situations11). Interviews are usuallyaudio-recorded; naturalistic observation protocols end up in an accurate description ofobservedevents andprocesses andinfieldnotes, whichare accounts describingexperiences and observations the researcher has made during the observation. Video-recordings of theobservedbehaviours and/or situations mayhelpinthis process.Qualitative research also makes often use of secondary data, like personal documents(i.e. anything personal written, photographed or recorded for private purposes), officialdocuments (e.g. speeches andvideorecordings of televisionshows andadvertise-ments) andarchivedresearchdata (whichmaye.g. containresults of previouslyconducted qualitative studies). The overall text data obtained in this way must then betranscribed in order to be analyzed (see next section).Data AnalysisData analysis consists of examining the database to address theresearch questions and hypotheses (see Creswell 2005, for a detailed description). Inquantitative research approaches, the researcher analyzes the data in order to test oneor more formulated hypotheses; however, explorative data analysis is also possible.The aim is to find out if the relationships between the observed variables (either of acausal or correlational nature) in one or more groups are statistically significant, thatis, generalizable to the population the sample is drawn from. The choice of astatistical test isbasedonthetypeofquestionsbeingasked(e.g. describetrends,compare groups, or relate variables), the types of scales used to measure thevariables (nominal, ordinal, interval or ratio), and whether the population is normallyor non-normally distributed. Confidence intervals and effect sizes may also be usedto provide further evidence. Quantitative analysis proceeds fromdescriptive toinferential (hypotheses-testing) analysis. Finally, the results of the analysis arepresentedintheformof statementssummarizingthestatistical results. Tables orfigures may also be used.Qualitativedataanalysisiscarriedoutonthepreviouslycollectedtextdata(i.e.transcriptions, memos and field notes) through content or thematic analysis. Contentor thematic analysis is based on the examination of the data for recurrent instances ofsome kind; these instances are then systematically identified across the data set, andgrouped together bymeans of a coding system(Silverman2004). Coding is aprocess of groupingevidence andlabellingportions of text sothat theyreflectincreasingly broader perspectives. The researcher first divides the text to be analyzedinto units (sentences, phrases or passages) and labels them, using terms that shouldcome from exact words of the participant. According to the observed similarities anddifferences between the labelled text units, the researcher groups labels together into11The roles of an observer may vary on a continuum: complete participant, to the participant-as-observer,observer-as-participant, and complete observer (Johnson and Turner 2003).10Someformsofopen-endedinterview(i.e. theinterviewguideapproachandthestandardizedopen-ended interview) correspond to some extent to what is generally known as semi-structured interview.276 Integr Psych Behav (2008) 42:266290themes (or content categories). These emergent themes are then re-labelled, using alanguagecloser tothelanguageof theresearcher andtothetheoryof reference.Finally, the themes (or content categories) are interrelated to each other andabstracted into a set of themes, which will receive new labels. This procedure allowsreachinggraduallyhigher levelsof abstractioninthedescriptionof thedata, andidentifying the constituents of the analyzed texts. The obtained data is thenpresented. Presenting qualitative results essentially involves a discussion of theevidence for the emerged themes and perspectives. The idea is to build a discussionthat persuades the reader that the identified categories and dimensions are effectivelygrounded in the observed data, and not imposed by the researcher. Figures, maps ortables may also be used to represent these results. Table 2 offers a syntheticdescriptionof the mainfeatures of data analysis inquantitative andqualitativeresearch.Data InterpretationDatainterpretationconsists of figuringout what thefindingsmean, andispart oftheoverall effort tomakesenseoftheevidencegathered. Inquantitative research, data interpretation consists of giving a meaning to theobtainedresultswithreferencetothetheorythehypotheseshavebeendevelopedfrom. Thisprocesscanalsobereferredtoasdeductive inference (Tashakkori andTeddlie 2003b). According to whether the design was experimental or non-experimental, conclusionsmaybedrawnconcerningcauseeffect relationshipsorcorrelations between variables in the population the sample was selected from.Theseconclusionsmaythenenabletoconfirm, extendorchallengethetheoryofreference.Inqualitative research, data interpretationis basedona process of inductiveinference(Tashakkori andTeddlie2003b), which refers to a process of creatingmeaningful and consistent explanations, understanding, conceptual frameworks,and/or theories drawing on a systematic observation of phenomena. In theseterms, qualitativedatainterpretationconsistsof givingameaningtotheobtainedresults with reference to the specific and particular context of the study (e.g.settings, participants). This process of contextualization is necessary to addressthe issue of qualitative internal validity (i.e. descriptive, interpretative andexplanatoryvalidity). Inwhichwaymayqualitativeresults(i.e. statementsaboutthe meaning and/or perspectives held by the participants concerning a specificissue) help us in increasing our systematic understanding of the issues underinvestigation? According to the kind of naturalistic research design used, theinterrelatedthemesand/or categorieswhichresult fromtheanalysismaybeusedto comprise a model (as in grounded theory designs), a chronology (as innarrative research designs), or comparisons between groups (as inethnographicdesigns). Aprocess of larger sensemakingshouldthenbeemployedtobroadenthe understandingandthe theoretical perspectives the results maycontribute todevelop. Inthis way, issues of qualitative external validity(transferability) maybe addressed. Contrary to quantitative research, where results interpretation istheory-driven and may lead to a confirmation, extension or questioning of an alreadyexisting theory, qualitative data interpretation aims at developing data-drivenhypothesisandnewtheoretical perspectivesandunderstandingof thephenomenaunder investigation.Integr Psych Behav (2008) 42:266290 277Mixed Methods ResearchWe have shown how quantitative and qualitative approaches are profoundly diverseatdifferentlevels(i.e.philosophicalfoundations,methodologicalassumptions,andresearch methods). Each approach has its strengths and weaknesses, and usually thestrengths of an approach may be considered as the weaknesses of the other approach,andviceversa. Thesedifferenceshavebeenperceivedbytheproponentsofeachapproachinterms of dichotomy, rather thancomplementarity. This has stronglycontributed to sustain a debate between quantitative and qualitative researchapproaches (QQD) over the years, leading to epistemological fragmentation,theoretical insularity, andempirical arbitrariness. However, inthepast decades anewresearchapproachhas beendeveloped, knownas MixedMethods Research(MMR). MMRcanbe definedasa research approach thatcombinesand integratesquantitative and qualitative research approaches. This research approach is, as in thecaseof quantitativeandqualitativeresearchapproaches, characterizedbyspecificphilosophical foundations, methodological assumptions and research methods.Thesewill bedescribedinthefollowingsections (for adetaileddescriptionseeTashakkori and Teddlie 2003a; Creswelland Plano Clark 2007).Worldviews and Philosophical FoundationsAfter aformativeperiodbetweenthe1950sandthe1980s, whichsawtheinitialinterest incombiningquantitativeandqualitativemethodsinastudy(e.g. Sieber1973), aparadigmdebateperiodoccurredbetweenthe1970sandthe1980s. Theprevailingissueofthisperiodwastheopportunityofintegratingthephilosophicalfoundations of quantitative and qualitative research. Some (e.g. Smith andHeshiusius 1986) argued that the underlying paradigms of these two researchapproaches were incompatible (see Smith 1983). In 1988, Bryman (1988) challengedthis argument suggesting howthe two research paradigms could be combined.Althoughthedebateisstill verylively, nowadaysthereisaconsistent agreementabout combining quantitative and qualitative research paradigms. Greene andCaracelli (2003) delineate four meaningful instances in mixing paradigms: (1)thinking dialectically about mixing paradigms, (2) using a new paradigm, (3) beingpragmatic, and(4) puttingsubstantiveunderstandingfirst. Thefirst twoconsiderparadigms essential for guidingresearchinquiry, but propose different solutions(Greene and Caracelli 2003). According to the dialectical stance, all paradigms maybeequallyvaluabletoguidescientificresearch.Forthisreason,researchersshouldintentionally engage in a dialectical way with multiple sets of philosophicalassumptionstowardbetterunderstanding(e.g. Greene2000). Theproponentsofanew paradigm suggest that paradigms may and should evolve in order to incorporatea broader set of beliefs and assumptions, and therefore welcome more diverse sets ofmethods. One example is the commonsense realism by Putnam (1990), according towhichsocialrealityisbothcausalandcontextual.Inthiscase, thecombinationofquantitative and qualitative methods is not only welcomed but actuallyrequired.The third and the fourth instances, by contrast, consider paradigms not primarilyrelevantinguidingresearchinquiry(GreeneandCaracelli2003).Accordingtothepragmatic(orcontext-driven)instance, whatmattersmostistheresponsivenessto278 Integr Psych Behav (2008) 42:266290thedemandsoftheinquirycontext. Pragmatistsareopenbyanallegiancetoanyparadigmthat fitsbest withtheresearchaims(seee.g. Howe1988). Finally, theproponents of the concept-driven instance claim, on the other hand, that conceptualortheoretical congruenceisthemost relevant issueinguidingempirical research.Decisionsconcerningtheresearchprocessaremadenot fortheircongruencewithparticular setsofphilosophical assumptionsbut ratherfor their abilitytoenhanceunderstanding of a particular set of concepts in a particular context (see for exampleCooksy et al. 2001).The philosophical foundations of MMR described above show how this researchapproachallowsfor multipleworldviewsandparadigms. Thismayenableaskingdifferentandmorecomplexquestionsand, consequently, lookingfordifferentandmore complex answers. We suggest that this is the first step to overcome thelimitationsconnectedtothesingleapplicationof either quantitativeor qualitativeapproaches.Methodological AssumptionsGeneral IssuesThe methodology of MMR can be described with reference to whatNewman and Benz (1998) called qualitativequantitative interactive continuum ofresearch. As the name suggests, this model considers an interactive continuum, andnot a dichotomy, between qualitative and quantitative methodologies. This model isbased on a unitary vision of science, according to which quantitative and qualitativemethodologiesmust interact inacontinuouswayinorder toallowresearcherstoanswer different andcomplementaryresearchquestions. Inextendinghis model,Newman and colleagues (Newman et al. 2003) focus on the researchers purpose asevenmorefundamental thantheresearchersquestion. Theyarguethat systemat-ically ordering ones research purposes may accomplish the linkages betweendifferent research questions and the correspondent methodologies, providing afoundation for MMR methodology.Sheddinglight onthedynamicofresearchpurposesisnecessarytounderstandMMRs methodology (Newman et al. 2003). In order to do that, Newman andcolleagues(Newmanet al. 2003) present atypologyof researchpurposes, eachofwhichisgenerallyassociatedwitheitheraquantitativeoraqualitativemethodology.These nine general purposes (and the correspondent methodologies) are categorized asfollows: (1) predictthrough quantitative methodology, (2) add to the knowledge basethroughquantitativemethodology, (3)havea personal,social,institutional, and/ororganizational impactthrough qualitative methodology, (4) measure changethrough quantitative methodology, (5) understand complex phenomenathroughqualitative research, (6) test new ideasthrough quantitative methodology, (7)generatenew ideasthroughqualitativemethodology, (8)informconstituenciesthrough qualitative methodology, and (9) examine the pastthrough qualitativemethodology. It is interesting to observe howeach of these different researchpurposeswith the respective quantitative or qualitative methodologymay flowinto, overlap with, and generate other research purposes. This may be characteristic of asingle- or multiple-studyresearchapproach. The nine researchpurposes outline agestalt showing howquantitative and qualitative methodologies mayrepresent aninteractivecontinuumalongwhicharesearcher mayplanhisstudyoscillatinginaIntegr Psych Behav (2008) 42:266290 279dynamicwaybetweengeneralizationandcontextualization, explanationandunder-standing, deduction and induction, and hypotheses-testing and hypotheses-generating.Research DesignsDifferent research designs in MMRhave been identified. Tashakkoriand Teddlie (2003a) have reported nearly 40 different types of mixed methods designsin the literature. Creswell and colleagues (Creswell et al. 2003) have summarized therange of these classifications. Finally, this summary has been updated, leading to a listof 12 classificationswhich span thepast 15 yearsofscholarlywritings aboutmixedmethodsapproaches(Creswell andPlanoClark2007). Inorder toprovideamoresynthetic, parsimonious andfunctional overviewof the different researchdesignsactually existing in MMR, Creswell and Plano Clark (2007) propose four major mixedmethods designs, each of one with its variants: the triangulation design, the embeddeddesign, theexplanatorydesign, andtheexploratorydesign. Theycanbeallocatedeither in one-phase or two-phase approaches (see Table 3).In one-phase approaches, qualitative and quantitative methods are appliedsimultaneously(forthisreasontheyarealsocalledconcurrent designs)andtothesame sample; this is the case of triangulationdesigns andone-phase embeddeddesigns. In two-phase approaches, the quantitative and qualitative methods areapplied one after the other (for this reason they are also called sequential design) tothe same sample or to different samples in the different stages of the study; this is thecase of explanatory designs, exploratory designs, and two-phase embedded designs.The four main mixed methods research designs are depicted in Fig. 1.The triangulation design (alsocalled convergencetriangulation design) representsthemost andwell-knownapproachtomixingmethods(Creswell et al. 2003). Itspurposeistoobtaindifferent but complementarydataonthesametopic(Morse2003;p.122)(seeFig.1).The underlying idea is that, to best understand a researchproblem, it is necessary to bring together the differing strengths and non-overlappingweaknessesof quantitativemethods(largesamplesize, trends, generalization) withthoseofqualitativemethods(smallN, details, in-depth) (Creswell andPlanoClark2007). Thisisespeciallythecasewhenaresearcherwantstodirectlycompareandcontrast quantitative statistical results with qualitative findings, or to validate or expandquantitative results with qualitative data. In the triangulation design, researchersimplement quantitative and qualitative methods during the same timeframe (one-phaseTable 3 Mixed methods research designs and their variants in one-phase and two-phase approachesOne-phase approach Two-phase approachTriangulation ExplanatoryData transformation model Follow-up explanation modelValidating quantitative data model Participant selection modelMultilevel modelEmbedded ExploratoryEmbedded experimental model Instrument developmentCorrelational model Taxonomy developmentEmbeddedEmbedded experimental model280 Integr Psych Behav (2008) 42:266290design) and with equal weight. It involves the concurrent, but separate, data collectionand analysis (see next paragraph). The two data sets are merged by bringing the resultstogether into one overall or by transforming one data set into the other, and the overallresults are then interpreted.Somevariantsexist (Creswell et al. 2003;Creswell andPlanoClark2007):thedata transformation model, the validating quantitative data model, and the multilevelmodel (see Table 3). The data transformation model is used when a researcher wantstoknowtowhatextentthedifferenttypesofdataconfirmeachother.Afterinitialdatacollection, onedatatypeis transformedintotheother datatype(byeitherquantifyingqualitativefindingsor qualifyingquantitativeresults (Tashakkori andTeddlie 1998; for application see Pagano et al. 2002). Researchers use the validatingquantitative datamodel whenthey wantto validate and expand on the quantitativeOne-phase approach(a) Merge the data:Triangulation design(b) Embed the data:Embedded designTwo-phase approach(a) Connect the data:Explanatory designExploratory design(b) Embed the data:Embedded designorQUANInterpretation ofQUAN (qual) resultsQUALInterpretation of QUAL (quan) resultsqual quanInterpretation ofQUALquanresultsQUAL quanorQUANInterpretation of QUAN (qual) resultsQUALInterpretation ofQUAL (quan) resultsqual quanqualInterpretation ofQUANqual resultsQUANInterpretation ofQUAN + QUALresultsQUANQUALFig. 1 Mixed methods research designsIntegr Psych Behav (2008) 42:266290 281findings from a survey by including a few open-ended qualitative questions (see e.g.example Webb et al. 2002). Finally, in the multilevel model (Tashakkori and Teddlie1998),differentmethods(quantitativeandqualitative)areusedtoaddressdifferentlevelswithinasystem.Thefindingsfromeachlevelarethenmergedtogetherintoone overall interpretation. For example, Elliott and Williams (2002) studied anemployee counselling service using qualitative data at the level of clients,counsellors and directors, and quantitative data for the organizational level.Theembeddeddesignisamixedmethoddesignwhereonedataset providesasupportive, secondary role in a study primarily based on the other data type (Creswellet al. 2003; see Fig. 1). This design is used when researchers need to include qualitativeor quantitativedatatoanswer aresearchquestionwithinalargelyquantitativeorqualitative study. Qualitative data couldbe embedded within a primarilyquantitativemethodology(e.g. anexperimentaldesign), orquantitativedatacouldbeembeddedwithin a primarily qualitative design (i.e. a grounded theory design).Avariant of this researchdesignis theembeddedexperimental model, wherequalitative data is embedded within an experimental design (either a true experimentor a quasi-experiment) (see Table 3). This variant can be used either as a one-phaseor a two-phase approach. For example, in a one-phase approach qualitative data canbeembeddedduringtheinterventionphase, whentheresearcherwantstoconductin-depth investigation of the participants perspective during the process ofintervention. Atwo-phase approach is instead used when the researcher needsqualitative informationbefore the intervention(e.g. inorder tobetter shape theinterventionor toselect participants) or after theintervention(e.g. toexploreindepththe results of the interventionor tofollowuponthe experiences of theparticipants about the intervention). For example, Evans and Hardy (2002a, b)conductedanexperimental studyof goal-settinginterventionfor injuredathletes,followed up by interviewing participants from each of the treatment group to betterinterpret theresults of theexperimental study. Another variant of theembeddeddesignisthecorrelational model, inwhichqualitativedataisembeddedwithinaquantitative design. Researchers conduct a quantitative correlational study, and at thesame time collect qualitative data to help explain the obtained results.The explanatory design is a two-phase mixed methods design. The overallpurposeistoobtainquantitativeresults, andthenexplainorbuildonthemusingadditional qualitative data (Creswell et al. 2003; see Fig. 1). Inanexplanatoryresearch design the researchers start with the collection and analysis of quantitativedata; after that, aqualitativephaseof thestudyisdesignedsothat it follows(orconnects to) the results of the first quantitativephase.Therearetwovariantsoftheexplanatorydesign:thefollow-upexplanationandthe participant selection model (see Table 3). In the follow-up explanation model theresearcher first identifies specific quantitative findings that need additionalexplanation (e.g. significantnon significant, outlier, or surprising results), and thencollect and analyze data from participants that can best help in explaining the results.Intheparticipant selectionmodel quantitativeinformationisusedtoidentifyandpurposefullyselect participantsfor afollow-up, in-depthqualitativestudy. Inthisvariant the focus is primarilyqualitative. For example, MayandEtkina (2002)collected quantitative data to identify students with high and low conceptual learninggains, and then completed an in-depth qualitative comparison between these groups.282 Integr Psych Behav (2008) 42:266290The last mixed method research design is the exploratory design. The aim of thistwo-phasedesignistousetheresultsof themethodappliedfirst (qualitative) tofurther develop or inform the results obtained with the second (quantitative) method(Creswellet al. 2003;seeFig. 1). Thisdesignisusedwhenexplorationofdataisneeded (e.g. measures or instruments are not available, little is known aboutvariables that have to be assessed, lack of guiding theory or framework). Researchersstart with qualitative data in order to explore in depth a phenomenon, and then stepto a second, quantitative phase.This design has two common variants: the instrument development model and thetaxonomy development model. The instrument development model allows developinga quantitative instrument based on qualitative findings. Through a qualitativeinvestigation it is possible to explore the research topic with a few participants. Theseresults are then used to develop items and scales, which will constitute the quantitativesurvey instrument. The taxonomy development model makes use of the initialqualitativephasetoidentifyimportant variables, developtaxonomyorclassificationsystems, elaborate an emergent theory; thereafter, the quantitative phase is used to test orstudy these results in a more detailed way (Tashakkori and Teddlie 1998). This modelallows formulating research questions or hypotheses based on qualitative findings, andtesting them within a quantitative framework (see e.g. Goldenberg et al. 2005).ValidityMixedmethodsresearchers, asquantitativeandqualitativeones,strivefortheaccountabilityandlegitimacyof their researchresults, whichisnecessaryfordrawing valid inferences (see data interpretation in the next paragraph). The issue ofvalidity in MMR is one of the most addressed issues in the literature (e.g. Tashakkoriand Teddlie 2003b).It is possible to distinguish between inference quality and inference transferability.Inferencequalityincorporatesthequantitativeinternal validityandthequalitativetrustworthinessandcredibilityofinterpretation.Itcanbedefinedasthedegreetowhich the interpretations and conclusions made on the basisof the results meet theprofessional standards of rigor, trustworthiness and acceptability as well as thedegreetowhichalternativeplausibleexplanationsfor theobtainedresultscanberuled out (Tashakkori and Teddlie 1998; p. 709). By contrast, inferencetransferabilitysubsumesthequantitativeexternalvalidity(generalizability)aswellas the qualitative transferability. It can be defined as the generalizability orapplicabilityofinferencesobtainedina studytootherindividualsorentities,othersettings or situations, other time periods, or other methods/instruments ofobservation (Tashakkori and Teddlie 1998; p. 710).Specific MMR designs may contribute to enhance inference quality and inferencetransferabilityindifferent ways. Triangulationdesign, for example, mayallowabroader rangeof inferences basedonthemergingof quantitativeandqualitativedatasets. Inanembeddedexperimental design, theoverall validityofthestudyisincreased by qualitatively addressing the process beside the quantitativeinvestigationof theproduct. Inafollow-upexplanatorydesign, thesubsequentqualitativeanalysismayprovideadditional meaningful informationtoexplainthepreviously obtained quantitative results. Finally, in an exploratory design, apreviously conducted qualitative investigation of a topic in order to develop aquestionnaire may lead to more precise and accurate results.Integr Psych Behav (2008) 42:266290 283The specific methodological assumptions of MMR allow to address different andarticulatedresearchquestionsthroughadialecticcombinationbetweenquantitativeand qualitative approaches. According to us, this represents a second essential step toget over the limitations of purely quantitative or qualitative approaches.ResearchMethodsDifferentmixedmethodsresearchdesignsarecharacterizedbyspecificproceduresused for data collection (which includes sampling strategies), analysis, andinterpretation. Thesemaypresent distinct issuesaccordingtowhether concurrent(one-phase) or sequential(two-phase) research designs are implemented.SamplingThe specific sampling strategies for quantitative and qualitative research (seeTable2) shouldbeappliedalsowhenthesetworesearchapproaches areusedincombination. One supplementary issue concerns participant selection: should the sameor different individuals be selected for the quantitative and qualitative sample? In thecase of triangulation, embedded and explanatory designs, researchers should select thesame individuals for both quantitative and qualitative data collection. If an exploratorydesignhastobeimplemented, theindividualsselectedforthefirst qualitativedatacollectionaretypicallynotthesameasthoseselectedforthefollowingquantitativephase. This is because the aim of such a design is to generalize the results to population.Another relevant issue is that of sample size: should the same number of individualsbe sampled respectively for the quantitative and qualitative data collection? Generally,the quantitative sample will be bigger than the qualitative one. An exception may beobserved in the case of triangulation design. In this case, the size of both quantitativeand qualitative samples should be as similar as possible, to avoid that differences insample size are reflected in differences in the two datasets.DataCollectionData collection inMMRcanbe concurrent (asintriangulation andone-phaseembedded designs) or sequential (asinexplanatory,exploratory,andtwo-phase embedded designs) (for a detailed account see Creswell and Plano Clark 2007). Inthecaseofconcurrent datacollection, dataiscollectedduringthesametimeframe,even though independently from each other (see Fig. 1). The collected data may haveequal or unequal weight (as in triangulation design vs. one-phase embedded designs).By contrast, sequential datacollectioninvolvesdifferent stages(seeFig. 1). Thedata is first collected (and then analyzed, see next section) either in a quantitative form(asin explanatory or two-phase embedded designs) or in a qualitative form(as in exploratoryand two-phase embedded designs). Decisions are then made about howthe results (eitherquantitative or qualitative) will be used to influence the following data collection (eitherqualitative or quantitative). Finally, a second and complementary phase of data collection(and analysis, see next section) builds on the first one. Either quantitative or qualitativedatacollectionmaybeweightedmoreheavily. Quantitativedatacollectionismoreweighted in the first phase of follow-up explanatory designs, and in the second phase ofinstrument development exploratory designs; qualitative data collection is moreweighted in the second phase of participantselectionexplanatorydesigns,and in thefirst phaseof taxonomydevelopment exploratorydesigns. Intwo-phaseembeddeddesigns, quantitative data collection is always more weighted than qualitative one.284 Integr Psych Behav (2008) 42:266290Data AnalysisAs in the case of data collection, also data analysis in MMR may beeither concurrent or sequential (for a detailed account see Creswell and Plano Clark2007). The aimof concurrent mixed methods data analysis is to look forconvergences resulting frommerging, or embedding the results fromdifferentdatasets. Concurrent data analysis involves conducting a separate initial analysis foreach of the quantitative and qualitative datasets. After that, the researcher merges orembeds the two datasets, so that a complete picture is developed from both of them(triangulationdesign), orsothatthesupportivedatasetcanreinforceorrefutetheresultsof the first dataset (one-phase embedded design).TwotechniquesareavailableformergingquantitativeandqualitativedatasetsinMMR: data transformation and comparison. Data transformation (see Onwuegbuzie andTeddlie 2003) may allowtransformation of one formof data into the other.Transforming qualitative data into quantitative ones is usually done in studiesinvolving content analysis (see Sandelowsky 2003). This procedure consistsessentially in reducing qualitative codes, themes and/or content categories to numericinformation, counting the occurrence of each previously identified category.Thereafter, a matrix can be developed, which combines the different qualitativecategories with their occurrences. Transforming quantitative data into qualitative oneshas received much less attention in the literature. An example is, however, provided byPunch (1998), where quantitative data were loaded into factors in a factor analysis, andthe factors were then viewed as aggregated units similar to themes.Data can be merged also by comparing the results of quantitative and qualitativedata through a matrix or a discussion. In the first case, for example, it is possible toidentifywithinthetext dataquotes, whichsyntheticallyrepresent thepreviouslyidentified qualitative themes. This information can then be introduced into a matrixtogether with the results of quantitative analysis, allowing a comparison between theresults from the two datasets. A discussion may also be used to compare the data. Inthis case, the quantitative results may be displayed and then discussed with referenceto the obtained qualitative results.The purpose of sequential mixed methods data analysis is to use the results fromthe first data set to inform the results which will be obtained with the second data set.Sequential data analysis therefore involves an initial stage where the first data set isanalyzed following the traditional quantitative (as in explanatory or two-phaseembedded designs) or qualitative (as in exploratory or two-phase embedded designs)procedures ofanalysis (see Table2). The resulting information is then usedto takedecisions concerning the analysis of the second data set.Data InterpretationData interpretation in MMR takes place after the data has beencollectedandanalyzedeither inaconcurrent or sequential way12. InMMR, theprocessofmakingsenseoftheevidencegatheredinvolvesacyclical combination12Thisisthecase ofmixed methodsdesigns, whicharedescribed inthepresent paper.Inmixedmodeldesigns (for on overview see Tashakkori and Teddlie 2003a), by contrast, interpretation takes place aftertheapplicationofeachquantitativeandqualitativestrandofthedesign. Theresearcherhasthentogothrough a process of meta-interpretation. The inferences developed for each strand of the design are thenintegrated. This process is called meta-inference (Tashakkori and Teddlie 2003a).Integr Psych Behav (2008) 42:266290 285between the processes of quantitative deductive inference (theory-driven hypothesistesting, verification oriented) and qualitative inductive inference (data-drivenhypothesisandtheorydevelopment, explorationoriented). Accordingtowhetheraconcurrent or a sequential design has been used,theremay be a different emphasison either the deductive inference, the inductive inference, or both. A major emphasisonquantitativedeductiveinferenceprocesses is characteristicof (a) triangulationdatavalidatingdesign(inorder tofindout towhat extent thequalitativeresultssupportthequantitativeones),(b)embeddedexperimentalandcorrelationaldesign(to find out how the qualitative resultsinform and help to explain the experimentalor correlational results), (c) explanatory follow-up design (to find out howthequalitative results help explain the quantitative ones), and of (d) explanatoryinstrumentdevelopmentdesign(tofindoutwhatitemsandscalesrepresentatbestthe qualitative results).Someother designs arecharacterizedbyanemphasis onqualitativeinductiveinference processes. These are the (a) classical embedded design (to find out how thequalitative results support or disconfirmthe quantitative ones), and the (b)explanatory taxonomy development design (to find out in what ways the quantitativeresultsgeneralize the qualitative ones).Finally, emphasis on both quantitative deductive and qualitative inductiveinferenceprocessesisplacedinthe(a) triangulationconvergencedesign(tofindout to what extent, how, and why the quantitative and qualitative data converge), (b)triangulation data transformation design (to find out to what extent quantitative andqualitativeresultsconfirmeachother), and(c) triangulationmultilevel design(tofind out how quantitative and qualitative results confirm each other at different levelsof observation).Thesedifferentcombinationsofquantitativedeductiveandqualitativeinductiveinference processes allowaddressing in different ways the issues of internal(inferencequality) and external (inferencetransferability) validity in MMR.Theresearchmethodsdescribed aboveprovide thepossibilityofa morereliableandvaliddatacollection, analysisandinterpretation. Convergencesat thelevel ofdata collection and analysis (e.g., quantitative and qualitative data are coherent witheachother, theresultsof quantitativeandqualitativeanalysissupport eachother)mayallowmoreconsistentandmeaningfulinterpretationsoftheresults.Incongru-ities, by contrast, may suggest to refine procedures of data collection and/or analysis,as well as to develop new research questions.MMR: Toward a More Comprehensive Psychological Research?Psychological research, developing out of a positivist perspective, has been aremarkably quantitative field. However, since the first half of the century, qualitativeresearch approaches have been developed within social and psychological research.Thishasledtothedevelopmentofanenduringdebatebetweenthesetwo opposedresearch approaches. The QuantitativeQualitative Debate (QQD) has been sustainedat the level of philosophical foundations (e.g. objectivismvs. subjectivismandconstructivism), research methodologies (e.g. explanation vs. understanding,predictionvs. interpretation, deductionvs. induction), andresearchmethods (big286 Integr Psych Behav (2008) 42:266290vs. small samples, numbersvs. narratives, statistical analysisvs. content analysis,hypothesis testing vs. theory generation).Inorder toovercomethis debate, MixedMethodResearch(MMR) has beenformallydevelopedsincethe1980s. AimoftheMMRapproachistocombineorintegratethetraditionalquantitativeandqualitativeresearchapproachesinordertomaximizetheadvantagesandminimizethedisadvantagesconnectedtothesingleapplication of one of the two approaches. Despite the attempt of integratingquantitative and qualitative research approaches, the QQD is still very lively. This istestified, for example, bytherecent articleof Toomela(2008), whichshows thelimitations of variable psychology for the development of a theory of mind due to (a)the inadequacy of quantitative variables to encode in a reliable and externally validway information about mental phenomena, and (b) the related misleadingconclusions statistical analysis may lead to.The scientific investigationof mindis a verycomplex issue. It requires thedevelopment of theories, whichestablishgeneral laws of functioningand, at thesametime, account fortheidiosyncraticdifferencesthat different individualsmaypresent. Italso requires thereferenceto multiplelevelofanalysis,both atan intra-individual level (e.g. the interconnections between biological and psychologicalstructures and functions, the relationships between motivational, emotional,cognitiveandbehavioural schemes, thedifferent waysof attributingmeaningstosituations andevents) andat aninter-individual level (e.g. the bio-psychosocialadaptation to the environment, the quality of interpersonal relationships withinfamiliar, social and cultural contexts). For these reasons, we believe that thedevelopment of an adequate theory of mind requires the cycling between approacheswhich, striving for integration, avoid dichotomous (either reductionistic orrelativistic) and therefore partial accounts of phenomena.WehavetriedtoshowhowMMRmayprovideauseful context for amorecomprehensive psychological research, of the extent to which it promotes adialecticinteractionof different perspectivesat different levels. At aphilosoph-ical level, MMRacknowledgesthenecessityof eventuallyreferringtomultipleworldviews and paradigms. This may help in asking more complex questionsfromdifferent perspectives, whichmayinturnallowseekingdifferent andmorecomplexanswers. At thelevel ofmethodology, MMRovercomesthedichotomybetween nomothetic and idiographic methodologies which, on the contrary,shouldbelocatedonaninteractivecontinuum. Inthisway, acyclical dynamiccanbeestablishedbetweengeneralizationandcontextualization, explanationandunderstanding, deductionandinduction, andhypothesis testingandhypothesisgeneration. Finally, at the level of research methods, MMR enables theintegration of data collection and analysis (either concurrent or sequential)which, inturn, mayallow(a) overcomingthetraditional limitationsconcerningboththeinformationencodedinquantitativevariables(seeToomela2008),andthe meaning contained in qualitative accounts, and (b) transcending the rigiddichotomy existing between deductive and inductive inferences, thus leading to anincreasedaccuracyandmeaningfulnessofdatainterpretation. Webelievethat inthiswayit will bepossibletoovercomethelimitationsofpurelyquantitativeorqualitative approaches, providing a fruitful context for a more comprehensivepsychologicalresearch.Integr Psych Behav (2008) 42:266290 287ReferencesAinsworth, M. D. S., Blehar, M. C., Waters, E., &Wall, S. (1978). Patterns of attachment: apsychological study of the strange situation. Hillsdale, NJ: Erlbaum.Blumer, H. (1969). Symbolic interactionism: Perspective and method. Berkeley: University of California Press.Bryman, A. (1984). Quantity and quality in social research. London: Unwin Hyman.Bryman, A. (1988). Quantity and quality in social research. London: Routledge.Campbell, D. T., &Stanley, J. C. (1963). Experimentalandquasi-experimentaldesignsforresearchonteaching. InN. L. Gage(Ed.), Handbookof researchonteaching(pp. 171246). Chigago: RandMcNally.Cook, T. D., &Campbell, D. T. (1979). Quasi experimentation: Designsandanalysisissuesforfieldsettings. Boston: Houghton Miffli.Cooksy, L. J., Gill, P., & Kelly, P. A. (2001). The program logic model as an integrative framework for amultimethodevaluation. EvaluationandProgramPlanning, 24, 119128. doi:10.1016/S0149-7189(01)00003-9.Creswell, J. W. (2005). Educational research: Planning, conducting, andevaluatingquantitativeandqualitative research (2nd ed.). Upper Saddle River, NJ: Pearson Education.Creswell, J. W., &PlanoClark, V. L. (2007). Designingandconducting. Mixedmethods research.Thousand Oaks. CA: Sage.Creswell, J. W., PlanoClark, V. L., Gutmann, M., &Hanson, W. (2003). Advancedmixedmethodsresearchdesigns. InA.Tashakkori, &C.Teddlie(Eds.),Handbook ofmixedmethodsinsocialandbehavioral research (pp. 209204). Thousand Oaks, CA: Sage.Danziger, K. (1985). The methodological imperative in psychology. Philosophy of the Social Sciences, 15,113. doi:10.1177/004839318501500101.Denzin, N. K., &Lincoln, S. L. (Eds.). (2005). TheSagehandbookof qualitativeresearch(3rded).Thousand Oaks, CA: Sage.Dilthey,W.(1989). Introductiontothehumansciences:Anattempttolayafoundationforthestudyofsocietyandhistory. InR. A. Makkreel, &F. Rodi(Eds.), Selectedworks. Princeton, NJ:PrincetonUniversity Press.Ellingson, S. A., Miltenberger, R. G., Stricker, J. M., Garlinghouse, M. A., Roberts, J., &Rapp, J. T.(2000). Analysisand treatmentof finger sucking. Journal ofAppliedBehavior Analysis,33,4152.doi:10.1901/jaba.2000.33-41.Elliott, M. S., & Williams, D. I. (2002). A qualitative evaluation of an employee counselling service fromthe perspective of client, counsellor and organization. Counselling Psychology Quarterly, 15(2), 201208. doi:10.1080/09515070210128991.Evans, L., & Hardy, L. (2002a). Injury rehabilitation: a goal-setting intervention study. Research Quarterlyfor Exercise and Sport, 73, 310319.Evans,L.,&Hardy,L.(2002b).Injuryrehabilitation:aqualitativefollow-upstudy.ResearchQuarterlyfor Exercise and Sport, 73, 320329.Geertz, C. (1973). The interpretation of cultures: Selected essays. New York: Basic Books.Goldenberg, C., Gallimore, R., &Reese, L. (2005). UsingmixedmethodstoexploreLatinochildrensliteracy development. In T. S. Weisner (Ed.), Discovering successful pathways in childrensdevelopment: Mixedmethods inthe study of childhoodandfamily life (pp. 283303). Chicago:Chicago University Press.Greene, J. C. (2000). Challengesinpracticingdeliberativedemocraticevaluation. InK. R. Ryan, &L.DeStefano (Eds.), Evaluation as a democratic process: Promoting inclusion, dialogue anddeliberation New Directions for Evaluation, No. 85 (pp. 1326). San Francisco: Jossey-Bass.Greene, J. C., &Caracelli, V. J. (2003). Makingparadigmaticsenseofmixedmethodspractice. InA.Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp.91110). Thousand Oaks, CA: Sage.Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park, CA: Sage.Guba, E. G., & Lincoln, Y. S. (2005). Paradigm controversies, contradictions, and emerging confluences.In N. K. Denzin, & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (pp. 191215, 3rded.). Thousand Oaks, CA: Sage.Hilliard, R. B. (1993). Single-case methodology in psychotherapy process and outcome research. Journalof Consulting and Clinical Psychology, 61, 373380. doi:10.1037/0022-006X.61.3.373.Howe, K. R. (1988). Against the quantitativequalitative incompatibilitythesis or dogmas die hard.Educational Researcher, 17(8), 1016.288 Integr Psych Behav (2008) 42:266290Johnson, R. B., &Christensen, L. B. (2000). Educational research: Quantitative and qualitativeapproaches. Boston: Allyn and Bacon.Johnson, R. B., &Turner, L. A. (2003). Datacollectionstrategiesinmixedmethodsresearch. InA.Tashakkori, & C. Teddlie (Eds.), Handbook on mixed methods in the behavioral and social sciences(pp. 297320). Thousand Oaks, CA: Sage.Kazdin, A. E. (1982). Single-case research designs: Methods for clinical and applied settings. New York:Oxford University Press.Kemper, E., Stringfield, S., &Teddlie, C. (2003). Mixedmethodssamplingstrategiesinsocial scienceresearch. In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioralresearch (pp. 273296). Thousand Oaks, CA: Sage.Kckeis-Stangl, E. (1980). Interpretative Methoden kontrollierten Fremdverstehens. In K. Hurrelmann, &D. Ulrich (Eds.), Handbuch der Sozialisationsforschung. Weinheim: Beltz.Krantz, D. L. (1995). Sustainingversus resolvingthequantitativequalitativedebate. EvaluationandProgram Planning, 18, 8996. doi:10.1016/0149-7189(94)00052-Y.Lamiell, J. T. (1998). Nomothetic and idiographic: Contrasting Windelbands understanding withcontemporary usage. Theory & Psychology, 8(1), 2338.Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.Maxwell, J.A.(1992). Understandingand validityinqualitative research. HarvardEducationalReview,62(3), 279300.Maxwell, J. A., &Loomis, D. M. (2003). Mixed methods design: An alternative approach. In A.Tashakkori, &C. Teddlie(Eds.), Handbookof mixedmethodsinsocial andbehavioral research.Thousand Oaks, CA: Sage.May, D. B., & Etkina, E. (2002). College physics students epistemological self-reflection and its relationshipto conceptual learning. American Journal of Physics, 70(12), 12491258. doi:10.1119/1.1503377.Michell, J. (1999). Measurement in psychology: A critical history of a methodological concept.Cambridge: Cambridge University Press.Michell, J. (2000). Normal science, pathological science and psychometrics. Theory & Psychology, 10(5),639667. doi:10.1177/0959354300105004.Miller, D. (1994). Critical rationalism. A restatement and defence. Chicago and La Salle: Open Court.Moran, D. (2000). Introduction to phenomenology. London: Routledge.Morse, J. M. (2003). Principles of mixed methods and multimethod research design. In A. Tashakkori, &C. Teddlie(Eds.), Handbookof mixedmethodsinsocial andbehavioral research(pp. 189208).Thousand Oaks, CA: Sage.Newman, I., &Benz, C. R. (1998). Qualitativequantitative research methodology: Exploring theinteractive continuum. Carbondale: Southern Illinois University Press.Newman, I., Ridenour, C. S., Newman, C., & DeMarco, G. M. P. (2003). A typology of research purposeand its relationship to mixed methods. In N. K. Denzin, & Y. S. Lincoln (Eds.), The Sage handbook ofqualitative research (pp. 167188, 3rd ed.). Thousand Oaks, CA: Sage.Noblitt, G. W., & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies. Newbury Park,CA: Sage.Onwuegbuzie, A. J., & Teddlie, C. (2003). A framework for analyzing data in mixed methods research. InA.Tashakkori, &C.Teddlie(Eds.),Handbook ofmixedmethodsinsocialandbehavioral research(pp. 351383). Thousand Oaks, CA: Sage.Pagano,M.E.,Hirsch,B.J.,Deutsch,N.L.,&McAdams,D.P.(2002).Thetransmissionofvaluestoschool-age and young adult offspring: race and gender differences inparenting. Journal of FeministFamily Therapy, 14, 1336. doi:10.1300/J086v14n03_02.Patton,M. Q. (1988).Paradigmsandpragmatism. InD. M.Fetterman(Ed.), Qualitativeapproachestoevaluation in education: The silent scientific revolution (pp. 116137). New York: Praeger.Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage.Polgar, S., &Thomas, S. (2000). Introductiontoresearchinthehealthsciences(4thed.). ChurchillLivingstone: Edinburgh.Punch,K. F. (1998). Introductiontosocialresearch:Quantitativeandqualitativeapproaches. London:Sage.Putnam, H. (1990). Realism with a human face. Cambridge: Harvard University Press.Reichardt, C. S., & Cook, T. D. (1979). Beyond qualitative versus quantitative methods. In T. D. Cook, &C. S. Reichardt (Eds.), Qualitative andquantitative methods inevaluationresearch(pp. 732).Beverly Hills, CA: Sage.Rosenberg, A. (1988). Philosophy of social science. Boulder, CO: Westview.Integr Psych Behav (2008) 42:266290 289Sandelowsky, M. (2003). Tables or tableux? The challenges of writing and reading mixed methods studies.In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research(pp. 321350). Thousand Oaks, CA: Sage.Sayer, A. (2000). Realism and social science. Thousand Oaks, CA: Sage.Sieber, S. D. (1973). The integration of field work and survey methods. American Journal ofSociology,78, 13351359. doi:10.1086/225467.Silverman, D. (2004). Qualitative research: Theory, method and practice (2nd ed.). London: Sage.Smith, J. K. (1983). Quantitative versus qualitative research: an attempt to clarify the issue. EducationalResearcher, 12(3), 613.Smith, J. A. (Ed.). (2003). Qualitative psychology: Apractical guide toresearchmethods.(2nded).London: Sage.Smith, J. K., &Heshiusius, L. (1986). Closingdowntheconversation: theendof the quantitativequalitative debate among educational researchers. Educational Researcher, 15(4), 412.Steckler, A., McLeroy, K. R., Goodman, R. M., BIrd, S. T., & McCormick, L. (1992). Toward integratingqualitative and quantitative methods: An introduction. Health Education Quarterly, 19(1), 18.Tashakkori, A., &Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitativeapproaches. Thousand Oaks, CA: Sage.Tashakkori, A., &Teddlie, C. (2003a). Thepast andthefutureofmixedmethodsresearch: Fromdatatriangulationtomixedmodel designs. InA. Tashakkori, &C. Teddlie(Eds.), Handbookof mixedmethods in social and behavioral research (pp. 671701). Thousand Oaks, CA: Sage.Tashakkori, A., &Teddlie, C. (Eds.). (2003b). Handbookof mixedmethodsinsocial andbehavioralresearch. Thousand Oaks, CA: Sage.Todd, Z., Nerlich, B., &McKeown, (2004). Introduction. InZ. Todd, B. Nerlich, S. McKeown, &D.Clarke (Eds.), Mixing methods in psychology: The integration of qualitative and quantitative methodsin theory and practice (pp. 316). Hove & New York: Psychology Press.Toomela, A. (2008). Variables in psychology: a critique of quantitative psychology. IntegrativePsychological & Behavioral Science, 42, 3. doi:10.1007/s12124-008-9059-6.Webb, D. A., Sweet, D., & Pretty, I. A. (2002). The emotional and psychological impact of mass casualtyincidents on forensic odontologists. Journal of Forensic Sciences, 47(3), 539541.Westermann, R. (1987). WissenschaftstheoretischeGrundlagender experimentellenPsychologie. InG.Lger (Ed.), Allgemeine experimentelle psychologie (pp. 542). Stuttgart: Fischer.Dr. Omar Gelo is Assistant Professor in the department of Psychotherapeutic Sciences and Co-coordinator of the Doctoral Program in Psychotherapeutic Sciences for foreign students at Sigmund FreudUniversity,Vienna.Hisresearchinterestsinthefieldofpsychotherapyresearchconcernthetherapeuticprocess, with particular relevance of metaphorical language, emotional-cognitive regulation, and theapplication of dynamic systems theory to the study of psychotherapy. He is moreover interested in linkingprocess and outcome in different psychotherapeutic orientations.Dr. Diana Braakmann is Assistant Professor in the department of Psychotherapeutic Sciences atSigmund Freud University, Vienna. She is psychologist and behaviour therapist with a specific training indialectic behaviour therapy. Her psychotherapeutic work during the last years was concentrated on treatingBorderlinePersonalityDisorder andPosttraumaticStress Disease. Her researchinterests focus onthephenomenon of dissociation as well as the connection between process and outcome variables inpsychotherapy.Prof. GerhardBenetkastudiedpsychology, history, sociology, andphilosophyat the UniversityofVienna, obtaining his Master degree in Psychology in 1989, PhD in Psychology in 1994, and habilitationof Psychology in 1998 at the University of Vienna. He is now Prof. of Psychology and Head of Institute ofPsychology at the Sigmund Freud University, Vienna. His research interests focus on history ofpsychology and psychoanalysis.290 Integr Psych Behav (2008) 42:266290